richardbiely/gaia-ecs

A simple and powerful entity component system (ECS) written in C++17

Hi everyone :)

I am a former senior programmer at Bohemia Interactive, where I was responsible for the Arma series. My main focus was on behind-the-scenes and low-level systems, optimization, networking, and security. Before that, I worked at CGC a.s as an integrator of various business-level security devices into a custom integrated security system SBI.

I am also the author of Gaia-ECS, a powerful entity component system written in C++. You can use the framework to elevate any project, big or small, in the gaming or business applications industry. Built on the principles of data-oriented design, Gaia-ECS aims to provide a simple and streamlined framework that allows you to realize your ideas while keeping your projects easy to maintain and performant. You can also use it as a great learning experience.

At some point, I became interested in voxel, which led me to create Voxe(lmetric) and a few more currently private projects.

Thank you to everybody who has supported my projects and their communities. Your help or advice is greatly appreciated and can influence their future. You rock!

gaia-ecs's Issues

Optimized threadpool

Support for multithreading on ECS level

Threadpool is already in place, we now need to make use of it in queries.

Current single-threaded model looks like this:

// Define a query
ecs::Query q = w.CreateQuery().All<Position, const Velocity>();

// Iterate
q.ForEach([](ecs::Iterator iter) {
  auto p = iter.ViewRW<Position>();
  auto v = iter.View<Velocity>();

  for (auto i: iter) {
    p[i].x += v[i].x * dt;
    p[i].y += v[i].y * dt;
    p[i].z += v[i].z * dt;
  }
});

We need to have the ability to run queries on multiple threads.

Currently proposed method is having:

// Run the query on some thread A
auto jobHandle = q.Schedule().ForEach([](ecs::Iterator iter) {
  ...
});
....
// Run the query on many threads in parallel
auto jobHandle = q.ScheduleParallel().ForEach([](ecs::Iterator iter) {
  ...
});

ScheduleParallel should optionally allow to define when the split happens, e.g. in term on entities.
This number can be higher than chunk capacity in which case the system fits in as many chunks
into the job as possible until the split condition is satisfied:

// Do at least 100 entities per job
auto jobHandle = q.ScheduleParallel(100).ForEach([](ecs::Iterator iter) {
  ...
});

We might consider sorting chunks by their entity count before scheduling them so it is easier to handle this when GAIA_AVOID_CHUNK_FRAGMENTATION is 0.
Otherwise, we might end up scheduling very different amount of work per thread. E.g. for split 100, and chunks
with sizes 1, 100, 1, 20, 100, 20, 50 we would schedule 3 jobs: [1, 100], [1, 20, 100], [20, 50]. With sorting it would be 100, 100, [50, 20, 20, 1, 1]. In case 1 we can see bigger amount of entities than necessary are processed. Also we jump from chunk to chunk more often.
Anyway, it is yet to be seen if this makes any sense. Maybe the sorting slows things down way too much (imagine what happens with thousands of chunks) and even if we did it on entity creation/deletion it might make things unnecessarily slower.

Scenes - world serialization

Add support for prefabs

Create a tracy-like editor for the library that would allow realtime view of everything

An editor that would give one an overview of worlds created by the framework (number of entities, chunk fragmentation, systems running, etc). Could exist either as an app or web page.
This would catapult the usability of the project to stratosphere.

Runtime types - ability to define components in runtime (useful mostly for scripting)

Improve query rules system

Query logic is currently hard-coded and supports matching of All, Any, None entities/components.
This by itself would be okay, however, it needs to be more general-purpose so we can support advanced features necessary for handling entity relationships.

auto carnivore = w.add();
auto herbivore = w.add();
auto wolf = w.add();
auto rabbit = w.add();
auto hare = w.add();
auto eats = w.add();

w.pair(carnivore, eats, herbivore);
w.pair(rabbit, Is, herbivore);
w.pair(wolf, Is, carnivore);

auto q0 = w.query().add({ecs::Pair(eats, esc::Any)});
q0.each( { /* run for everything that eats something */ });

....
etc

Doxygen or alike for docs auto-generation

Improve how systems are defined and organized

Current version:
System is a virtual class with overridden member functions. Systems are managed by SystemManager. They are not connected. On the outside this looks very similar to what DOTS does.

class PositionSystem final: public ecs::System {
  static constexpr float dt = 0.01f;
	ecs::Query m_q;

public:
	void OnCreated() override {
		m_q = world().query().all<Position&, Velocity>();
	}

	void OnUpdate() override {
		m_q.each([](Position& p, const Velocity& v) {
			p.x += v.x * dt;
			p.y += v.y * dt;
			p.z += v.z * dt;
		});
	}
};

ecs::SystemManager sm(w);
sm.add<PositionSystem>();
sm.add<OtherSystem>();
...
sm.update(); // run all systems once

Proposed version:
Systems are entities with a System component attached. Maybe even inheriting from Query (so they are cached). This gives us a lot of flexibility. Enabling the system = enabling the entity. Systems can be queried (e.g. disable all systems that do this-and-that easily).
Organized into stages. Stages can be user defined, probably implemented as components. In order for the grouping to work correctly, relationship support would come handy.

auto s = w.system()
        .stage<LateUpdate>() // optional, if not used, the system goes to the default stage
	.all<Position&, Velocity>()
	.any<Rotation>()
	.OnCreated([](ecs::Query& q) { // event when created
	})
	.OnStopped([](ecs::Query& q) { // event when stopped
		...
	})
       // other events
        ...
       // function to run every update (equivalent to ecs::query::each)
	.OnUpdate([](Position& p, const Velocity& v) {
		p.x += v.x * dt;
		p.y += v.y * dt;
		p.z += v.z * dt;
	})
	.commit();

Necessary:

• support for entity naming
• entity sorting/grouping
• entity relationship support

Add support for data relationships

Current roguelike example does something like this:

enum class ItemType : uint8_t { Poison, Potion, Arrow };

struct Item {
  ItemType type;
};

poison = world.CreateEntity();
world.AddComponent<Item>(poison, {ItemType::Poison});
potion = world.CreateEntity();
world.AddComponent<Item>(potion, {ItemType:: Potion});
arrow = world.CreateEntity();
world.AddComponent<Item>(arrow, {ItemType:: Arrow});

auto q = world.CreateQuery().All<const Item>();
q.ForEach([](Item item){
   if (item.type == ItemType::Poison) {
     // the item is poisonous
   } else if (item.type == ItemType::Potion) {
     // potion here
   } else if (item.type == ItemType::Arrow) {
     // arrow
   }
});

Each item is identified by an enum. While this might is be necessarily bad, it is not scalable.
What if we want to query all potions? Given the code above, we'd have to check each item:

containers::darray_ext<Item, 128> potions;
auto q = world.CreateQuery().All<const Item>();
q.ForEach([&](Item item){
   if (item.type != ItemType::Potion)
    continue;
   potions.push_back(item);
});
// do something with potions now
// ....

Another solution would be creating a tag for each item type:

struct Poison{};
struct Potion{};
struct Arrow{};
struct Item {};
...
poison = world.CreateEntity();
world.AddComponent<Item>(poison);
world.AddComponent<Poison>(poison);
potion = world.CreateEntity();
world.AddComponent<Item>(potion);
world.AddComponent<Potion>(potion);
arrow = world.CreateEntity();
world.AddComponent<Item>(arrow);
world.AddComponent<Arrow>(arrow);
...
q.ForEach([&](Potion potion){
  // ... do something with potions
}

But isn't this too much code? What if we could tell that each of those 3 specific item types actually are items?

struct Poison{};
struct Potion{};
struct Arrow{};
struct Item {};
...
world.Make<Poison>.Is<Item>();
world.Make<Potion>.Is<Item>();
world.Make<Arrow>.Is<Item>();

poison = world.CreateEntity();
world.AddComponent<Poison>(poison);
potion = world.CreateEntity();
world.AddComponent<Potion>(potion);
arrow = world.CreateEntity();
world.AddComponent<Arrow>(arrow);

auto q1 = world.CreateQuery().All<const Item>();
q1.ForEach([](Entity e){
   // ... all these entities are Items by default.
   // This covers Poison, Potion and Arrow as well! 
});

auto q2 = world.CreateQuery().All<const Potion>();
q2.ForEach([&](Potion potion){
  // ... do something with potions
}

This would give us something very powerful. Mostly when combined with new query traversal features.
This example is extremely basic so it might not fully describe the point, though.

Cleanup chunk clean system

When the last entity on chunk is deleted, its lifetime countdown starts. Once the counter reaches zero (currently deduced by one every world::update()) the chunk is added to the removal list.
This list is updated from the Chunk class while the world manages the state. However, all state data is stored in the Chunk (ChunkHeader) that doesn't even need to know.

This might need a cleanup. Lifetime data should probably move to where the lifetime is managed and the way the list is updated should probably also be changed form a hard-coded list to some sort of observer the world can register to.

On the other hand, chunk storage is basically for free because all the information is encoded within the header and is very small.

Nevertheless, evaluate.

Don't restrict entity-only chunks with capacity of MAX_CHUNK_ENTITIES

Single header missing code in v0.8.3

The implementation of void join_thread(uint32_t workerIdx) missing code for auto& t = m_workers[i];. i is undeclared.

Add more allocators

Currently, all allocations (save for chunks) are done using the global new operator and depend purely on the system.

Specific allocators are needed because:

• they give complete knowledge over allocations done by the library
  • we already can sort of do this via GAIA_PROF_ALLOC, GAIA_PROF_ALLOC2 but it only works with Tracy and we would need to profile the program from the start which is a bummer
• help manage memory better and help performance, also gives users to swap this for their own impementation (e.g. when used with some engine)