An Introduction to Data Oriented Design with Rust

Whilst it’s impressive you’ve managed to hold an argument with yourself I think it’s more the case that anything as broad as a programming paradigm will naturally hold multiple approaches. Is immutability a defining feature of FP? Let the battles commence.

Likewise critics might focus heavily on inheritance or some other feature of OOP that is easy to critique.

And whilst I’m not particularly interested in defending OOP I think describing it as uniquely plagued with bad code is not something anyone should just take axiomatically.

This is only "bad OOP" if you need to buy 10K books at once and it is actually too slow and there is a significant amount of fat to trim (if you need to call a web service once for every book, incoherent memory access is likely to be negligible).

Otherwise it's obvious, cheap to write, easier to get right than more complicated approaches, and good enough. True OOP is OOP that meets goals.

> the Rust game dev community is overly fixated on ECS.

Not just Rust. The game dev community everywhere is infatuated with ECS.

It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your performance problems are related to caching), so you see a lot of game devs slavishly applying it to their code in hopes that the "go as fast as a AAA game" Gods will land on their runway and deliver the goods.

Every time I see a new ECS framework in JavaScript, I die a little on the inside.

Rust lends itself to ECS a lot more than it lends itself to behavior hierarchies using inheritance. That's an over simplification, but a useful one.

I’d agree to a point but it really crosses the whole gamut of hobby game engine development. It’s weirdly self reinforcing even in the face of more interesting architectural choices like DOOM Eternal’s.

Basically it boils down to making use of Objective-C protocols or multiple inheritance, with data separated from behavior in regards to implementation.

So a 1986 concept rediscovered by game developers and cargo culted ever since, although it has its roots on the OOP they love to hate.

That's okay, now that you're familiar with the term, you'll end up noticing it a lot more (or maybe it's just that articles that happen to lend themselves to it showing up in their comments look slightly more appealing now?).

NP. You may also want to check out Norvig's Paradigms of AI Programming if this interests you. A lot of that text involves creating data-driven programs, and some of the benefits (like creating a grammar lets you recognize sentences that satisfy it, or with little extra effort generate sentences from it).

It seems likely to me that Rust will use ECS queries as the idiomatic way to write struct-of-arrays code.

In archetype-based ECS libraries (which the ecosystem seems to be converging on or near) you write a query asking for a particular subset of "component" types, where each type is stored in its own array(s), and then for each iteration of the query you get back one reference to each component type.

As a result, all the extra zipping (which I'm assuming is what people find less readable about the article's example) is handled in the query implementation, and you get this sort of hybrid between the two, syntactically speaking:

    for (location, velocity, acceleration) in players.query::<(Location, Velocity, Acceleration)>() {
        *location = (
            location.0 + velocity.0,
            location.1 + velocity.1,
        );
        *velocity = (
            velocity.0 + acceleration.0,
            velocity.1 + acceleration.1,
        );
    }

> The interesting thing to me from these patterns is the possibility of using proc_macros to write code in the "array of structs" style while the code gets desugared to "struct of arrays".

This is not possible in the general case, because you can have a general pointer/reference to a single struct in an "array of structs" but a "struct of arrays" can only be accessed by indexing each array separately. So only very simple and self-contained programs can possibly be "desugared" in this way.

This is Jonathan Blow's goal with his in-development language Jai, see for example this video:

https://www.youtube.com/watch?v=YGTZr6bmNmk

That is changing over time as post launch support grows and some games shift to games as a service

> It's not a weakness to integrate more information about the problem space into the solution. It's engineering.

You want your solution to depend on the right abstract model of the problem, not on the particulars of the problem.

Otherwise your solution is difficult to extend, and is generally expensive and error prone to change when your problem changes, which happens with 100% probability.

Lets say you have "struct Point {int x; int y;};", and all things are fine and dandy. You write your OOP as such (or your DOD version), and things are going great.

Suddenly, you want "struct Point3D {int x; int y; int z;}". How would you change the code?

In OOP, maybe you'd compose a Point3D as such: struct Point3D { Point xy; int z; }; You can now "capture" all of the Point xy; functionality, and extended it to a 3rd "z" coordinate just fine. All old Point code still works, and you have a few functions that work with Point3D now (maybe with a bit of redundant wrapper functions, but such code is straight forward)

The equivalent in DOD seems less obvious. You'd have to make a new "z" array. But the arrays of Point and Point3d wouldn't line up anymore. (If you had 300 Points, and 20 Point3Ds, is Point#315 a Point3D, or is it a Point? Do you "pad" the "z" arrays to ensure that the struct-of-arrays line up?)

There's no obvious way to benefit from a "z-vector", extending the functionality off of your Point to Point3D. I think the DOD design would create a 3rd "z-vector", and shove a "dummy value" into z for all of the points, to make sure all the vectors would have an ideal set of indexes.

Which demonstrates the coupling factor: its not really easy to compose objects in DOD. While in OOP, composing and inheriting classes and objects is the very point.

I would phrase it in the inverse. OOP implies some encapsulation/abstraction that you give up when you use sets of arrays instead.

From what I've seen in DOD systems, there are generally common structures that hold data for the entire application. The entire application has to know about these structures in order to function.

Take ECS [1] as an example.

In order to create a new entity on the system you might need to talk to a location component, a health component, a velocity component all to register a brand new entity on the system. Now, you might have an entity creation service that hides those creation details from you, and that's fine. But you need to make sure as you are talking to each of the component you are doing the right thing.

This is the coupling.

The traditional OO approach here would be to create a new entity with health, position, velocity, all contained within the same object and potentially referencable via a common interface for each.

Now, for ECS in a game there are definite benefits to this coupling. For example, it is a LOT easier in games to create universal systems which handle the physics for all objects irrespective of object type. Further, composing behavior onto entities can be much easier. You are free from a strict hierarchy. There's a reason this approach is popular there.

Now consider a very simple rest webservice. Now imagine trying to do that as an ECS system. You might have one component that is the header component and one component that is the body component and an entity that contains both. Now imagine processing that entity through the system. Who would be in charge of making sure all the right interactions happen at the right time? Who would be in charge of responding? Who would be in charge of deleting requests that had been fully handled? I'm sure that's all possible (I wonder if anyone has tried it? could be a fun side project) I'm also sure I'd very likely not want to deal with such a system, no matter how fast it is. It would be unreasonable to try and figure out all the life cycles of everything.

For business applications with a lot of rules to follow, this sort of system would be nightmarish to maintain. It'd be almost impossible to track down what is changing what. You need a lot of gatekeepers because the goal of business apps isn't to enable novel and unexpected interactions, but rather to very explicitly define what happens when and why. For a game, those novel and unexpected interactions are a huge boon and a feature. They make games fun!

As a side note, the coupling of ECS systems is one thing that makes it hard to properly thread games. That's because you've got this globally shared mutable state being accessed through a ton of systems throughout the game. It's frankly impressive that games are threaded at all!

[1] https://en.wikipedia.org/wiki/Entity_component_system

Likely, yes, but the corollary is if your data structure stays similar and well defined, then you'll have smaller, cleaner and less refactors for reasons that are unrelated to a fundamental shift in your actual data

ECS style DOD arguably makes putting things together easier though, because instead of having to update your actual object when you want to make a change, you only have to add the right component that contains the functionality you need to the given entity ID.

I like Hooks a lot but I’m not sure how it’d apply in the Rust UI sense (maybe well, maybe not). It is nice to not be forced into building classes and to also ensure that specific state changes will force partial re-renders.

I’m talking more about https://relay.dev/docs/en/fragment-container. Creating opaque types that can be fetched from a data source and passed off to a child component is more what I am thinking of.

I mention it because I recall working on a Qt product where you have to parse data from a source, then you bring it throughout your program and try to keep it in the right form for every UI component that uses it.

It was easy for the project but we had no composability of components.

With fragments, you don’t really care and duplication of data between fragments on the same source of data generally just works. That way you can pass the fragments to each component as required.

The idea behind that being that, you can have “efficient” and opaque data structure received from a data source which can be blindly passed to a component to consume.

I say data source but in this instance I mean GraphQL... so maybe it doesn’t apply to a generic UI library.