ECS, Job System and lua integration. Possible?

Hi,

I am very brand new to ECS. In fact, have’t really touched it yet, other than read and watched multiple conferences and tutorials. But I am excited to look into porting my project, from OOP to ECS. Also, by implementing lua, I want keep game modable.

Basics Concept
In principles, I got many objects in scene, interacting with each other. For simplicity let them be boxes. Each has some behaviors. Some the same, some unique. Up to that point implementation (porting to) ECS seams feasible.

Lua
However, I got lua, moonsharp, which defines and drives behavior of my boxes. This is something I got working well with OOP. To keep simple, for example movement left right, or rotation. Problem I think, I may be facing, is integration of lua, which is OOP based, with ECS and Job System.

OOP System With Lua
Currently what I have, is simple for loop, iterating through my components and executing relevant lua script, which is calling exposed to lua methods.
For example:

List <Box> boxes ;
...
for ( int i = 0 ; i < boxes.count; i ++ )
{
  box = boxes [i] ;
  box.runLuaBrain () ;
}

...

public class Box
{
  // method called by lua brain
  public void _Rotate ( Vector3 angles )
  {
 ... add rotation
 }
}

Lua Integration With ECS?
Thing is, lua moonsharp is a third party library. And is quite extensive, to even attempt looking for transition into ECS.

I am not looking for strict code solution, but guidance, of what steps I would need to take (if possible), to make integration feasible, and actually porting my project itself into ECS worthwhile?

I can assume, something to do with Hybrid ECS? But I am not familiar with it, as of yet.

If I can not call from lua brain (OOP), directly to ECS components, one way of doing it, as I can think of, is to store called references and methods by lua. Which is with relevant inputs (angles) within array/list.
Then make ECS system, recall that array/methods/instructions back, within a JobSystem loop, to apply actual box physical behavior.

Any thoughts on the problem?

Many thanks

Is there any point? Introducing stuff like Lua will entirely remove any and all the benefits of ECS’s performance. You would be best off just using ECS in hybrid mode with Lua, and you’d be doing this mostly for the design pattern, not for any perf gain.

What I can suggest is you just see ECS as a way to do a lot of things to a lot of data, and do not integrate with Lua at all. Although Lua can control the high level start/stop of jobs and make the high level decisions if that was your plan.

2 Likes

The initial point was, to Introduce lua, to make game more moddable and more accessible. This way, script could define, how box should behave.
With lua scripting a simple modification, it could be as silly, as:

  • Get user input forward key
  • Move box left
    or
  • Destroy
    This works relatively well on OOP based architecture.
    Additionally I use ingame Blockly, for lua visual programming, to increase accessibility.

But I see your point.
I did quick recap study on ECS + Job System.
I understand that I can control individual systems with lua, on higher level. Which is at least something.
But I guess, I will miss main point, on lua modding with ECS + JS.
Either I will need drop the lua modding concept, or find other solution.
Or worse case scenario, drop ECS + JS.

I will need give a big thinking about this.

1 Like

There is no JS, only C#

By JS I meant Job System as an abbreviation.
Instead of writing Entity Component System + Job System. ECS + JS look better (shorter) :wink:

1 Like

Sorry, my error. I thought perhaps you were returning to unity after a hiatus and may have missed the whole “whoops we got rid of JS thing that happened” :slight_smile:

1 Like

I would build the normal ECS/JS game-system. And then build a Lua-library (basically a bunch of functions) which can emit components on certain entities which the jobs can rely on.

For example:
PlayerDie() Lua function would emit global commands, like a ‘Die’. Then would create a job which gets this global commands (just like the input system wired in into the jobs/systems) and make sure that the proper component will be emitted on the Player entity, and then the DieSystem can rely on the Die component on the entities so the entity can die. And so on. It won’t be particularly fast, but it would probably work somehow.
It would be relatively safe, because you cannot do anything with Lua which isn’t propagated through this job, but the downside is, that you will have to propagate every single function you would like to provide for the modders.

So basically, check how the input system is wired in jobs and systems and emulate the same with the Lua interface.

2 Likes

Not to worry. I just assume you were tired :stuck_out_tongue:

LurkingNinjaDev
Thx for a response.

As you appointed, as I have currently with OOP based project, exposing only relevant methods for lua, is something I am interested with for ECS. By what you describing, seams as it is feasible. Maybe event having lua running on separate core. But not that matters atm. Unless ECS + JS is smart enough, to switch to idle cores, when getting busy, so would be even less problematic.

So, I will be looking into input interface to ECS. Yet, you have mentioned regarding example on DieSystem ():

.

Would you mind to elaborate, on what you mean by “not fast” in this case? Is that, because is relays on external inputs to ECS? Or because, it need look through DieSystem components and entities? Does that may affect overall performance benefits of ECS + JS?

Some lua scripts may require, to be as fast, as Unity FixedUpdate clock. For example movement triggers, or getting entity position. But many methods, would be either one of, or maybe very so often, or can be executed on slower frequency. This is something, I had working on OOP based project. That should reduce potential overhead, during propagation through exposed methods, to trigger relevant systems. Unless I misunderstood on propagation, which is expected to happen inside ECS, checking upon systems.

Either way, I am happy at this stage, that there is feasible way, of potential interlinking lua to ECS via inputs. More study will be required for me on the topic, never the less.

Many thx.

I need to say, that of many sources available on the net, I found good relevant tutorials on ECS, involving discussed inputs and much more.

Unity ECS Tutorial | Making a Survival Shooter, Part 3 - Input Handling
by
Infallible Code

Also, worth to look at other parts and content.

Sure. ECS + JS will be a super-powerful toolset to write very fast code. Basically feed the CPU with data on a rate it can handle.
If you introduce a separate lib, which works somehow (depends on how the Lua interpreter has been written), and your jobs will depend on it, it’s something you can’t really calculate with.
I mean your jobs will run pretty fast and everything, but your Lua code will still be some OOP interpreter, which will communicate with your running jobs.
So depending on what kind of parts you propagate into Lua-territory it can be devastating effects on the performance.

For example, the graphics and sound and everything will be handled super-efficiently if you put your rendering and core mechanics on the ECS + JS, but if you put your input handling through Lua, that means it will be a lot of delay or lag, depending on how the data propagate from Unity to the Lua libs and back.

If you can run Lua on a separate thread, then it CAN do good things with your system, but do not forget that Unity also can use multiple cores right now and your Lua interpreter may fight for the available threads with Unity’s jobs (internal or yours).

Probably you will need to build a proof of concept and actually measure if the performance is good enough for the purpose you’re intending it for.

Thx, that is great response, with additional explanations.

Expecting near time starting experimenting with ECS + JS. And indeed, I will be building first prototypes, to test behaviors and performance. From what I have gathered till now, my vision with lua is slowly clearing out. But until I have something working, this will stay in the sphere of dreams :wink:

I got fairly an idea / concept, how to pass data in / out, with minimum disturbance. Even if I loose some performance on the lua implementation with ECS + JS, I hope this will be never the less, massive gain, to what I have now.

I do wonder, if anybody else, did lua implementation with ECS?

Scary part is, I may need to rewrite just over 100 my own scripts. Seams like writing new project from almost scratch :hushed: :wink:

Hi! I’m sorry to necro this thread a bit but I wanted follow up…

Would you please satisfy my curiosity and answer a few questions?:

  • So how did it end up for you? Have you succeeded in creating a moddable game like that?
  • If you did succeed then what parts of your game have turned out to be moddable with this approach?
  • What Lua implementation were you using? (I’m using Moonsharp and it generates and interprets bytecode, but I know there are some implementations targeting IL?)

I’m asking because I’m working on something similar, a ECS-based game with the ECS being accessible from Lua up to the point of being able to create custom components, functions and such. I’m not using UnityECS and I can imagine that this approach is terribly counter-performant on a greater scale.

But making a game both modable and not dependent on some scripting language seems impossible unless you force the users to mod it with C# in Unity. And I’m not sure if the optimizations that UnityECS provides are actually worth the hassle if the game doesn’t strive for having tens of thousands simultaneously active entities… though I try to do all the performance-critical stuff in C# of course…

All in all I have it all working by now but this is nearly the only post I have found where someone else discusses something similar. So I would be grateful for any insight or optinion on the matters.