I have some older code that I’m trying to convert to jobs/ECS and one thing that I use a lot in that code is async/await, specifically for things like waiting for a database query to run.
I want to jobify the DB code, so that I can schedule say a “FetchPlayerDataFromDB” job, put it as a dependency of my “InitPlayer” job, etc.
However, I can’t figure a way to do it. The obvious thing to do is to Schedule() a job that calls ‘sqlConnection.Open()’, then creates the reader and executes the query, all synchronously. The problem is, job structs (or lambda jobs) can’t have managed members - which the MySQL DB driver objects are!
I can write a lot using blittable structs but when it comes to databases, I have to use a 3rd party .NET dll that provides only managed objects. Has anyone figured out a way to make databases work nicely with the job system?
Async/await seems to be a better option for this kind of operations. From my understanting, jobs are meant to maximize the frame slice throughput, not to work like Tasks or Threads.
Hmm I was wondering about that myself. I use some jobs for long-ish running build jobs at the moment, but maybe it’s different at runtime. I know they are definitely tailored to the ECS framework which is all about updating as many things as possible within a single frame but so far I haven’t seen anything that would prevent detached (aka not system-bound) tasks to run for extended periods of time.
If anyone has any experience with this way of using jobs, please feel free to share!
The problem is, when you fetch query from DB, you receive string. This need be parsed into usable data accordingly, in your app. Hence I don’t think you can jobify SQL queries.
What you can do however, after fetching and converting into usable data, you can process it inside jobs if needed.
Client side you shouldn’t be making db calls period. The server provides feature level api’s to the client. Request/response or in some cases the server just persists stuff as a result of some actions.
Server side it gets a bit complex. As db calls are generally done async, but async is also generally all or nothing.
So this pretty much forces a hard separation between something like ECS and all of your logic that deals with persistence. Most games just separate it out completely into a service oriented design, with the services running on separate servers. And the client might be communicating with more then one server, or a single server that itself is making some requests to another server.
For prototyping or early on there are simple versions. But what matters most is the right separation. That’s far more important then how you actually persist your data. Really can’t stress this part enough.
I would start out with a simple request/response flow based on messages. That will force you to design the ECS integration in a good way. Something you won’t have to completely refactor later.
And your persistence layer could just be in memory dictionary/lists to start with. Maybe serialize it all as a blob to a file on demand or at key points. It could also be a db. Whatever is the simplest that works. The key here is persistence is completely abstracted away, you could switch to something different without impacting anything higher level.
And that design you could start out with all on the client. Once you have a server part of it moves server side but it’s the same flow and probably the same messaging.
I addressed the question as more of a persistence flow question as I though that was more useful. But there is also another common area here which is third party api’s that use async. Many games just move those to the server and the client accesses them via your own messaging flow. Generally for other reasons like the api’s require authentication anyways, or you just don’t like the idea of third party stuff on the client making external calls. But it also solves the whole async issue.
So in practice async should not really be an issue on the client. Other concerns will remove it from the equation for the most part.