Unity has adopted Machine Learning, OOP and Data-Driven (ECS) development approaches within Unity are there other development ‘architectures’ that Unity should or could consider that would benefit game development?
ECS/Jobs/Burst provide multithreaded super fast optimization of game code.
Machine Learning Toolkit - Provides a teachable AI system that can build up complex behaviours and solve problems within your games.
Classical Monobehaviours a good OOP API that lets you quickly build up and prototype games in the unity game engine sandbox.
Personally, I only think GPU programming is needed as a Burst compiler target to give access to the power of the GPU (when it’s not busy rendering).
What do you think Unity needs to complete its programming toolbox?
Well, you could already use multi-threading before Jobs and ECS only Unity is making it much easier to do and building it into the engine with Burst compiler technology. If this works as well as they predict, it would be very hard to use C# multi-threading and get similar results in the engine.
Unity has created the Burst compiler technology that converts our code to the optimal assembly for the target platform e.g. taking advantage of SIMD and similar instruction sets.
It just so happens that people have noticed that the new ECS Mathematics library is remarkably similar in API style to GL shader code, therefore Unity could already be working on a Burst to GPU compilation target (given that this is an unusual API style choice and the raw potential of Jobs/ECS on GPU).
So if Unity can do such a great job with ECS/Burst why stop there why not go for GPU coding as well, after all, they are controlling the compiler now so compiling to GPU could just be like adding a few new compile targets for Burst.
There are probably a similar number of GPU types as CPUs e.g. ARM, Intel, AMD although like the CPUs different types will have different capabilities (probably more so than the ARM or x86 CPUs).
However, I was trying to ask if people think there are other programming paradigms that would add to what Unity already provides?
ECS/Jobs solved a big “can’t do” for a huge amount of developers. They wanted perf without doing hand written asm, they wanted threaded code without race conditions and locks.
So ECS/Jobs is a wonderful addition (but depending on who you ask, it’s the future of all programming, so not just an addition).
So I prefer not to think about what might be needed. How does that help you or me, or anyone? everyone needs very different things. Instead… question will be “what can you not do?”
Before, for a lot of people, they couldn’t write safe incredibly optimised code. Now they can, and that is still a WIP.
As a programmer, I absolutely love the new programming tools in Unity 2018.1, like the Jobs system and ECS. I haven’t been this excited since they added DrawMeshInstanced in Unity 5.5. However, I definitely agree that there needs to be an official visual scripting option for designers who don’t want to program. There are many designers who have asked for visual scripting over the years, and that feature is long overdue.
I think ECS would be a great candidate for visual scripting, you need just a few data items to process and lightweight processes to act on them for maximum throughput. Most of the code is ‘boilerplate’ and could be generated by a visual scripting system with only the unique data formats and logic setup visually by the user.
Another very useful debugging feature for me is “Set Program Counter”, which also does not work with Unity.
Thus, I don’t need more programming systems or whatever you call it, I just want that the tools I use to actually work, rather than “yeah this feature is great, but doesn’t work with Unity”…
Well, you probably would have said the same about Data Driven Programming only Unity have a good system in the works that brings great benefits to game developers.
Imagine a Unity Game Agent-based programming system that makes it easier to develop complex NPC behaviours and works well with ECS and ML it could provide a mid-level system that links the two.
Yeah. I’ve long been hoping that once the new foundations are more mature, this is the sort of thing they will stick on top.
Since the GPU and compute shaders were mentioned earlier, I also have similar hopes on this front for the medium term. eg although its a bit early to tell, it seems reasonable that the new VFX system for HDRP is really generating compute shaders behind the scenes. The VFX system will likely have a somewhat narrow focus, ie particle-based effects, but those efforts could expand in other interesting directions one day. Or not, I dont really mind, but I do find this era to be fascinating with many interesting opportunities. Its gonna take time, but I will enjoy the ride!
No I wouldn’t have since my code is generally like ECS and always has been (not subscribed to the whole OOP paradigm). ECS has massive body of work behind it, a lot from huge companies putting it into practise every day.
Agent-based programming isn’t even complete or used much. I’m not sure what it even solves. Can you tell me what it is actually solving?
I don’t know. I think of ECS in purely performance terms. I definitely stereotype visual scripting as a less performant solution that might be used for simple things like a door opening script. Maybe there would be some kind of visual wizard that could be used to build certain common use cases for ECS.
You’re not alone. My brief attempts at searching for basic information let alone the advantages and disadvantages AOP leads me to believe that it was invented more for the sake of inventing it than actually needing it. Wikipedia has almost nothing to say on the subject and Stack Overflow was strangely in the same boat.
It depends on how the visual scripting works if it’s just a great UI/UX that generates and parses code then it’s just a code interface, not some node based abstract FSM.
I think Unity as a game engine could use some kind of higher level NPC behavioural system, be that an agent-based framework, a behavioural tree, Bayesian decision tree or FSM system… A system that makes it easier to build up NPCs with interesting behaviours.
A rule or Bayesian or fuzzy logic behavioural system that would complement the current Machine Learning system and interface well with the ECS and Classic API could really make it easy to populate games with NPCs with interesting behavioural patterns.
I’m mo Game AI expert so I’m not sure what would be the best system I just get the impression it would plug a big gap in the Unity toolbox.
What you are describing here would be handled better as an asset than as a core engine feature. Maybe a group of programmers could build an open source NPC AI toolkit as a free asset that could act as a broad base for building interesting NPC in a variety of games. I don’t think such an NPC AI toolkit should be part of the engine, though.
Where ECS is awesome is when building super tight code that run many units in the game. ECS gives programmers a way to build extremely scalable systems. A visual interface for constructing ECS solution would probably not be as flexible or as optimized. Honestly, it probably would not even be well received.
The main use case for visual scripting is for designers who want to quickly do something (such as open a door in a house) without needing to program and/or hand off to a programmer. By contrast, ECS is for coding scalable lightweight systems, which is typically not what a designer would be working on.
There may be a useful visual wizard that could be built to help build ECS solutions. But my guess is that it would probably not be the visual scripting solution designers are typically asking for when they mention visual scripting.
This is being solved using machine learning, which is much more effective and being used by large studios in the real world. This is an actively ongoing area of research at Unity. They in addition to this are also solving the animation side of it too.
There is no need to complicate anything. Training AI will suit small and large teams. The small team will lean heavier on it for most things (which is sub-optimal but very helpful) and the large team will use it contextually for say, figuring out a good way to cut through traffic or figure out where a player is more likely to go.
No need for purely theoretical solutions. Fuzzy logic is just another name for parsing some data, something that ML does so much better.
The reason for it being a great fit is that ECS kind of is node based programming at a conceptual level, very small programs but lots of them, with tight constraints. This significantly narrows down how much work a visual scripting system would have to do and to boot leads to the fastest code Unity can generate.
This I hope, is something Unity are considering and will delay visual scripting for because it’ll be worth it.
OK, let’s try a hypothetical situation where ML is the ideal solution but has problems:
A Vehicle combat game, the ML has to drive, pick up powerups and shoot.
An ML could learn to play this game very well no problems.
Problem: The game design needs multiple vehicles, weapons and play styles for the NPC characters.
ML Solution: Train multiple AI for each one.
Behavioural / ML Solution: Train one expert ML for each action needed: driving (steering), pickups*, shooting (different gun styles) then use the Behavioural system to combine and control (e.g. reduce competency/reaction times) them as needed for each play style.
*pickups - would be a strategic and vehicle decision choosing what pickup to go for in any given situation.
The potential benefits here are a massive reduction in the number of ML needed their complexity and training. Also, a hierarchical behavioural system could work with and without ML so can provide none ML powered NPC with basic behaviours.