Exciting developments in upcoming Entities 1.0 releases

Unity Engine
, ,
1

Howdy. I’m Joe Valenzuela from the DOTS team. We’ve been hard at work making changes in both the Unity Editor and various packages as part of our initial Entities 1.0 experimental release. There are all sorts of bug fixes, optimizations, and new APIs that we think will allow Unity developers to make their games more ambitious, more performant and, in general, more awesome.

Thanks to everyone who used these packages in preview and provided feedback in the forums or elsewhere - our experimental 1.0 release has relied on your insights and efforts. As part of that process we’ve taken a careful look at our existing APIs and in some cases decided to deprecate, rename, or remove types and functions. We’ll have a detailed upgrade guide for folks who have been following along with Entities before full release, but in the meantime I wanted to show off some things I think are cool about the experimental release.

Conversion → Baking
Conversion has been the heart of the Entities data pipeline. In the 1.0 experimental release, we’ve reengineered it to the point where we decided it needed a new name: Baking.

In the old Conversion system, component data is authored in the familiar Unity environment you know and love, and data is converted from this to a fast, memory-ready format either at run-time (via ConvertToEntity) or at Editor time (by virtue of being in a Subscenes).

In Baking, you still author data in the Unity Editor and get fast, memory-ready binary data at runtime, but the mechanism is substantially different. We’ve split the process into a two-pass system in order to do the most expensive processing with ECS, so it’s faster. The dependency tracking is precise, deterministic, and robust, meaning you only update the data that’s changed, which improves live update with open subscenes. And the code itself is simpler and easier to understand, and as a consequence has fewer bugs and edge cases. The API is safer as well, making it harder to make mistakes (and simpler as well, we think, but that’s for you to decide!).

Here’s a small example of the API:

public class SelfDestructAuthoring : MonoBehaviour
{
    public float TimeToLive;

    class Baker : Baker<SelfDestructAuthoring>
    {
        public override void Bake(SelfDestructAuthoring authoring)
        {

            AddComponent(new SelfDestruct {TimeToLive = authoring.TimeToLive});
        }
    }
}

One thing developers should be aware of ahead of time is that we’ll be deprecating runtime conversion - like ConvertToEntity - and removing it before our full 1.0 release. We know this is a commonly used feature, but runtime conversion has been a constant source of bugs (because it requires maintaining a separate pipeline for moving authoring data around) and isn’t possible to maintain with our goal of keeping editor functionality out of the DOTS runtime. Thankfully, runtime conversion is (almost) never necessary, and in the few cases it is, the subset of functionality needed is straightforward enough to support on the client side.

Build Configs
Standalone builds previously required the use of Build Configuration Assets and the separate “build config” menu options.

1600×752 118 KB

This will no longer be the way of building standalone projects using DOTS.

Having a completely separate build tool allowed us to rapidly prototype configuration options for DOTS, and it’s necessary for some work we’re doing unrelated to Entities 1.0. But having a different way of making standalone builds was a frequent source of confusion for new users, and consolidating build systems is not possible in any reasonable timeframe. While we think Build Configs may be a great solution someday, they aren’t strictly necessary today, and we want to prioritize the simple solution that works for almost every case. So we’ve made the classic Unity build page work for the vast majority of cases. Developers who need more specialized build configuration can use the Scriptable Build Pipeline, a powerful API for building content.

Build Configs will still operate as usual in the experimental build, and building with the classic Unity build window will be available as a toggle-able option (currently Preferences → Entities → Builtin Builds Enabled, but subject to change). By release we plan to make building via the build window the default.

Transforms
While developing Entities, we gradually realized that Transforms - the components and related systems that maintain an object’s position, orientation, and hierarchy - needed revision. Based on our own experience and those of our partners, we’re rolling out a substantially improved API and backend. In addition we’re taking the opportunity to prevent errors by precluding the use of features at runtime that are difficult or impossible to support well, like non-uniform scale (non-uniform scale for static data is handled at Bake time, and we have special case path to allow some common uses of non-uniform scale).

There are some finishing touches we have to put on before fully switching over to the new Transform system, but when we’re done I think you’ll find…

It’s simpler.

Transforms are now baked from the GameObject Transform down to a wafer thin Entity representation of only 8 floats. The Entity component, in turn, is transformed by fewer systems in a more streamlined fashion, using fewer systems, and operating on less data.

It’s cheaper.

Gameplay code frequently deals with rotation and position. Previously, transforms were based completely on matrices, making extracting rotation expensive (at best). Now we maintain data in its quaternion form, the format in which it’s most often used.

It’s robust.

Our previous Transform data protocol had 15 different component types, some of which were very infrequently used, but all had to be supported. The new data protocol is substantially smaller and easier to support.

It’s more correct.

The previous Transform data protocol didn’t put any real world constraints on what was feasible in game code - literally just offering a matrix - so every system made up its own rules on what it could handle. This can lead to unintuitive errors as systems ignore features of the provided transform (like non-uniform scale).

idiomatic foreach
When Entities.ForEach was introduced, it was the quick and easy alternative to writing full-blown job code. However, the fullness of time revealed some limitations. The biggest problem was that nested loops were impossible, making it cumbersome to model many typical solutions. And after the introduction of IJobEntity, the barrier to jobified code was now no longer as steep, so we went back to the drawing board to express the most common remaining use case for Entities.ForEach - main thread immediate code - using more natural C# syntax. We call the mechanism idiomatic foreach.

foreach (var (rotateAspect, speedModifierRef) in
                SystemAPI.Query<RotateAspect, RefRO<SpeedModifier>>())
     rotateAspect.Rotate(time, speedModifierRef.ValueRO.Value);

The biggest takeaway is that there are fewer edge-cases where APIs that work outside of a loop don’t work inside, and there’s some behind-the-scenes improvements as well. Entities.ForEach remains in 1.0, but we’re strongly encouraging folks to use idiomatic foreach and jobs instead. To that end, we’re not supporting Entities.ForEach in ISystem (it will remain in SystemBase) and plan to phase out support post 1.0.

Aspects
One of the most obvious new API features is the introduction of Aspects. Aspects offer a view on commonly used data (like translation and rotation) along with methods in order to provide a low-cost, fully-burstable interface. This allows developers to write code against an Aspect without coupling themselves to irrelevant implementation details. You can also write your own Aspects to make your code easier to use correctly.

Here’s a simple example to keep something oriented toward another entity’s position. The code is straightforward, but requires us to query both the Translation and Rotation component. If one of these became redundant (unlikely in this case, but stay with me) you’d have to audit every query to make sure you were using the minimal set of parameters or else you’d be introducing superfluous dependencies. (Small note - currently Aspects will pull in dependencies for all contained components whether they are used at the call site or not, so there’s room for performance improvement in future releases).

public partial struct LookJob : IJobEntity
{
    [ReadOnly]
    public ComponentLookup<Translation> lookup;

    public void Execute(ref Rotation r, in Translation t, in LookAt_RotationTranslation lookAt)
    {
        float3 head = lookup[lookAt.Other].Value;
        float3 forward = new float3(head.x - t.Value.x, t.Value.y, head.z - t.Value.z);

        r.Value = quaternion.LookRotation(forward, math.up());
    }
}

protected override void OnUpdate() {
    new LookJob{lookup=GetComponentLookup<Translation>(true)}.Schedule();
}

With the TransformAspect, the specific mechanism used to encode position and orientation are kept up to date. Write your API against the aspect and it’s trivial to keep up to date. And best of all, we use the same query mechanisms behind the scenes, so you still get performance by default.

public partial struct LookJob : IJobEntity
{
    [NativeDisableContainerSafetyRestriction]
    [ReadOnly]
    public ComponentLookup<Translation> lookup;

    public void Execute(ref TransformAspect ta, in LookAt_TransformAspect lookAt)
    {
        float3 otherPosition = lookup[lookAt.Other].Value;

        ta.LookAt(otherPosition, math.up());
    }
}

protected override void OnUpdate() {
    new LookJob{lookup=GetComponentLookup<Translation>(true)}.Schedule();
}

We’re providing two Aspect interfaces to start - TransformAspect and RigidBodyAspect - and are looking forward to hearing your feedback on the feature.

Enableable Components
One of the most far-reaching changes we did was to introduce the concept of Enableable components. Previously, if you wanted to exclude a set of entities in the same Archetype from a query, you’d have to either group them with SharedComponent filters or change their Archetype by adding and removing tag components. Both options require expensive structural changes, and increase memory fragmentation. Furthermore, neither operation can be performed immediately from job code; the desired operations must be recorded into an EntityCommandBuffer and played back later in the frame, leaving the entity in a potentially inconsistent state in the meantime.

Enableable components can be efficiently enabled and disabled at runtime without triggering a structural change (even from job code running on a worker thread). A similar effect can already be achieved with a component containing a one-byte “isEnabled” field, but enableable components are a first-class feature of the Entities package, fully supported by EntityQuery and Entities job types. Disabling a component on an entity prevents that entity from matching a query that requires the component, which means that jobs running against this query will automatically skip the entity as well. SIMD-optimized query-matching and chunk-iteration code keeps the overhead of skipping disabled components to a minimum. Enableable tag components are ideal for high-frequency state changes, without increasing the number of unique archetype permutations. In our tests, toggling enableable components from a parallel job runs 9x faster than adding/removing the same components using structural changes.

public struct TargetEnableable : IComponentData, IEnableableComponent
{
    public float3 target;
}

// ...
var archetype = m_Manager.CreateArchetype(typeof(EcsTestData), typeof(TargetEnableable));
var entities = m_Manager.CreateEntity(archetype, 1024, World.UpdateAllocator.ToAllocator);

EntityQuery query = GetEntityQuery(typeof(TargetEnableable));

// Disable the target on one of the entities.
// entities[4] will now be excluded from the query.
m_Manager.SetComponentEnabled<TargetEnableable>(entities[4], false);

int fooCount = query.CalculateEntityCount(); // returns 1023

ISystem & IJobEntity
While they aren’t new in 1.0, we’ve added support throughout Entities packages to make ISystem and IJobEntity more useful than ever.

ISystem is an interface implemented by unmanaged component systems. It’s specially designed to keep you on the fast path using Burst and away from C# managed memory, which introduces garbage collection. Support has been added throughout Entities (and other packages) to keep us on the fast path as much as possible.

While it can’t write your code for you, It’s never been more straightforward to stick to the “high performance” subset of C# (HPC#) that unlocks next-gen performance.

[BurstCompile]
public partial struct TheFriendlyOgreSystem : ISystem
{
  // Store reference to a friendly ogre entity
  Entity friendlyOgreEntity;

  [BurstCompile]
  public void OnCreate(ref SystemState state)
  {
    // Create a temporary ComponentType NativeArray.
    using var friendlyOgreComponentTypes =
    new FixedList64Bytes<ComponentType>
    {
      ComponentType.ReadWrite<HealthData>(),
      ComponentType.ReadWrite<AliveTag>()
    }.ToNativeArray(Allocator.Temp);
 
    // Create an Entity of Archetype (HealthData, ShieldData)
    var friendlyOgreArchetype = state.EntityManager.CreateArchetype(friendlyOgreComponentTypes);
    friendlyOgreEntity = state.EntityManager.CreateEntity(friendlyOgreArchetype);
  }

  public void OnDestroy(ref SystemState state) {}

  [BurstCompile]
  public void OnUpdate(ref SystemState state)
  {
    // Check if the friendly ogre is alive
    if (state.EntityManager.IsComponentEnabled<AliveTag>(friendlyOgreEntity))
    {
      // Get HealthData of the friendly ogre. and kill it over time. (10HP per second)
      var healthData = SystemAPI.GetComponent<HealthData>(friendlyOgreEntity);
      healthData.Left -= 10f*SystemAPI.Time.DeltaTime;
      if (healthData.Left > 0)
        SystemAPI.SetComponent(friendlyOgreEntity, healthData);
      else
        state.EntityManager.SetComponentEnabled<AliveTag>(friendlyOgreEntity, false);
    }
  }
}

Part of this effort has gone into making it easier to move code from the main thread to a job, or from your update to a utility function, or from a SystemBase system to an ISystem one.

In part because of this expanded support, IJobEntity is now the recommended job type to use when convenience is desired.

partial struct UpdateChunkBoundsJob : IJobEntity
{
    [ReadOnly]
    public ComponentTypeHandle<BoundsComponent> ChunkComponentTypeHandle;

    void Execute(ref ChunkBoundsComponent chunkBounds, in ChunkHeader chunkHeader)
    {
        var curBounds = new ChunkBoundsComponent
                        {
                          boundsMin = new float3(1000, 1000, 1000),
                          boundsMax = new float3(-1000, -1000, -1000)
                        };
        var boundsChunk = chunkHeader.ArchetypeChunk;
        var bounds = boundsChunk.GetNativeArray(ChunkComponentTypeHandle);
        for (int j = 0; j < bounds.Length; ++j)
        {
            curBounds.boundsMin = math.min(curBounds.boundsMin,
                                           bounds[j].boundsMin);
            curBounds.boundsMax = math.max(curBounds.boundsMax,
                                           bounds[j].boundsMax);
        }

        chunkBounds = curBounds;
    }
}

But wait there’s more!
There’s a lot to look forward to in Entities 1.0 (some of these code samples include “sneak previews” of other changes we’ll be discussing in upcoming posts) and we’re all looking forward to finally getting our work in front of y’all. While we’re proud of the work we’re doing, the real payoff is seeing what all the incredible Unity developers do with it. Thanks for reading and we will keep you posted for more detail as we approach the official release of Entities 1.0!

40 Likes
2

Hi @jivalenzuela**
**
Thanks for this post :slight_smile:

  • Can you explain more how can we scale entity non-uniformly in new transform system?
  • Can we use ComponentLookup outside of Job?
3

Based on known bugs in this topic I have one question:

Why you try to regenerate user code? Why not just generate new code that can be called manually so we can write whatever we want and never face bugs that our code can not be reconverted to some another form?

Current bugs is exactly result of fail on trying to regenerate from user code.
Can you stop doing this or provide or provide option to disable unity generation and write our own or support alternative way maybe something like this:

// we write
    [SgenECSJob, Schedule, AggresiveInlining]
    private static void LookJob(ref TransformAspect ta, in LookAt_TransformAspect lookAt, in ComponentLookup<Translation> lookup )
    {
        float3 otherPosition = lookup[lookAt.Other].Value;
        ta.LookAt(otherPosition, math.up());
    }
  
    [SgenECSJob, Schedule, AggresiveInlining]
    private static void MoveJob(ref TransformAspect ta, in Move_TransformAspect move )
    {
        ta.Move(move.amount);
    }
  
    protected override void OnUpdate()
    {
        OnUpdate_SGen( );
    }
  
    // Sgen Auto Generated
    public partial struct LookJob_SGen : IJobEntity //there full implementation of job
    {
        [NativeDisableContainerSafetyRestriction]
        [ReadOnly]
        public ComponentLookup<Translation> lookup;
  
        private void Execute(ref TransformAspect ta, in LookAt_TransformAspect lookAt )
        {
            LookJob( ref TransformAspect ta, in LookAt_TransformAspect lookAt, in lookup );
        }
    }

    public partial struct MoveJob_SGen : IJobEntity //there full implementation of job
    {
        private void Execute(ref TransformAspect ta, in Move_TransformAspect move )
        {
            MoveJob( ref TransformAspect ta, in Move_TransformAspect move );
        }
    }
  
    protected override void OnUpdate_SGen()
    {
        new LookJob_SGen{lookup=GetComponentLookup<Translation>(true)}.Schedule();
        new MoveJob_SGen{}.Schedule();
    }
4

A lot of this stuff was badly needed. I look forward to using the new Transforms.

You may want to think about removing Entities.ForEach in SystemBase before 1.0. The reason I say this is because a lot of old tutorials use Entities.ForEach. Many new users are probably going to be using those old tutorials. If anything is going to force the rewriting of tutorials it will be the 1.0 release. I just think it makes sense to get the API stabilized at least as far as the fundaments are concerned before 1.0.

3 Likes
5

Can you please make IsComponentEnabled return false for absent component instead of assert
In this case we can use this API as replacement for Adding/Removeing tags without code rewrite
and actually mix and match both approaches on per entity basis. Some entities disable component some just remove it.

6

Can you explain more how can we scale entity non-uniformly in new transform system?

There’s a post-transformation matrix you can use to apply arbitrary linear multiplies with your final LocalToWorld. They don’t propagate hierarchically and won’t be used by the Transform system.

Can we use ComponentLookup outside of Job?

Yop!

1 Like
7

You may want to think about removing Entities.ForEach in SystemBase before 1.0. The reason I say this is because a lot of old tutorials use Entities.ForEach. Many new users are probably going to be using those old tutorials. If anything is going to force the rewriting of tutorials it will be the 1.0 release. I just think it makes sense to get the API stabilized at least as far as the fundaments are concerned before 1.0.

Thanks for the suggestion. It’s been a hotly debated topic on the team, especially since 1.0 represents the best time to clean up the API. While we’ve got a lot of confidence in the replacements for Entities.ForEach, removing it in the anticipated timeframe is unlikely.

8

Good to see Entities finally reached 1.0.

If runtime conversion and ConvertToEntity are dead, then what’s the best approch to dynamic load prefabs and convert it to an entity, pure or hybrid?

So in 1.0 all pure entities must be placed inside a subscene? What about the hybrid entities? How to understand the memory usage and incremental updates if we put all our prefabs into a subscene? The release schedual of DOTS addressable?

3 Likes
9

ISystem allows easy access to main-threaded burst-compiled code however if we need burst-compiled code that is run outside a system, or doesn’t need to act like a system, is there going to be an API that allows us to run some main-threaded burst-compiled code? Or is the official planned way to achieve this through the Burst FunctionPointers?

1 Like
10

Just put [BurstCompile] on a static function and its declaring type, and when you call it it should be the bursted version; this has worked for several burst versions now.

2 Likes
11

Great to see the experimental 1.0 version.
Everything looks pretty solid.

One sentence concerns me though:

I am currently using a custom Addressable Entities implementation to load and convert the prefabs at runtime.
I would love to replace this with a solution that converts the prefabs at edit time but so far this is not possible.
Subscenes seem to be the only solution to properly convert game objects which really leaves a big gap on the assets/prefabs side.

I really hope addressables for entities will fill this gap. Sad it didnt make it into the experimental release.
Would be optimal to just swap my addressables with the official implementation to make it work.

We are using a custom format for tile maps which are populated at runtime with tilesets loaded from prefabs.

This hard cut between edit and runtime data might be problematic with tools used in game and in editor in general.
E.g. when building a level editor there are only 2 options:

  • Build 2 separate editors (One for the Unity Editor editing GameObjects, another for ingame editing Entities)
  • Build one editor which edits entities (ignore GameObjects and baking)

I opted for the second option which brings some difficulties with it as it requires running the ECS world in edit mode.
The paradigm of edit and runtime data just doesn’t fit well here.

5 Likes
12

Is there any Entity 1.0 samples or demos? Even work in progress.
Running into a bunch of issues trying to upgrade the ECSSamples for entity 0.51.

1 Like
13

There is a post somwhere with link to Unity training github repository, for Entities 1.0. Try to find one here on Unity DOTS forum. It maybe of help for you. Srr I am on the mobile atm.

2 Likes
14

There is 1.0 branch in ECSSamples repo

2 Likes
15

Oh dear, thank you!

16

Is the new transform component intended to be deterministic across machines?
This says transform will be 8 floats so I’m thinking we’ll still have indeterminism across multiplayer simulations but just wanted to ask since determinism would be such a great checkmark.

1 Like
17

Does it mean I can finally make a editor only assembly definition without getting compile error anymore when try to build player runtime and also sometimes has weird compile error at authoring MonoBehavior?

I think I already stressed how incredible important this build config feature quite number of times that build config enables the possibility to get different player runtime build easily. Without build config I can’t imagine how could I achieve the same thing. Instead of spending a lot of time consolidating build systems I think making this platform package to be first class citizen for both dots and OOP projects is much better decision. From what I see platform package is technically implemented on top current build systems. I think just continue to maintain this platform package and slowly improve it is the better road to go. You can make it even better in future by using Scriptable Build Pipeline as what u mention to make it become something like URP and HDRP that is built on top Scriptable Render Pipeline as long term plan. What I want to bring out is instead of just directly deprecate platform package you can slowly improve platform package and make it compatible for both dots and oop projects

1 Like
18

Can you continue to explore the way to write parallel jobified code off the main thread that still using idiomatic foreach approach that can be as easy as Entities.ForEach? IJobEntity just too many boilerplate typing for most of the use case that is just too time consuming that in future I want main thread code to become jobified code I need to spend quite some time to make it work which is really tedious and boring. And also I think it’s really critical for the last minute optimization when ur game is going to ship very soon in few days.

Want to confirm that can I just use idiomatic foreach approach at SystemBase or this idiomatic foreach approach is at ISystem only?

Do u mean that in future release, Aspects can be evolved to codegen truly efficient code that system only read/write the only required data? For example, when using TransformAspect u only read Translation component then will codegen only read Translation component.

Any plan to make ISystem much more less typing? At least remove the need to add lots of BurstCompile attribute to struct ISystem and all the ISystem methods.

1 Like
19

Thats also the reason I use the platforms package for all of my projects regardless of DOTS or not.
Having different build settings per “platform” is just not sufficient enough.
I often require different build settings/configs for different storefronts or VR devices or even building a level editor and a game/main application (different scenes) from the same project.
It was also nice to integrate additional build steps. Such as injecting a version number.

I would have maintained/extended my custom build manager if I hadn’t relied on Unities platforms package for so long.
No I will have to start from scratch I guess.

But the decision here seems to be pretty final

1 Like
20

DOTS samples-1.0-exp
https://github.com/Unity-Technologies/EntityComponentSystemSamples/tree/release/samples-1.0-exp

DOTS training samples.
Look for recent branches
https://github.com/Unity-Technologies/DOTS-training-samples/tree/master

4 Likes