Advanced Baking Pain Points

[DreamingImLatios]

I’m right in the middle of migrating all my authoring scripts to use bakers. And I have quickly discovered some pain points that I wanted to bring up.

1) Control over which bakers and baking systems are allowed to bake

I’m aware there’s an assembly filter per project to help with this, but let’s be honest. Most people on the asset store are not going to be disciplined enough to split all their bakers into a bunch of different assemblies to allow for the selective customization people are going to need.

For example, I want to override baking for skinned meshes. Non-skinned meshes are fine, as well as everything else in Entities.Graphics. But skinned meshes require some special processing depending on the project. That’s not possible today.

But the most baffling part is that we have ICustomBootstrap, which does offer an opportunity to have extremely granular control over the systems added to a world. Though I have seen my fair share of issues with package samples included a custom bootstrap that conflicts with the project. So…

A) Can we have a custom baking bootstrap that lets us build the baking world and pass in a list of Baker types?
B) Can bootstraps work like Render Pipeline assets where we can drop in the bootstrap we want to use somewhere in the editor and change it on the fly (with a warning for baking bootstraps that all subscenes will need to be rebaked)?

2) There doesn’t seem to be an efficient way to build Blob Assets during baking
I had a conversation with @bogdancoder and @Aede-unity in a now locked thread about this issue. They suggested that I do not build blob assets in baking systems but rather build them inside of bakers directly due to data tracking concerns. Well, the only example we have today of generating blob assets via the baking workflow is Unity Physics. And guess what they do? Literally the exact opposite of what I was told to do. They bake their blob assets in Baking Systems using BlobAssetComputationContext and borrow the BlobAssetStore from the BakingSystem.

So I am really confused as to how to proceed. Here are some use cases I am trying to design for:

A) Someone paints 1000 instances of a prefab in a subscene, and it all gets baked. The same asset and inputs go into baking for all 1000 instances. While I don’t know if those assets and inputs are different from a previous bake, I do know that all 1000 instances will lead to the same blob asset. So I would like to only generate the blob asset once.

I think this can be solved with a little hackery where a Baker can see special baking systems that keep a cache of input-blob pairs and call methods on those systems to generate or retrieve existing blob assets. The baking systems would then clear the cache in their own OnUpdate().

B) Let’s say that someone decides to rerun full baking a subscene that needs 1000 different blob assets. This takes time, because Bakers run single-threaded. The blob generation algorithms don’t parallelize well for a single blob (can you even split a BlobBuilder to multiple threads?) so it is just one thread grinding through while the editor is locked up.

This seems like the problem the physics team ran into which is why they did what they did.

C) Now let’s suppose that I am an asset store developer, and I want to make an asset that can do cool things with a particular type of blob asset. But I don’t want to impose unnecessary limitations on where and how a user stores such an blob. They may want just a single blob in an ICD, or they may want multiple in a dynamic buffer, or they might want exactly three sitting side-by-side in an ICD struct along with some other metadata. How do I make an easy-to-use API to service this use case?

The simplest solution is to just provide a single static method that takes in inputs and spits back a blob asset. However, this runs into the issues that (A) and (B) have.

My old conversion-based solution solved (A), (B), and (C) all at once. But I need a new solution for baking.

3) There’s a dragon with strongly typed authoring references
Here’s the deal, let’s say I have AuthoringA which has a public reference member of type AuthoringB exposed in the inspector. It is super easy in a Baker to read data from AuthoringB without declaring a dependency. So then, how do you declare a dependency on a reference you already have? Well, with the Baker API right now, you re-fetch the reference using GetComponent. That’s a waste of performance, and unintuitive. And in extreme cases, GetComponent could even return the wrong reference if there is more than once instance of the ComponentB on whatever GameObject has the referred-to instance.

The answer for this one is simple. You just need to either add API or account for it in DependsOn (and make it obvious in the docs because reading the code it looked like this was overlooked).

Anyways, I’m really hoping for some quick feedback, thoughts, opinions, and insights as I am very quickly running up to a wall trying to update my projects. It would have been really nice if Entities Graphics still provided conversion so that I would have a checkpoint and could try out the new runtime features. But it is what it is.

[eizenhorn]

@DreamingImLatios

A) Someone paints 1000 instances of a prefab in a subscene, and it all gets baked. The same asset and inputs go into baking for all 1000 instances. While I don’t know if those assets and inputs are different from a previous bake, I do know that all 1000 instances will lead to the same blob asset. So I would like to only generate the blob asset once.

Isn’t Baker have hash based methods where you can check if blob already exist in BlobAssetStore? Or I’ve missed your point?

[topher_r]

@eizenhorn

Isn’t Baker have hash based methods where you can check if blob already exist in BlobAssetStore? Or I’ve missed your point?

I think what they’re pointing out is that checking if the blob exists typically means creating the blob data itself and hashing it. So you paid the full cost of the computation just to throw it away. And indeed, in this case you want to do it in Systems. The blob APIs in Bakers are for very simple cases that also give you automatic dependency management.

[topher_r]

@DreamingImLatios

I’m right in the middle of migrating all my authoring scripts to use bakers. And I have quickly discovered some pain points that I wanted to bring up.

1) Control over which bakers and baking systems are allowed to bake

I’m aware there’s an assembly filter per project to help with this, but let’s be honest. Most people on the asset store are not going to be disciplined enough to split all their bakers into a bunch of different assemblies to allow for the selective customization people are going to need.

For example, I want to override baking for skinned meshes. Non-skinned meshes are fine, as well as everything else in Entities.Graphics. But skinned meshes require some special processing depending on the project. That’s not possible today.

As a work around, you can write BakingSystems that strip things from existing SkinnedMesh Baker that you don’t want. However, just overriding is not as simple as it sounds. What if two packages want to override the same thing? Is it additive overriding? What if they conflict? Do we want users of said packages to have a menu where they have to enable/disable Bakers they don’t understand?

Can you elaborate on the use-case? What do you need to override specifically?

[quote]
2) There doesn’t seem to be an efficient way to build Blob Assets during baking
[/quote][quote]
Well, the only example we have today of generating blob assets via the baking workflow is Unity Physics. And guess what they do? Literally the exact opposite of what I was told to do. They bake their blob assets in Baking Systems using BlobAssetComputationContext and borrow the BlobAssetStore from the BakingSystem.
[/quote]
This release is an experimental drop, we’re not done with samples and guides yet. That said, you’re right in highlighting that doing intensive per-subscene global blob asset building should be done in a system. But I need to point out that Physics was ported from Conversion to Baking by my team, and we simply ported it as-is. It was doing the same blob computation in Conversion, more or less. This is why the computation context is still around, it is a legacy. When we do further work on blob assets after 1.0, we will tackle this and move Physics over too.

C) Now let’s suppose that I am an asset store developer, and I want to make an asset that can do cool things with a particular type of blob asset. But I don’t want to impose unnecessary limitations on where and how a user stores such an blob. They may want just a single blob in an ICD, or they may want multiple in a dynamic buffer, or they might want exactly three sitting side-by-side in an ICD struct along with some other metadata. How do I make an easy-to-use API to service this use case?

The simplest solution is to just provide a single static method that takes in inputs and spits back a blob asset. However, this runs into the issues that (A) and (B) have.

My old conversion-based solution solved (A), (B), and (C) all at once. But I need a new solution for baking.

Did your old Conversion-based solution work correctly incrementally? I’d love to see it, if so. But can you elaborate on what you’re missing here today that Conversion had?

[quote]
3) There’s a dragon with strongly typed authoring references
[/quote][quote]
Here’s the deal, let’s say I have AuthoringA which has a public reference member of type AuthoringB exposed in the inspector. It is super easy in a Baker to read data from AuthoringB without declaring a dependency. So then, how do you declare a dependency on a reference you already have? Well, with the Baker API right now, you re-fetch the reference using GetComponent.That’s a waste of performance, and unintuitive. And in extreme cases, GetComponent could even return the wrong reference if there is more than once instance of the ComponentB on whatever GameObject has the referred-to instance.

The answer for this one is simple. You just need to either add API or account for it in DependsOn (and make it obvious in the docs because reading the code it looked like this was overlooked).
[/quote]

What needs to be accounted for here exactly?

So then, how do you declare a dependency on a reference you already have?

You call DependsOn() as you say. Can you elaborate on what you think it doesn’t account for?

[DreamingImLatios]

First off, thanks for listening and replying. You asked a lot of questions, so prepare for a massive wall of text. (I personally don’t mind the questions, but it does require lots of words to answer them all. :p)

@topher_r

I think what they’re pointing out is that checking if the blob exists typically means creating the blob data itself and hashing it. So you paid the full cost of the computation just to throw it away. And indeed, in this case you want to do it in Systems. The blob APIs in Bakers are for very simple cases that also give you automatic dependency management.

Nailed it!

@topher_r

As a work around, you can write BakingSystems that strip things from existing SkinnedMesh Baker that you don’t want. However, just overriding is not as simple as it sounds. What if two packages want to override the same thing? Is it additive overriding? What if they conflict? Do we want users of said packages to have a menu where they have to enable/disable Bakers they don’t understand?

Can you elaborate on the use-case? What do you need to override specifically?

In these cases, the user should be responsible for deciding who wins. If the user wants to override the bakers themselves, then it is assumed they either understand the bakers they are overriding or the original bakers were garbage to begin with.

But why don’t I explain how I hacked in a solution into Entities 0.50 and why:

I host a GitHub package that can be installed through the Package Manager, which you can find here: https://github.com/Dreaming381/Latios-Framework/tree/v0.5.7
It is broken up into different modules with different features. Some of those features have internal functional dependencies, and do to the lack of git dependency support in Package Manager, I put everything in one package.

One of these modules is my physics solution. It is capable of converting legacy colliders. However, most will want to use Unity Physics, as my solution is specifically targeting users frustrated by specific limitations in Unity Physics design. So I want users to be able to specifically enable or disable conversion of legacy colliders. There’s no real “overrides”, just a matter of turning a feature on or off.

Another one of these modules is my skeletal animation and rendering solution. My solution is highly optimized, is able to consume optimized hierarchies, and operates a bit differently compared to Entities Graphics. In particular, my solution reparents all rendering entities to the converted Animator rather than the root bone. When there are multiple materials, it also designates one of the rendering entities as a the binding entity and reparents all the others underneath it. Also, my solution doesn’t use skin matrices in a dynamic buffer. That aspect is abstracted and computed on the GPU. This design is very different from how Entities Graphics works, and trying to clean up the mess Entities Graphics creates would be awful. I would rather just disable the bakers just for skinned meshes and do everything myself. But in no way do I want to touch non-skinned meshes.

My solution to the problem was to hijack conversion world creation using a combo of asmref and reflection magic to monitor every conversion world created and then monitor when the mapping system was created and hook in at that point. I then searched the project for an ICustomConversionBootstrap and activated it with a similar-ish set of functionality to ICustomBootstrap where a user could either customize the list of conversion system types or directly create and add the conversion systems they wanted to use. I then defined public “Installer” methods that a user could call to turn on or off features, and those methods would do their own conflict checks. For physics, either the legacy collider conversion systems would be installed if the method was called, or they wouldn’t if the method wasn’t. For animation, its installer would disable the Hybrid Renderer’s skinned mesh renderer conversion and then install its own conversion systems.

I think I could pull off a similar stunt with the latest packages today, although I would have to listen to every managed system created from every world since the world creation event got removed. My goal is to make this all work whatever way possible without modifying the packages directly. I don’t usually care if it only works with a specific version of the packages. Unity is fairly slow at releasing updates that I can keep up.

However, having a more official workflow for this would be awesome!

@topher_r

This release is an experimental drop, we’re not done with samples and guides yet. That said, you’re right in highlighting that doing intensive per-subscene global blob asset building should be done in a system. But I need to point out that Physics was ported from Conversion to Baking by my team, and we simply ported it as-is. It was doing the same blob computation in Conversion, more or less. This is why the computation context is still around, it is a legacy. When we do further work on blob assets after 1.0, we will tackle this and move Physics over too.

So would you recommend I try to adapt my existing backend conversion systems using computation context as baking systems in the interim?

If I go that route, I’m thinking of defining an API like this:

[TemporaryBakingType]
public interface ISmartBakerData<T> : IBufferElementData where T : Component // User-defined
{
    public bool CaptureInputsAndFilter(T authoring, SmartBaker<T> baker);
    public void Process(EntityManager entityManager, Entity entity); // Will be invoked by a baking system
}

public abstract class SmartBaker<T> : Baker<T> where T : Component
{
    // Fill with extra functions for requesting blob assets
}

// User must define a subclass of this but don't need to do anything else
public class SmartBaker<TAuthoring, TSmartBakerData> : SmartBaker<TAuthoring> where TAuthoring : Component where TSmartBakerData : ISmartBakerData<TAuthoring>
{
    public virtual bool RunProcessInBurst() { return true; }
   
    public sealed override void Bake(TAuthoring authoring)
    {
        // Do stuff here...
    }
}

Any issues with this design?

@topher_r

Did your old Conversion-based solution work correctly incrementally? I’d love to see it, if so. But can you elaborate on what you’re missing here today that Conversion had?

Could it have? Yes. Did it? No. The mechanism I wrote I called “Smart Blobbers” detailed here: https://github.com/Dreaming381/Latios-Framework/blob/v0.5.7/Documentation~/Core/Smart%20Blobbers.md

The reason I never implemented incremental conversion is that subscenes in general didn’t work very well for me. They would process without Burst and had synchronization issues. Plus for me they would crash in builds when invoking AudioClip.GetData during conversion (though based on more recent reports, this might be a specific issue with my system as others built just fine with the code). Anyways, I believe bakers solve most if not all of these issues.

@topher_r

What needs to be accounted for here exactly?

This might be tough, but catching cases like this and generating a warning would be nice:

[DisallowMultipleComponent]
[AddComponentMenu("LSSS/Spawning/Fleet Spawner")]
public class FleetSpawnPointAuthoring : MonoBehaviour
{
    public FactionAuthoring           faction;  // Read by slots
    public SpawnPointGraphicAuthoring spawnPointGraphic;
}

public class FleetSpawnPointBaker : Baker<FleetSpawnPointAuthoring>
{
    public override void Bake(FleetSpawnPointAuthoring authoring)
    {
        AddComponent(new TimeToLive { timeToLive = authoring.spawnPointGraphic.lifeTime }); // Bug here
        AddComponent<FleetSpawnPointTag>();
    }
}

@topher_r

You call DependsOn() as you say. Can you elaborate on what you think it doesn’t account for?

I was digging through the code, and it looked like DependsOn does asset and existence dependency registration whereas GetComponent does “Structural” dependency registration. I haven’t fully reverse-engineered that codebase yet, so if adding the line…

DependsOn(authoring.spawnPointGraphic);

…is the correct solution, then this concern is satisfied (though I would prefer you document it as such).

[DreamingImLatios]

Some questions I’ve encountered while trying to flush out a design:

1. What is the difference between [BakingType] and [TemporaryBakingType]?
2. When it comes to baking types and baking-only entities, how deterministic do I need to be? Can I pack these with Object.GetInstanceID and UnsafeList<UnsafeList> allocated with World.UpdateAllocator inside a Baker and read this stuff using baking systems?
3. Is World.UpdateAllocator even supported correctly for baking worlds? Or does it just allocate forever?
4. Are worlds recycled for incremental bakes?
5. Is it safe to destroy entities in baking systems?
6. In 0.5, I would instantiate a clone of one of the converted GameObjects, and cache it so that multiple conversion systems could interact with clone and extract data from it. Is there a way I could do the same thing here? Or am I better off creating and destroying the GameObject for each individual baker? Or is this sort of thing not even possible anymore?

[eizenhorn]

@DreamingImLatios

1. What is the difference between [BakingType] and [TemporaryBakingType]?

I have an answer for this at least
TemporaryBakingType components will signal that they are baked in that baking pass, while BakingType components may have been produced in previous baking iterations, in incremental baking (while in play mode). (Answer from the Unity)

• from their summary

TemporaryBakingType strips the component at the end of baking process. As it is removed after a single baking run, it will not be present in subsequent baking runs unless it is (re-)added by a Baker. This also means that the component doesn’t show up at runtime.

BakingType automatically adds the component to each entity that the baking process creates. The component is stripped out automatically at runtime.