Content Pipeline Modernization

[Schmunkee]

Hi everyone,

During the roadmap talk at Unite (19 September 2024) we gave you a glimpse of the future. We’re working on the next generation of the Unity engine.

One of the big reasons we believe it’s time for a generational change is that we are determined to improve iteration time. For years, we’ve seen your projects get bigger and more ambitious – and that is awesome! Unfortunately, the more assets your project contains, the more time you generally have to wait while things import.

Nobody wants to be interrupted by a progress bar while in the middle of creating something amazing. We want you to stay in an uninterrupted state of flow.

To truly tackle iteration time, we need to make some fundamental changes to the engine and especially to its content pipeline.

Unity’s content pipeline handles your assets, from they’re imported into the Editor and until they’re loaded in your game on device.

One of the changes we’re making to Unity’s content pipeline has to do with how and when assets are imported.

Import less
In the next generation of Unity, we only import what the Editor needs when it needs it. Say, you are opening a level after getting latest, the Editor will only to import the assets needed by that level. You no longer have to wait for the whole project to be imported before you can be productive.

Import faster
In the next generation of Unity imports happen in parallel using all available CPU cores.

Import non-blocking
The Editor stays responsive while you’re importing with many imports no longer triggering blocking progress bars. After all the assets used by the current level are imported, Unity will continue to import the rest of the project in the background.

These changes alone will have a huge impact on Unity’s “time to interactive” and how quickly you can iterate in the Editor. That’s not all we’re doing with the content pipeline, though, and we can’t wait to tell you more about it.

Sincerely,
Stine & the content pipeline team

[logicalerror]

One thing I’d really love to see, is the ability to add custom metadata to assets. Think adding subtitles to audio, or adding a material-type enum to materials which could be used to look up footstep sounds etc. By connecting data directly to assets, it’s easier to manage them since they’re harder to get out of sync.
This could even be used to pass additional data to custom importers, which could then be edited on the asset in the inspector itself

[EricFFG]

There are still 3 Elephants in the room with Asset Management in Unity

1. There is a massive amount of mess in each unity Project because Unity asset filters shows all bloat assets from third party packages

(image component → look up a texture) (Mesh rendered > look up a material, look up a mesh filter etc)
As there are no naming conventions from third parties, each Unity project which even has the slightest amount of third party packages has a truly massive amount of bloat in the search and making the entire Project feel completely unordered and messy.

Solution: Add a tab in the search field for assets in the Plugins folder, so Team assets and third party assets are split. Add a setting to remove imported (dead) placeholder materials from model files from the project searches.

2. You cannot copy paste in the project view, leading to massive amount of scrolling and a very confusing and lost experience.
If you have a project with a normal amount of folders and subfolders, having to drag and drop things to move them is a pain.

Solution: Add a copy paste function in the Project

3. You cannot read asset names in the project which follow a proper naming convention.
The project view which shows icons (which is critical for all sorts of art assets) only allows a very limited amount of letters, even though there is plenty of space available to render 2 lines of text.
This is a horrible oversight that goes on for comically long.

Solution: Allow the Project view to render 2 lines of text for names instead of one. Also increase maximum scale, because why not. Often its hard to identify textures such as normal maps even on max size.

[Thaina]

Are there any update relate to package and package manager ?

Especially about dependency system. Do we have any update about git dependency on git package?

[PixelDough]

@EricFFG

Add a tab in the search field for assets in the Plugins folder, so Team assets and third party assets are split

This is such a huge point I haven’t seen talked about much. Anything downloaded from the Package Manager should categorized separately, so I can filter on things that were added to the project manually by me and my team. However, idk how much this has to do with the Content Pipeline directly as much as a sort of mix of responsibilities between the Package Manager and the Editor’s asset searching system.

[aras-p]

@EricFFG

Solution: Add a copy paste function in the Project

I’m pretty sure (back when I was at Unity) we added this in like Unity 2021.

[JohnPontoco]

Extremely minor, but I would love for the mouse back/forward buttons to work in the Project window!!

As for importing, I mentioned this in another thread, but I’d love to be able to treat importers more like a pipeline. Being able to create custom ‘virtual’ assets that depend on the import results of other assets, adding custom import steps which can store metadata per asset, etc. There’s a ton that could be done here with the right tools.

[PixelDough]

A whole expansion of the scriptable importers would be nice. For a project of mine, I have to manually create a scriptable object for each of a certain group of textures I have in game. I want to be able to have a custom importer that reads for the naming scheme “SceneBG_” and collects all the matching ones in a folder and finds/creates the scriptable object for me in the same project folder.

You’ve heard of scriptable render pipelines, now we need scriptable content pipelines

[DrummerB]

Sounds great!

Maybe this would be a good time to finally implement a .unityignore file as well.

Support for .unityignore file

[PixelDough]

Seriously, sounds like the perfect time to implement it! Would really love to see that

[simonm_unity]

Hi, thank you for the feedback! I believe some of our plans might make what you are looking for possible. But it would be great to understand how you imagine adding this metadata. Are thinking there would just be an API or should this ideally be possible in the inspector, similarly to how you add a component to an game object?

[aras-p]

@JohnPontoco

Extremely minor, but I would love for the mouse back/forward buttons to work in the Project window!!

Hah. That was the last thing I worked on in 2021 before leaving Unity (not only in project window, but “everywhere” including various graph editors). It got into “stuff basically works” stage, “just” needed code review or something. Someone else was supposed to finish it after I’m gone, but then they got laid off later. Oh well!

[Schmunkee]

Thanks for the feedback @EricFFG and for the +1 @PixelDough !

Really happy to get your thoughts on how we can improve asset management. We’re keenly aware that it’s an area with room for improvement.

I have some follow-up thoughts and questions below:

Elephant 1. It sounds like you’re looking for solutions to two problems, is that right?

• Problem A: When you’re searching for an asset, you want to limit the search results to either the assets produced by your own team OR to 3rd party assets. Can you tell me a bit more about that? Which Search field(s) do you want to do that for? The one in the Project Window? Do you see it as an issue with the Object Picker as well?
• Problem B: You want a way to remove placeholder materials, etc. from the search results. Is it to prevent some team members from accidentally using them? Do you still want to keep those materials in the Project?

Elephant 2. To @aras-p 's point, this is already solved! We haven’t advertised this a whole lot, so I’m sure you’re not the only one who hasn’t even noticed.

Elephant 3. Fair points and great suggestions. I’ll pass this feedback along to the team working on the Project Window for consideration.

[Thaina]

Regarding the package manager UI, I would like to have all dependencies separate into another tab

I mean, I want to always see what dependency was already in my project. But it was distraction from the package I intentionally installed and want to uninstall or update. So it should be sorted or splitted into the bottom of the list

[BrunoL_Unity]

@PixelDough

now we need scriptable content pipelines

oh that sounds good!!
Hey PixelDough!
We have this initiative cooking in our team to enable greater importer customization, don’t embrace the name just yet!
Beside the scene management at import time you mentioned, I’m curious to know if there’s more situations where it could be helpful? Is there operations you’d like to do on the asset itself?
thanks!

[Thaina]

Interesting feature

Is it possible to have one file spit multiple asset? There was a time I have an image that encode normal map and height map together in difference channel. I want it to generate height map and normal map instead of duplicate it and change the import setting

There was also a time when I have psd with many layer of sprite for UI. I wish I could split each layer in the psd into difference sprite or sprite sheet. Same for svg with multiple object. I wish we could make separate UIElement vector object from SVG node too

[JesOb]

In case of ObjectPicker I want to have editor option to filter assets by previously selected asset.

i.e. If I have Material field with setted material on path assets/maps/map01/materials/mtlname_01.mtl
when I open object picker to change material I want it to be prefiltered to path assets/maps/map01/materials/** and the same for all UnityEngine.Object fields. Also show project directory structure in object picker like in project window se I can havigate through established structure instead of peek from one giant list.

This will improve because in project all related assets lie near each other.
I rare cases when I need to choose completly another asset I can find it by navigating through project structure in object picker or just clear filter and then find by name

[JohnPontoco]

@Schmunkee

To @aras-p 's point, this is already solved! We haven’t advertised this a whole lot, so I’m sure you’re not the only one who hasn’t even noticed.

Heck yeah!!!

[PixelDough]

@BrunoL_Unity

oh that sounds good!!

Please feel free to use it lol, I think the name makes a lot of sense and makes it clear what it does! Plus it fits into the existing naming schemes well

@BrunoL_Unity

Is there operations you’d like to do on the asset itself

Off the top of my head, one thing that has always come to mind is wanting to make my own adjustments to how certain things like 3D models import. For example, we’ve all likely encountered issues with importing FBX files, as different 3d modeling programs use different axis orientations. I only use Blender, and have to use some specific export settings in Blender to try to fix the issue, but it doesn’t work in all situations, such as when objects are nested. It’d be great if I could just make a custom version of the FBX importer, and set it to automatically rotate to the correct orientation without me neeting to share obscure blender export options around to my team members to ensure everything stays in sync.

Or maybe I want model files with the name prefix “Fbx_SubSplit_” to generate separate meshes based on the submeshes, which was a very recent need I solved last night by manually running model files through a custom editor tool to generate separate meshes in a folder. The issue with doing it that way is that I now effectively have duplicate meshes in the assets, and they don’t automatically update when the model changes. Being able to have these submesh meshes automatically regenerated on reimport of the FBX file would save quite a lot of iteration time in the long run!

I think this kind of system would tie in really really well with the already existing presets system, which I already use a lot for streamlining import settings based on custom asset naming schemes