Runtime Scene Serialization package. Status?

Hello. There is manual page for Runtime Scene Serialization package, and we can install it by “Add Package by Name”

“com.unity.runtime-scene-serialization”

https://docs.unity3d.com/Packages/com.unity.runtime-scene-serialization@0.2/manual/index.html

But there is no anymore info about this amazing package. And I’ve no success to make it work on runtime, only in editor.

Is this package in active development, or is it abondoned?

Hi there - I am having the same question and am bumping this thread for that reason :slight_smile: Did the package get removed from the registry? Best, ReinekeFux.

Hi there!

Thanks for reaching out about this preview package. It exists to support the MARS Companion workflow, which is still in beta, and indeed under active development! The 0.2 version of this package is based on com.unity.properties and com.unity.serialization version 1.5.0-preview from the DOTS family of packages, and has become reasonably stable. However, because properties is still in preview, so too is runtime scene serialization. As with any preview packages, we do not provide official support, however I’d be happy to help troubleshoot any issues you are having! Please do heed the warning, though, that you may encounter issues in Unity 2020 and above, and especially if you are using other DOTS packages in your project. The package is still available and can be installed by name or by modifying your project manifest.

I’m sorry to hear you’re having trouble. The same APIs that work in the Editor should work at runtime, as long as the codegen step is done. If you are using the latest version, you must go to Project Settings > Runtime Scene Serialization and manually enable codegen for the assemblies/classes you would like to serialize at runtime. Earlier versions (0.2.1-preview and below) generated code for all serialized types across all assemblies by default, but we encountered edge cases which caused build errors for some customers with the MARS Companion packgage in their project. Because of this issue, we updated the package to treat codegen as “opt-in,” which has the added benefit of keeping your build size/time down if you aren’t using runtime serialization.

Hope this helps. Good luck!

Thank you lots for the help & input. I will take a look at the package and see if it is a fit for the project that I am working on. Really curious where this will be taken in the future. :slight_smile:

Best,
ReinekeFux

Hey guys, anyone using this now and can share some info on current usability?

Hey, testing this out now and seems to work nicely.

I’m not sure of how much this package is supposed to handle, but when I try to serialize a running scene in editor I get warnings that GUIDs is missing for all materials and meshes in scene - which I half expected.

But this begs the question how to fix this so I can import the scene and meshes and materials survive.

I’m currently thinking about adding a layer that serializes the GUIDs on objects so while app is running this information is available and can be used to link up assets. But I almost half expected something like this in the package so just wondering if what I’m doing sounds sensible :slight_smile:

I tried rebuilding project using 2019 LTS and built-in render pipeline but same issue, meshes and materials can’t be serialized (any assets references get lost if you serialize scene at runtime).

I enabled all code-gen settings in project settings to be safe but still same issue.

Can anyone confirm if you are supposed to be able to save and then import a scene and assets references survive?

Hey there! Thanks for your interest in this package. Sorry for the confusion–we’re still working on improving the documentation for this package, which currently doesn’t explain how to use an AssetPack to handle guids and asset references.

Essentially, what you’re missing is an AssetPack object which you can use to collect asset references. In Edit mode, you will need to serialize your scene using an Asset Pack, which can then be used to map between guid/FileId and object reference. An AssetPack is a ScriptableObject which can be saved as an asset for future use, or you can just instantiate one in memory if you do not need it to persist.

Take a look at Editor/MenuItems.cs for an example of how to use an AssetPack. When you choose the File > Save JSON Scene... menu item, this code will create an AssetPack (or re-use an existing one) for the currently active scene and use it during serialization to collect asset references. Then, this AssetPack can be used in Edit Mode or at runtime to look up asset references that were serialized as guid/FileId in the json representation of the scene.

If you want to serialize at runtime or in Play mode, you will still need an AssetPack pre-populated for any asset references that you might encounter. We can’t use guids at runtime, so there is no way to reference a new asset whose guid we did not look up at runtime.

There is one exception for this, which is the IPrefabFactory interface, which allows an external class to provide prefab references based on guid. It still requires that you use an AssetPack when deserializing.

2 Likes

Hello, I am going through some exercises to learn Unity and it looks like this is something that’d enable saving and loading a game without needing to address each GameObject or component individually. Is that an intended use for this package? Checking ‘Editor/MenuItems.cs’ it seems to rely on some assemblies only available in the editor rather than in a standalone build. Is there an example of loading/saving a scene in a standalone build (e.g. in response to some keypress or UI interaction)? Thank you

Hi there!

Yep. This package is intended to help you save/load scenes, although it is still experimental and doesn’t yet include proper documentation and samples. This is something we plan to improve, but for now this package exists to serve the AR Companion App workflow. We do not officially support using it on its own.

I wouldn’t recommend trying to integrate this package if you are new to Unity. There are some solutions on the Asset Store which serve a similar purpose and provide good documentation. I’m not familiar enough with them to endorse any particular product, but a quick search for serialization or save/load will get you started.

As for your question about Editor/MenuItems.cs, the part that I was calling out in my last message is about how to use an AssetPack. The rest of it is somewhat specific to the Editor context, so you might instead look at the play mode tests for an example of how to serialize a basic scene with GameObjects. The high level is that you want to use SceneSerialization.SerializeScene to save and SceneSerialization.ImportScene to load. However, pretty much any useful scene is going to have asset references (even the default primitives are asset references) so you’ll need to worry about AssetPacks, as described above.

Just a quick update here: we’ve updated the package with a few minor fixes, and improved the documentation slightly to include some of the questions asked in this thread. Hopefully it will help other early adopters like yourselves :slight_smile:

https://docs.unity3d.com/Packages/com.unity.runtime-scene-serialization@0.3/manual/index.html

1 Like

Thank you for the documentation! I believe I need to somehow use the Editor itself to “export” my scene and assets it refers to into an AssetBundle containing an AssetPack. If I’m not mistaken, the new documentation says I’m not really going to create such a AssetPack/AssetBundle in code at runtime. Instead, I should look at the Editor UI for such an export once I’ve added separate package " AR Companion Resource Manager ". Then, with that exported AssetBundle, I would be able to reference some code in that AR package which can load the exported AssetBundle at runtime. Have I understood this well?

Not exactly. I mentioned the AR Companion package as an example of how the package can be used to create AssetBundles, but you can’t use it as a general purpose tool. It immediately uploads the asset bundle without saving a local copy, so you won’t be able to use the generated AssetBundle in your own projects. However, reading the code in that package may help you write your own scripts. Generally speaking, projects that use AssetBundles use editor scripts to automate the process of building a bunch of AssetBundles at the same time as part of a larger build pipeline. Then, at runtime more code is needed to load up the AssetBundle and extract the asset references. You may want to put multiple AssetPack assets into one bundle to support multiple scenes, or build one bundler per-scene, or come up with some other strategy for managing your assets. This package makes no assumptions about how you use AssetBundles, but it sounds like a basic example of building/loading is something we could add as an improvement. In the meantime, I think this should get you started.

In the editor:

using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEditor.Build;
using UnityEngine;

namespace Unity.RuntimeSceneSerialization
{
    static class BuildAssetBundles
    {
        [MenuItem("Assets/Serialization/Build AssetBundles", false)]
        static void Build()
        {
            var builds = new List<AssetBundleBuild>();
            foreach (var selected in Selection.objects)
            {
                if (selected is AssetPack assetPack)
                {
                    var path = AssetDatabase.GetAssetPath(assetPack);
                    if (string.IsNullOrEmpty(path))
                    {
                        Debug.LogError($"Could not get asset path for {assetPack}");
                        continue;
                    }

                    var guid = AssetDatabase.AssetPathToGUID(path);
                    if (string.IsNullOrEmpty(guid))
                    {
                        Debug.LogError($"Could not get valid guid for asset at path {path}");
                        continue;
                    }

                    // Check existing asset bundle name to avoid overwriting it
                    var assetImporter = AssetImporter.GetAtPath(path);
                    if (!string.IsNullOrEmpty(assetImporter.assetBundleName) && assetImporter.assetBundleName != guid)
                    {
                        Debug.LogError(assetPack.name + " is already part of an AssetBundle, and cannot be built to without overwriting its AssetBundle name. You need to temporarily set its AssetBundle name to None in the inspector in order to build this asset.");
                        continue;
                    }

                    builds.Add(new AssetBundleBuild
                    {
                        assetBundleName = path,
                        assetNames = new[] {path}
                    });
                }
            }

            var outputPath = EditorUtility.SaveFolderPanel("Build AssetBundles",  new DirectoryInfo(Application.dataPath).Parent.FullName, "Bundles");
            if (string.IsNullOrEmpty(outputPath))
                return;

            Debug.Log("Building AssetBundles");
            var manifest = BuildPipeline.BuildAssetBundles(outputPath, builds.ToArray(), BuildAssetBundleOptions.None, EditorUserBuildSettings.activeBuildTarget);
            if (manifest == null)
                throw new BuildFailedException("Failed to build AssetBundles");
        }

        [MenuItem("Assets/Serialization/Build AssetBundles", true)]
        static bool ValidateBuild()
        {
            if (EditorApplication.isPlayingOrWillChangePlaymode)
                return false;

            var isValid = false;
            foreach (var selected in Selection.objects)
            {
                if (selected is AssetPack)
                {
                    isValid = true;
                    break;
                }
            }

            return isValid;
        }
    }
}

Note the error about assets already having an AssetBundle name. I added this check to prevent this from interfering with any existing AssetBundle setup you may already have. This code would clear out those names otherwise and potentially break things. This example of a manual context item serves as a fine example to walk you through the process, but your ultimate solution will probably be a little different. Instead of building separate bundles for each asset pack, you may want to combine them or name them slightly differently.

And in your game/app, something like this:

using System.IO;
using UnityEngine;

namespace Unity.RuntimeSceneSerialization.Test
{
    class LoadSceneWithAssetBundle : MonoBehaviour
    {
#pragma warning disable 649
        [SerializeField]
        string m_ScenePath;

        [SerializeField]
        string m_AssetBundlePath;
#pragma warning restore 649

        // TODO: Fix initialization bug to allow loading from Awake
        void Start()
        {
            PropertyBagOverrides.InitializeOverrides();
            var sceneJson = File.ReadAllText(m_ScenePath);
            var assetBundle = AssetBundle.LoadFromFile(m_AssetBundlePath);
            var assetPack = (AssetPack) assetBundle.LoadAllAssets()[0];
            SceneSerialization.ImportScene(sceneJson, assetPack);
        }
    }
}

The important things to remember here are to call PropertyBagOverrides.InitializeOverrides once before doing any scene imports, and to wait at least until the end of the first frame, otherwise you will hit errors because property bags haven’t been initialized yet. Going through this example actually made me realize we can probably fix this, so it was a good exercise! I’ll try to get these sample scripts into the package soon so users don’t have to come here to find them.

This was a very thoughtful reply @mtschoen , thanks for helping especially with the code samples and consideration for those integrating into more mature projects! Agree that example or perhaps a video might help folks. I think I have following questions still, hopefully they’re simple to answer

  • I realized menu item “File->Save JSON Scene…” produces a .asset file in the root of the project’s Asset folder. Apparently this is the asset pack that I just need to have selected in the Project pane before I click the menu item “Assets->Serialization->Build AssetBundles”. I don’t think there’s a need to make a separate asset pack - I can just select that produced one (as least until my scene changes)

  • When I use menu item “Assets->Serialization->Build AssetBundles”, I’m creating a “Bundles” folder inside my root Assets folder and just telling the dialog to use that. It produces the following file structure. But when I put breakpoint from code taken from your LoadSceneWithAssetBundle class, it suggests the result of

m_AssetBundlePath = Application.dataPath + "/Bundles/Bundles";
var assetBundle = AssetBundle.LoadFromFile(m_AssetBundlePath);
assetBundle.LoadAllAssets()[0]```
is not of type AssetPack but rather AssetBundleManifest. is it obvious what I've done wrong ?

<path/to/project>/Assets/Bundles
│ assets.meta
│ Bundles
│ Bundles.manifest
│ Bundles.manifest.meta
│ Bundles.meta

└───assets
scenename.asset
scenename.asset.manifest
scenename.asset.manifest.meta
scenename.asset.meta

I think we’re almost there!

Yes, when you do File > Save JSON Scene... this process will create an AssetPack and save it to an .asset file next to the scene with the same name as the scene. This is an asset representation of the AssetPack ScriptableObject which was used to collect asset references when serializing the scene. If you select this asset, you can use the Inspector to see what asset references were collected when serializing the scene. When you build an AssetBundle, these dependencies get included automatically. You are right that this is what you should have selected when building AssetBundles.

The directory structure created when building a single AssetBundle can be a little confusing, but the file you want from what you pasted is Bundles/assets/scenename.asset. That is the particular bundle that corresponds to the AssetPack you had selected. The path to each AssetBundle within the Bundles folder (or whatever you decide to call it) will match its path within the Assets folder. You can ignore the other files that are generated. If you don’t put the output folder in the Assets folder, you also won’t see the .meta files. Note that you can also make multiple selections and build them all at once.

Unfortunately, the ScriptableObject for the asset in the Assets folder and the AssetBundle you build have the same extensinon: .asset. But they are distinct representations of the AssetPack, and both are needed. There won’t be any files called .assetbundle or .assetpack in the process, just .asset. The original scenename.asset` file, which lives in the Assets folder is simply a list of asset references, with their guids and fileIDs stored as strings. If you use text-based asset serialization, you can easily read its contents in a text editor. The AssetBundle representation is a binary file which includes the AssetPack with those strings and asset references, but also the assets themselves transformed into whatever platform-specific representation is needed in order to be loaded in the Player. The same stuff that happens to your models and textures when you click “Build” for your app is happening when you build AssetBundles.

It is important for the ScriptableObject .asset to remain in your project after the scene is saved. First of all, you wouldn’t want to build AssetBundles every time you save the JSON scene because, as you have seen, it takes a lot longer to build the AssetBundle than it does to save the scene. Leaving the asset in your project after the scene is saved allows you to save a number of scenes, then select their AssetPack assets and build bundles.

Furthermore, you need to build a different AssetBundle for each build target and save/load them in separate paths, etc. In other words, if you want to build your app for iOS and Android, you need two AssetBundles for each AssetPack: one built for iOS and one for Android. Leaving the ScriptableObject asset in the projects allows you to switch build targets and do your build as many times as you like without having to open the scenes and serialize them again. This is one of the main reasons why our simplistic example here is only going to get you so far. You would need to use a separate path for the iOS vs. the Android bundle and choose which one to use based in the current platform. And, of course, just working with the filesystem on mobile devices can be tricky–just trying to get the AssetBundles onto the local filesystem where the app can access the can be a challenge.

You probably don’t want to put your AssetBundle output directory in the Assets folder. With the code I shared, it should default to the project folder, not Assets folder. That’s what the new DirectoryInfo(Application.dataPath).Parent.FullName was for. The Editor doesn’t need to “know” about the built AssetBundles; they are only used by the Player. You can avoid getting those extra .meta files generated, and Unity will spend some time importing them having them in Assets will bog down your project.

You are also right to think that you’ll need to re-build the AssetBundle every time the scene/AssetPack changes. If a new asset reference is added, you will need to rebuild the AssetBundle (remember, one per-platform) in order to reference the asset. If a reference is removed, you probably want to rebuild to make the bundle a little smaller. This is why, in the case of the Companion App workflow, the Resource Manager helps track what scenes have been updated, and we use a cloud service to transfer files around. Doing it all “by hand” isn’t really sustainable, so you will want to take the building blocks you see here and integrate them with some sort of content delivery system. You may also want to consider using Addressable Assets to provide a consistent reference to the AssetPacks in your project. Or you may find that it is more helpful to create custom scripting to combine all the asset references you need into one giant AssetPack. It depends on what you are trying to create.

At this point it may have become clear why we consider this package experimental. As I explained above, the primary purpose of this package is to support the AR Companion App workflow. This allows us to focus on building out a workflow that serves AR developers while incubating this general-purpose scene serialization functionality. It goes beyond the scope of the AR Companion project to support users with their own serialization workflows, but It’s great to hear that some are trying.

Very generous reply, thanks again @mtschoen ! Think I’m on the final stretch now; I’m seeing many “could not find GUID” and “could not find asset metadata” warnings in the console when I try serializing the scene from play mode. In AssetPack’s GetAssetMetadata() method, I’m noticing that
m_AssetLookupMap.TryGetValue(obj, out assetData)
returns false seemingly only because the key in m_AssetLookupMap that looks like it should match has a different InstanceID than the obj passed in. I imagine since the dictionary lookup relies on this equality definition between Unity Objects that we expect the InstanceIDs to match. Any idea why in my case they might not? I’ll keep debugging but figured worth asking if you’d seen that during your development too.

Do you already have a fully-populated AssetPack for those assets? Even though I think we technically could still look up guids for some types of assets in Play Mode, we don’t because Play Mode is intended to be a simulation of how your app will behave in the Player. If we “cheat” in Play Mode, it would hide the fact that the same behavior in the Player will fail. One way or another, any time we need asset metadata, we need to be in edit mode.

In other words, you shouldn’t expect serialization to work in Play Mode unless you already have an AssetPack containing the asset references you need. If you are trying to serialize a brand new scene that was built in Play Mode, you will need to provide an AssetPack that has been created at edit time containing all of the potential assets you made available in your editor. In other words, if a user can place a tree in the scene with a mesh, a material, and a texture, that tree had to come from somewhere, and wherever you set that up, you need to generate an AssetPack with those references. Using Prefabs can be helpful, but you will also need to instantiate those prefabs with a PrefabMetadata script attached containing the guid of he original prefab. This is where IPrefabFactory can be helpful. We use that in the AR Companion app to provide extra prefabs for users to play with that are bundled up separately from the scene. This prefab list contains its own mapping for prefab-to-guid.

It depends on what the object is and what you are trying to use it for. For the same reasons I described above, there is no direct equivalence between instances of objects in Play Mode and Edit Mode; Play Mode is a sandbox. I think the issue you’re seeing the fact that you have some references to assets in your scripts, which you use to create instances in Play Mode. Then you want to serialize a scene that references these instances. I think that maybe if the AssetPack is also built into your scene this may work, but it doesn’t allow you to change the AssetPack after the build, so I wouldn’t recommend going this route.

The most straightforward way to use this package is for your workflow to start in the Editor, so that you can build your scene as normal, pack it up with all its assets, and then let users modify it within a Player build, without introducing new assets after the fact. That way you can either do what I described and just build the AssetPack right into the build, or keep the scene external, and start by deserializing the JSON scene in the player along with its AssetPack (loaded via AssetBundle), and then you can be sure every asset reference came out of the AssetPack, and the instance IDs will all match.

If you really want your users to start from scratch, you can pack a bunch of prefabs and assets into an AssetPack by either serializing an arbitrary scene containing them all, or writing an Editor script to manually set up an AssetPack. Then you can list out the included prefabs in your UI somewhere and let users pick and choose from them.

Just remember that, fundamentally, you always need an AssetPack in Play Mode or Player builds that contains references to the assets you need, mapped to their metadata (guid and fileID). This applies to both serialization and deserialization. Both processes need to be able to either convert an object reference into its metadata or metadata into an object reference, and the AssetPack is where they get that mapping.

Hi there!

I wanted to share a quick update on this package. We just released the 0.3.5-preview version which contains some bug fixes as well as an AssetBundle Sample which includes the scripts I shared above. We’ve also improved the documentation to describe how to use the sample and provide more information about how to use AssetPacks with AssetBundles within your project. I hope this helps!

1 Like

Hey awesome progress on the package!

I’ve ran into some issues now. It seems UnityEvents with parameters cannot be serialized. Not a big deal for me but maybe good to know (let me know if you want me to open a ticket, not sure how this stuff works with preview packages).

But another more blocking problem is that I get an error when serializing scene on android builds.

I get ‘Missing property bag error’ regarding a type that’s a compiled monobehaviour (dll). This reference has been added to the AOT Code Generation Settings. Is this expected to work? It works fine in editor at least, I’m going to try to decompile the class and check again (it’s a third party asset).

I did try to call RegisterPropertyBagRecursively with the type during editor mode but it doesn’t respond with anything (no console logs) so not sure if I’m using it correctly.

Any help much appreciated!

Hello again! I can confirm that if you have a compiled dll that contains a serializable class you can’t serialize it properly on Android targetting IL2CPP (maybe other targets doesn’t work either, but we have to target IL2CPP so I can’t verify that).

After decompiling the dll containing the monobehaviour into a regular source file serialization worked, so it seems compiled dlls are the problem.

Would be awesome if this could be supported, but I’m not aware of the technicalities of doing this, in the meantime we’ll start doing our own custom serializer for the troublesome class.