After spending a substantial amount of time creating a useful build system (command line build-all, build-target, etc) and making all the hidden code settings mouse-clickable, I added a basic asset bundle building system. Then it grew. Now you can just click a couple buttons and have a player and asset bundles based on a folder structure.
It has:
Automatic build versioning (manual or auto-increment or UTC time ticks)
Asset Bundles produced with hashes, so bundles that don’t change don’t need to be downloaded again
A custom bundle Manifest per version that your built player loads to know what bundles to download
A fully working loader scene with a smooth progress bar, with everything async loading behind it
Automatic bundle cache management, so old data is deleted and only new bundles d/l when needed
Command-line build handler for individual platforms or all platforms
Individual build settings for bundles and player options stored per platform, easy to check into version control
Allows partial asset name matches so you can actually rearrange where files are in different bundles without having to change any of your code or data.
What’s unique about it?
Bundle builds are great, of course, because the loading times are much faster than using single-threaded loading in the Editor. But building bundles can take a LONG time on a big project. So, I built a differential bundle system that lets you download (or build) a set of bundles by version, then click one button and AutoBuilder generates an asset bundle with all the new assets packaged in it, which it uses automatically when you run in the Editor. No coding required. It’s seriously amazing for iterative workflow, because in a big project, the load times can be much longer than the time it takes to make a small override bundle.
So… I think it’s pretty amazing. Obviously, or I wouldn’t have bothered packaging it all up for you guys. I’m using it in my own projects, and expect there to be small changes and fixes as time goes on.
Any comments or questions? I’m happy to answer them.
Super excited to announce that AutoBuilder hit the asset store yesterday! Anybody looking to use out Asset Bundles, or wants a complete multi-platform command line driven (or menu driven) setup, with automatic caching on startup… check it out!
Even more useful is a way to work in the editor using asset bundles all the time, with a quick-to-create override bundle.
I just pushed a small fix to AutoBuilder that has corrected paths for the package so everything lives underneath the project folder instead of having a few things outside it.
Turns out that several people are buying multiple assets from us, and some of the folder names were the same under the company folder. So I cleaned up the structure a bit. While I was in there, I slightly improved the documentation and fixed a minor bug in the code.
I was just starting a new prototype the other day and pulled Split/Scenes and AutoBuilder into it. The coolest thing was, in about ten minutes, I had a root scene with several child layers with different stuff on each that I could use in-editor, then dragged all the scenes to the Bundles folder, configured the AutoBuilder profile, built the prototype. No kidding, in ten minutes, I had an executable that didn’t have any direct linkage to the game (the root scene is loaded by name), asset bundles asynchronously downloaded automatically from a CDN, the root scene async loaded and when it launched, Split/Scenes automatically async loaded all the child scenes and started them. I didn’t have to declare the scenes in the build other than the loading scene that comes with AutoBuilder.
It was pretty epic! As a framework for building asset-bundle-based games, this combo is definitely a huge win.
I’ve just come across this, it seems amazing and I am very interested. My game is massive, and it’s all in one scene. Is it possible to use this to make asset bundles easily without having to tag all the different assets Ive used? Can I feed it my scene to do the work? I would really love to get my asset bundle workflow sorted once and for all and this seems to be the ticket… Are there options to automatically limit the size of the bundles by splitting them up etc? My scene is 300meg and I believe the assets in use are about 100gig so it’s a pretty challenging project to work with… would love to know if you think this asset can help me out!
edit: PS. my assets are spread all over in MANY MANY messy dirs, will this be a problem? I mean it could take me until next xmas to manually specify them, which is why im looking for automation.
Sounds like a pretty daunting task. Before promising you the moon, I would ask what you expect asset bundles will do for your massive project? Do you need incremental loading as you move across space, or the ability to change content after shipping without updating the whole thing?
The AutoBuilder asset makes organization really simple. Just specify the root folder and each subfolder becomes a bundle. That takes care of your organizational effort, for the most part. The issue will be if you have cyclical dependencies somewhere, it will fail (and tell you what the problem assets are). Assets can depend on things in other bundles, but one must load before the other, and if A depends on B, B loads first, then A. If B depends on something in A, that object will not load properly. So AutoBuilder tells you, at which point you have to move some files around (or factor out shared files into their own bundle).
What you do with bundles once they are created is up to you. So roaming and loading CAN be done, but that isn’t something I can do for you. Loading is a single line of code though, so basically trivial to script if you can figure out how to organize bundles spatially. AutoBuilder does come with a loader scene that simply loads everything for you as an example.
The build process itself is pretty smart about not building unnecessarily, and uses a slightly different manifest file layout so that you can have different versions of your game hosted side by side without duplicating bundles that haven’t changed. And you can run bundles entirely locally without hosting anything just to kick the tires on it and test your workflow. The fancy feature nobody else has is an override bundle that generates only the deltas from a specific build version into one bundle which let’s you keep testing without having to do a full rebuild until you’re ready.
If you have specific questions, I can answer them, though.
Hey, I may have over estimated how much memory will be in use. I load the scene entirely into the memory and it runs just fine in the editor, so I think it uses about 16 gig of RAM on high quality (I think my build would hopefully be less than 20 gig). I am targeting PS5/High End PC ultimately so I don’t mind about the RAM, ie I dont need to load stuff in and out of memory (yet)… I turn on and off parts of the scene as needed but everything is pretyt much in memory as soon as the scene finishes loading.
The main reason I need asset bundles (in order of importance) is:
A) I have exceeded the hardcoded 4 gig texture limit allowed in a scene. I don’t notice in editor but if I build out my game all the graphics are corrupted. This is a known unity issue that can apparently be completely avoided by using asset bundles.
B) Updating the game without having to get it all again, as you said. There will no doubt be many changes to all kinds of assets, I guess it would be nice to keep asset bundles lower than say 1 to 5 gig so new ones can be grabbed relatively easily.
C) Speed, I am hoping the time for the game to play may speed up, as right now it takes about 5 minutes to start after I tap play.
I guess my main question is:
how much manual work will be needed on this massive and very messy project. The root folder I would need to specify would simply be assets/ . All my content is illogically spread across many subdirs etc over the last 8 years. There is A LOT (200gig) in there that I don’t need in my build. I need some way of using the scene file to work out which things to include in my bundles automatically, it would be impossible by hand I would say (since theres no way to just tag objects in the hierarchy).
Well, I have good news and bad news. The good is that bundles are fast to load, cache well, and generally solve the post launch updating problem really well, after all the work that I went through to make AutoBuilder do a good job caching, I mean.
The bad news…code does not go into bundles, ever, so you would have to make a proper patch for that. Also, AutoBuilder does a lot of things, but it does not strip assets for you. Bundles are literally everything you tell them to contain, which is up to you to pare down.
Even as easy as AutoBuilder is to generate bundles, it is definitely a manual process you have to do. In the case of explicit references, dragging whole folders into bundle folders, unity will not break because it finds assets via guid, as I’m sure you know. That’s why I built AutoBuilder such that organizing files on disk IS the bundle structure.
But if you’re looking for something to remove unneeded assets, look into some of the build report tools out there that can tell you what is not referenced anywhere, based on what doesn’t go into the build. I think one was called AssetH Pro? I haven’t used it, but it sounds like something you might look into.
Once you get down to a definitive set of assets you actually want to make into asset bundles, that’s when you want AutoBuilder.
I am starting considering if I should use Addressables or AssetBundles. At this point I have too little knowledge on that to decide, but your AutoBuilder seems to be close to what I need, so I’d like to learn more on that. I have a similar problem as radiantboy, but I also need to be able to build quickly different versions of the same app, but with different data and object configurations (positions, textures etc.). I will have many app templates and will build those apps after providing small customizations based on customer requirements. My current approach is that I have many scenes which use reusable prefabs. Then, once I get customer’s data and requirements, I will change the app and build it.
Initially, I will need a solution that will speed up loading the first scene and will load the main one later. For example, the first scene would load with some low resolution textures and then either a bundle would load from a local drive (ready customer app) or from internet (demo app with demo data).
Without asset bundles Unity itself recognizes what is used by the scenes and includes only needed assets in the build. Correct me if I am wrong, but your AutoBuilder will not recognize automatically what is actually used, will it? It will just create bundles based on folders even if some assets are not used by the current build. If you agree, then it will cause similar problem as Resources folder that I stopped using for the same reason - the build always includes assets in that folder. If you agree, would it be difficult to develop this kind of auto detection?
Finally, how would you compare your AutoBuilder system with Addressables? Does it have some advantages? Does it change anything to the diagram on Complexity/Development time included in the below document?
On vacation right now so I don’t have a laptop in front of me to answer more thoroughly, but I will try to hit the high points quickly.
What it sounds like you want is something to generate bundles based on a set of root assets and their dependencies, which is not what AutoBuilder does. Unity builds do that naturally, as long as you add a scene to the build and out the prefabs into that scene. I recommend you could easily get what you want by having the last scene in your build have all your resources in it and just never launch that scene. Then your built client is fully self contained and has all the data and only the data you need.
Could I write something to flag unused assets? Yeah. It might be a bit more of a manual process to decide if you wanted to delete them or move them away from the bundle folders or something like that. I don’t know at this point how it would work. But walking dependency trees is easy. The problem is aasets can be loaded by full pathname, (or in AutoBuilder by partial path), and there is no way I can determine if that functionality is going to be used, so I have to assume everything in a folder is necessary, unless you tell me otherwise by calling some menu operation indicating how to remove unrelated assets.
As for addressables, I can’t really say. It seems like it is similar to how I have implemented partial path loading, where the bundles you mount can be reorganized without changing the code that loads assets. But I haven’t spent enough time looking at the way addressables load or are directed to cdns. Someone somewhere has to create a manifest of where files live and where to fetch those addressables, so the process is going to be similar to building bundles. Maybe they are similar to if you were to make each asset and it’s dependencies a single bundle? Maybe it’s smart enough to fold each subdependency together if there is a texture in multiple places? I can’t say.
Will AutoBuilder do what you want out of the box? Some of the things you want, yes. Some no. I hope this helps.
I’m excited to announce that a new version of AutoBuilder hit the store today.
What’s new?
A couple of minor issues fixed with the override bundle generator during tight iterations.
Added support for all the new build features Unity added up to 2019.3, and the new platforms too.
Added a toggle to enable/disable logging so you can see exactly what is going into each bundle.
Added 3 flavors of simple filename based filtering: Exact Path, Contains, and EndsWith. You can do just about anything (short of regular expressions) with this. Filtered files also show up in red in the logs, so you can catch accidentally filtered files easily.
I hope you all enjoy the update and look forward to suggestions, bug reports, and positive reviews!
Quick question, does AutoBuilder work with the MRTK package installed. Developing industrial app for HoloLens 2. I don’t want to spend forever working on this project. Thanks.
I don’t see why not? All the C# code for AutoBuilder is completely namespaced, so there won’t be any conflicts with other code. You can move the folder and it’s tolerant of that.
Have you bought AutoBuilder and are running into a problem with it? If so, shoot me an email at support@reachablegames.com with details.
My company has multiple teams and partners along with external suppliers all of which provide somewhat convoluted materials in information. It goes through a scrubbing process and into databases. This app will be delivering a fairly diverse visualizations process for the IoT data. I need to load scenes (hopefully), and at a minimum objects. From the description your tool should help me accomplish this. I will be purchasing it today. If you are interested, I will let you know how it goes. Thanks for the reply and the honest assessment. BTW my handle is my linkedin name.
Yes, indeed, AutoBuilder will allow you to load a scene from a remote-loaded asset bundle and anything stored inside it. Technically, Unity performs this feat, AutoBuilder just makes it really easy to generate the bundles. One thing to note, the executable must already have all the C# code compiled into it before trying to load any bundle that has code bindings, otherwise it won’t work properly. That’s a limitation of how asset bundles work, not something I can fix.
Unfortunately, I don’t know anything about how purchase orders are fulfilled by the Unity Asset Store. A quick search didn’t turn anything up. If you don’t find what you need after contacting the store (support@unity3d.com), reach out to me directly and we can discuss the best way forward for you.
Purchased, downloaded/imported. Followed initial setup instructions. With Unity 2019.4.10f1 I had to build the editor configuration for AutoBuilder to work.
I will test this for a while and see if it is something we will be able to deliver to our government customers. Some as you might imagine are not able to establish Internet connections or to purchase from an AssetStore. I really hope this will work for us.
Can you explain further, C# scripts must all exist in the app at compile time and be delivered as part of the packaged app? That seems to limit the utility fairly significantly.
For anyone using this tool with current Unity releases, 2019.4.x, if you have any problem, before doing any troubleshooting follow the instructions to use Unity main menu: Tools/ReachableGames/AutoBuilder/Build [Current] for Editor. So far this has worked for me to get things working as expected. I am at Unity 2019.4.10f1 LTS.
