FBX importer bugs and missing features

Unity Engine
1

Hello, I am currently checking out Unity 5 as a replacement of my custom engine, mainly because of C# scripting and the editor, but so far I have run into many problems as soon as it is gets more complex.

Unity choking on huge FBX files:
When I try to import a 945 MB FBX file (slightly more than 375000 nodes in the model hierarchy), my own engine needs 20 minutes to successfully process everything. I had to let the Unity 5 editor run over night because a few hours were not enough, only to find out the next morning that it has given up. The imported model was empty and the console mentioned some problem about file id boundaries. While the FBX file is complex, it is perfectly valid, so the error lies in Unity itself. I would totally understand this to fail in older 32 bit editors, but in Unity 5 the editor is 64 bit and there is no real reason to fail. Splitting the model is no feasible solution for a few reasons.

Unity merging distinct materials:
With another FBX file, not nearly as complex but with a few badly named materials, Unity just looses a few of them. I have f.e. two different materials that are both named “Material #842”, Unity just merges them into a single material with that name. I would very much prefer if Unity would keep the materials and material assignments as they are, at most with automatic material renaming like “Material #842 (2)” if needed. In the FBX file itself both identically named materials are present and correctly assigned, after all names are only cosmetic and not used to identify stuff in there. The current ModelImporter settings have no options to prevent this merging. A “Material #844” gets removed completely and all references get replaced to a reference to “Material #843”. I have no idea how this can happen, maybe an uncommon way of assigning materials in FBX files.

Forced automatic renaming of nodes:
I had made a small test file with a few nodes that were to be identified by name, some of the nodes had the exact same name. Unity automatically changed the names by appending numbers to make them unique (before calling AssetPostprocessor.OnPostprocessModel on custom post processors, so no automatic way of saving the original names and restoring them afterwards), even for nodes that are in different parts in the hierarchy. Renaming isn’t even necessary, Unity can handle identically named nodes even as children of the same object. While the names can be changed back manually for small objects, it is error prone. For complex objects with thousands of nodes it is far worse. Therefore there really should be a ModelImporter setting to turn this behavior off. Renaming can be turned on by default to keep the current behavior, but it must be possible to turn it off.

Forced automatic sorting of nodes:
I have another issue with my small test file. Unity sorts child nodes alphanumerically (after calling AssetPostprocessor.OnPostprocessModel on custom post processors, in that method the original order is still present) for imported models. I presume that is to keep identical behavior to times when Unity didn’t support arbitrary ordering. But Unity does support arbitrary ordering now since it is needed for the UI draw order. It can be equally necessary for logic attached to 3D models. Sadly, there is again no way of turning this automatic reordering off. So there should be a ModelImporter setting for it, just like with node renaming.

Are these issues known or even worked on? Are test files needed? In that case I have to create FBX files first which reproduce the issues. I can’t give out the problematic files from the first two problems.

Edit:
Found the issue tracker (sorry, first time user here), currently building test cases and making proper issues.

2

Welcome Neomi
Other devs may be able to provide some helpful workflows.
Here is another one.
The problem isn’t Unity - the problem is proper naming conventions. Using proper naming conventions will correct 3 of the 4 issues identified.

3

Could I ask why you’re trying to import a single FBX thats nearly 1GB, thats insane.

I’ve never encountered a workflow or an engine where such a large file is a good idea, even exporting that from the initial 3D Software must take quite some time.

Could you explain what it is and what the reason for needing a single FBX thats so large possibly is?

4

I agree that identically named materials are bad style, but they still are valid since names are only cosmetic (at least in FBX files). Since Unity can’t handle multiple materials with the same name in the same folder, having duplicates from the source automatically renamed would be a far better solution than discarding the material and replacing references to it with references to another material that just happened to have the same name (bad style, but valid). I don’t even propose to alter the default behavior, I just want additional settings to allow for (in my case) better behavior.

As for the other problems where names are the issue, I don’t even think they are bad style. Automatically renaming objects because their name isn’t unique is bad if this is forced. Suppose I have an object Left_Hand with the children Thumb, Index_Finger, Middle_Finger, Ring_Finger and Pinky. Then I have another object Right_Hand with equally named children, why are they renamed? Not because Unity needs to do so, but because a developer decided to implement it this way and didn’t offer an option to turn this destructive operation off. And not only that, the children get reordered to Index_Finger, Middle_Finger, Pinky, Ring_Finger and Thumb. Again an unnecessary (Unity supports arbitrary ordering) destructive operation without any option to turn it off. And again I don’t even ask to change the default behavior, I just want settings that allow to change the behavior and should be easy to implement.

Of course, changing names would be a workaround. As soon as I encounter a problem, I could ask the artist for a fix. The artist that doesn’t live and work in the same city (because collaboration doesn’t require this nowadays) and has already gone home for the day. Takes him maybe a few minutes to fix when he sees my message the next day, a few more minutes to export and a few hours to upload the new export to me while I have already lost an entire day. Or I could spend a lot of money for software licenses that I only need to import, fix little issues that shouldn’t be issues in the first place and export again, which still takes a considerable amount of time. And in some cases I could change every script that relies on naming and/or order with the risk of introducing errors.

So as for “The problem isn’t Unity”: yes it is. I’m not saying Unity is a terrible engine, but it does have its problems. If these problems are acknowledged, they can be fixed which in turn makes Unity better. While it is possible to work around many of these problems, I would rather spend my time differently. Unity should adapt to the users workflow instead of forcing their workflow to adapt. Importing a valid FBX file should just work and properties like material assignments, object names and object order shouldn’t be destroyed in the process.

In a time where 4 GB of graphics memory are becoming more common (the Titan X even has 12) and more and more gaming rigs have upwards of 8 GB RAM (mine at home has 32) this isn’t insane at all. I would say that it is insane to impose arbitrary limits on complexity in 64 bit environments. Sure it can get slow if it gets too complex, but there is no reason for it to not work at all.

In my case, it isn’t even gaming, I am evaluating Unity for a product visualization application. My FBX file is a 150% model of a car. 150% means that all configurable variations are stored in the same file. Imaging the hierarchy for a normal car, each wheel hub has a rim object as a child. In a 150% model, there are 20 rims, all children of an object named f.e. “switch_rims”. If an external logic now sets the switch “switch_rims” to 5, the children at indices 0 to 4 and 6 to 19 get disabled and the one at index 5 gets enabled (here is where preservation of the original order is crucial, and that is not alphanumerically sorted). And since a car has more than one wheel and all rims must be switched simultaneously, all identically named switches must be collected and synchronized (here is where preservation of the original name is crucial, and “name starts with” doesn’t suffice for some of the switches). Now the car has a few chassis variations to begin with, another switch of its own. Depending on the length of the chassis, the wheels are positioned differently, so the entire undercarriage is replicated and adjusted for each chassis.

If you add switches for different steering wheels, transmissions, seats, decorations, optional ash tray and so on, you get to a couple hundred unique switches and thousands if you count every node. Of the more than 375000 nodes in a hierarchy that is more than 30 levels deep, depending on the current configuration there are maybe 5000 to 6000 nodes active at any given time, a rather reasonable number. So, why don’t I just split the file into many smaller FBX files? Well, other than their position, rims are the same across chassis variations, using instances of the very same mesh. The same goes for many other meshes. Despite the big number of nodes, there are less than 8000 unique meshes in the file. Some are only used once, others are used many times (I even saw triple digit numbers) across the hierarchy. If I split the FBX into many files simple enough for Unity to handle, they couldn’t share mesh instances anymore, so each smaller FBX would get copies. I could easily end up with a total of 10 GB of “small” FBX files instead of this “big” 1 GB file. And these redundant mesh copies will stay after the import has been done, eating away even more resources. Despite being a lot less efficient, it would be hell to manage. So, splitting is not really an option.

I have written a custom engine on my own that can handle this kind of data. It can import this file in 20 minutes (first time only, results get cached) and display it fluidly. A lot of work went into my scene graph so that I keep interactive frame rates even when animating the root objects transformation, and it works. But my engine is inferior to Unity in many other ways. I have no scripting integration yet. I have to edit scenes and even materials with a text editor (all XML files). I could easily sink 10 more years into an editor and still not be at the level of the current Unity editor. And lots of other stuff that adds up. As a single developer I cannot realistically keep up with big teams, so I want to switch to a mature engine and concentrate on the custom logic parts. That is where I am coming from.

As of right now (after a few weeks of testing), Unity is no option for me because of these problems. But it is promising, that’s why I do report the problems I have. Not to talk Unity down but to get the problems acknowledged and hopefully fixed.

5

I understand your desire to edit the software to fit your need, but the most effective process is to edit your workflow to get the job done if the software has identified limitations.
Commonly in game development we are asked to modify our workflows to work within the constraints of the technology.

No it’s not. The problem is not being flexible in a process to get the job done.
Uncommon workflow practices - like not having proper naming conventions to start with - is a recipe for disaster when working with hierarchical systems where names of assets aren’t “cosmetic”.
Another uncommon workflow practice is exporting instanced geometry and expecting the instanced information to be kept instead of converted to unique mesh data.

Bad workflow to begin with. Proper naming conventions for left and right should be propagated down the hierarchical chain in this example.

Name changes aren’t a work around - Proper naming conventions should be how hierarchies and assets are created to begin with. The artist in this example should start out with proper naming conventions so when his assets are delivered and used further down the production pipeline - problems like this aren’t encountered.

The fix to work within the constraints of the software. All identically named switches should be identified by association in this example. Right Rear Tire Switch, Left Rear Tire Switch, Right Front Tire Switch.

This is a very interesting issue. It sounds like where ever the fbx is coming from is putting together instanced prefabs which is something that Unity does itself. So importing all the instanced mesh and connection data is being converted not to be instances but actual copies. This isn’t a Unity limitation, rather it’s a fbx limitation. Mesh instances are not kept as instances through a fbx export. They are converted to unique polygonal data.
The instanced mesh and connection data (prefab) you are trying to retain is not retain-able through the export/import via fbx.

The examples you provided in this section

These are common setups within Unity called prefabs.

No disrespect intended here -
Expecting Unity, or even fbx to keep instanced mesh with the same names and the connection data using those non-unique meshes is comparatively like expecting the software to flawlessly import a character, use the proper rigging algorithm, identify all the animation cycles ranges, create the animation state machine with proper layering and blend trees, setup a the character controller, automatically create the blend shapes, auto mask the added bones in the control rig, automatically link the inputs to the desired character actions and understand that all left side meshes are non-unique instances of the right side mesh because the character is symmetrically identical but the animations will drive that data differently based on the rigging information which is also instanced symmetrically - Because that’s how I had it setup in 3D Max so that’s how it should work in Unity.
You may not understand the slight humor in that if you haven’t experienced setting up a character in Unity, but anybody who has experience setting up a moderately complex character in Unity will understand the comparison and understand asking Unity to perform the complete automated “fix this” character setup on import is very lofty unrealistic expectations.

I don’t think you will find any off the shelf real-time engine that meets your desires to “fix” uncommon development practices upon asset import.

May have to consider editing prior workflows to work with the constraints of the technology…

6

And right back again: yes it is. I would use a few exclamation marks here, but those never helped anyone to get a point across.

Renaming the parts in my example with the hands would be easy, but it would still be a workaround to circumvent the destructive operation Unity applies to names and object order. But the hands were only an example, the problem is the big model. Changing the original data is no option here. The names and order are as they are for reasons that are out of my control, they are intended and needed to be this way. Changing the export would be very error prone and take ages. But I guess I’m just not flexible enough to spend the rest of the year modifying a single export to work for Unity while integration into my engine would take only hours.

Your “proper” naming convention is just what you prefer to work with. Other naming conventions don’t imply a wrong workflow, just a different one. So which one is the best? The answer is the same one as for this question: which tool is better, a hammer or a screwdriver? It depends on what you want to do. Use the right tool for the job. And for what I am trying to do, keeping names and object order as they are would be the correct way. But Unity is not flexible enough to not mess them up.

I’ll say once again, the options I am proposing are only options. You would not be forced to adapt your workflow to mine, nothing would change with default settings. And the options are really easy to implement (as long as the internal importer code isn’t totally screwed up), all they need to do is skip the renaming part and skip the reordering part. Unity doesn’t need objects to be uniquely named or to be alphanumerically sorted, you can rename and rearrange objects as you like in the editor. It is just the importer that messes this up. Very clearly a Unity problem, not a “wrong” workflow.

You are underestimating FBX, of course they can be retained. Open any file in your Max version and export it as FBX, there will be an options window with export options (because one setting doesn’t fit all) you have to confirm. Expand the “Include” section, expand “Geometry” and you will see “Preserve Instances”. Check this option and FBX handles instances as such instead of copies, it works fine. It is still an option because not all applications that import FBX files can handle instances properly. Among the problematic ones is Unity, when using instances Unity will loose even uniquely named materials (that is why “Material #844” gets lost in my original post).

But instances are not the issue with the big file I am trying to import. FBX doesn’t assume it knows better than the user what the user wants to do, therefore it preserves the names, order and connections exactly as they were set up on export. If I set one objects name to be “xyz” and a second one to be “xyz” as well, I expect FBX to respect my choice and keep them that way. And FBX doesn’t disappoint me here, it keeps this information. Unlike Unity, which messes it up on import without any need to do so.

No matter how common or uncommon these practices are, they are valid. I don’t want Unity to “fix” anything that isn’t broken in the first place. I just want Unity to not break something that was perfectly fine before. I want the options to not rename objects that are named as intended (even if that isn’t your preferred naming convention) and to not reorder objects that are explicitly ordered as intended (even if that is not alphanumerically). So Unity not breaking stuff, I think that is a very reasonable expectation.

To be honest, I think it is ridiculous how you defend obvious shortcomings of Unity as the way it has to be.

7

Totally understandable - the point is this needs to be taken into account from the start, with the asset creator setting up the original file correctly with proper naming conventions, so this isn’t an issue. - For the next project, not this one.

Each one has a different name and support assets. If there both named screwdriver and I say bring me the screwdriver but I mean hammer I’m not going to get the hammer when I ask for it. Same as Unity. Proper naming conventions are a standard workflow for a reason, not just personal preference or just in game development, all development. There not just “cosmetic”.

My point is - not to be disagreeable, though I think it’s coming across that way. To the point - I don’t want to be editing names and associations in Unity simply because - if I have to re-import again, any/all that work has to be redone. Proper naming should be the same in the authoring software as it is in Unity. So in my view it’s important to have proper names from the beginning - or at the very least just before exporting into Unity.
Setting up a custom weapon swapping system would be really difficult if 10 characters all had Lhand [01] Rhand002.
The importer would not mess this up if the names were straight in the authoring tool.
Understandably I get your point of not wanting to worry about this and have the tool do it for you, but the solution isn’t the tool - the solution is helping the tool understand by using simple workflow processes to begin with.

Thanks for the fbx instance information. I learned something new that I will never use because I deal with complex character hierarchies, but now I know it is possible. Similar to your complex model.
My preference is based on years of development experience, not a desire to make everybody conform.
I have experienced similar issues several times, and one of the simple solutions was to name stuff correctly so software isn’t messing it up. Exclamation point would fit here nicely also.
However you do raise a valid point and I should of emphasized better in my first post. For sloppy asset development it would be nice if Unity offered renaming options upon import.
But then what about for those who aren’t sloppy? They will have to make several un-needed clicks to tell Unity - no my model is already set up correctly, my names are unique and accurate, please don’t interfere with it.

I’m not defending Unity’s short comings, well maybe a little, but only via offering alternative workflow solutions to the issues you are encountering. We have to be flexible in our processes to get er’ done.
If we sat around all day waiting for the engine to solve our problems we’d all be playing duck hunt still.

So what is the solution to your issue? How are you going to get the job done today? I understand renaming all associated parts and maps isn’t an option. For the next complex model - may want to consider proper naming of assets and maps, and setting up all the prefab instancing in Unity instead of expecting the engine to accept external setups.
Heck - Unity doesn’t even understand (without help) vertex based modifier animations which have been around since pre-2000, but that doesn’t stop people from getting an animated bend into the engine without using bone based deformations.
Maybe these links can help.
https://www.assetstore.unity3d.com/en/#!/content/13582
https://www.assetstore.unity3d.com/en/#!/content/22110
https://www.assetstore.unity3d.com/en/#!/content/18268

8

You are mixing a few things up again.

The names are properly and correctly set up exactly as they are intended and needed, there is nothing sloppy about it. And then Unity messes it up on import. Having multiple identically named objects doesn’t make them wrong, forcefully renaming then on import however does.

Guess what would happen if I had the option to turn renaming and reordering off. I wouldn’t need to rename and reorder stuff after every import, they would automatically be the same in the authoring software and in Unity. I don’t want the tool to take care of the names and ordering for me, I want the tool to leave these things alone.

Guess what happens if you have a prefab with unique names all the way through and place two instances of it in the scene. I can tell you, your names are no longer unique. Only the second root object gets automatically renamed, but not it’s children. You will have stuff like “/Enemy/LHand” and “/Enemy 1/LHand”. So while instancing prefabs Unity doesn’t even do what it forces upon my model on import, it is inconsistent. And I very much prefer these exact names. If you know the object name of the character, you automatically know the name of the left hand by appending “/LHand”. If names were unique, you would have to identify the correct hand object first.

And my preference is based on more than 12 years of being an engine programmer. Whenever the engine screwed something up, I fixed the problem. I didn’t force the artists and game designers to change their workflow. Not even when deadlines approached and work time ramped up to 80+ hours a week.

And that would be the current behavior. But since this behavior is forced instead of optional, that sadly messes up my totally unsloppy hierarchy as well.

I have stated quite a few times that the current behavior can be default. Users that want everything to stay as it is now (with renaming and reordering) wouldn’t have to do any additional clicks, they wouldn’t even have to know of the existence of these currently not existing options. The options I am requesting would basically add the “my model is already set up correctly, my names are accurate, please don’t interfere with it” way in the first place. There is no need for names to be unique.

Oh, I did get my job done. Luckily my job wasn’t to get this stuff running in Unity, but to evaluate if Unity is suited for the job. The result is that Unity is not yet mature and flexible enough. The solution is to not use Unity for complex projects in the near future. But I will keep an eye on Unity to see if that changes.

9

Not needed if common naming conventions are used during asset creation.

10

Maybe the naming conventions in my model are uncommon for Unity users because Unity would screw it up, but that doesn’t make them uncommon in a broader sense. If you assume they are not common, you assume way too much. Unity is not the whole world.

Even if these conventions were uncommon and nobody ever used them before, they would still be valid and messing them up is still not justified. But they are not uncommon, far from it. These naming conventions are used by multi billion dollar companies that design and construct cars for example. My model with its oh so impressive 375000 nodes is just a massively stripped down version of the complete CAD model that is roughly 20 times as complex. That is where these naming conventions are used. Care to guess how many engineers and designers (smart people that know what they’re doing) work successfully and collaboratively with this convention? Can you imagine the reaction if you tell them that they are doing it wrong? I could imagine some laughing, some shaking their heads in disbelief and a couple of facepalms here and there.

I don’t care that you don’t like this naming convention. It is valid, proper, precise, exact, intended and correct. It is the right convention for the job, it works and changing it is not an option. As long as Unity screws it up (and yes, this is absolutely a Unity problem), Unity is not an option. And if Unity someday gets fixed in that regard, there is still the issue that Unity completely fails to import anything at all with not even 20% of the required complexity (utter failure with 65535 nodes, I didn’t test any lower).

11

Duplicate names, pre-export prefabs, and instanced hierarchical systems may work for proprietary purposes, but when you shop outside your isolated industry you will find (as you just did) most times additional work is required to get your specific content to work with an off the shelf real-time engine that is used by numerous industries, and mainly game design.
If the engineers will scoff at the mention of changing there setup, someone else will have to do it, to get it setup.
You will not find an alternative off the shelf engine that provides you the flexibility import options either, because there are reasons duplicated instanced geometry is renamed, and material textures are renamed upon import.
Maybe you should gander at Ngrain or some other less streamlined interactive CAD viewers to fit your needs.

I’ve dealt with similar CAD models just as complex for ‘another’ large industry and have experienced similar issues. Refusing to put in the hard work required to get something to work correctly isn’t a reflection of Unity’s maturity.

12

I am astonished by the amount of ignorance. Again you belittle this working (just not in Unity so far) naming convention.

Unity does it, so there must be a reason to do it, so every other engine must do it too. What kind of reasoning is that? Unity does lots of crap that other engines don’t do. For instance (pun intended) this totally unworthy graphics and input setup in a launcher instead of an in game menu. Which other engine does that? There may be reasons Unity does it. Like f.e. not being mature or flexible enough.

I doubt very much that f.e. the Unreal Engine would mess this up. I will try it, not in the next few days but someday. And if something doesn’t work as required, in contrast to Unity I will at least have the option to change it. Unreal does give its users full access to the source code, engine and editor. If you ever looked at another engine than Unity, you might have noticed that.

Similar issues? So these CAD models were perfectly valid, set up as intended and worked everywhere except in Unity? My sincere condolences if that were true, but I doubt it by the way you defend Unity and its shortcomings.

Refusing to put in the hard work, thanks for calling me lazy. What I refuse to do is putting in lots of effort to break something that is fine and working, just so that it does work with that one little broken engine as well.

Excuse my harsh words, I don’t really hate Unity, I still think it has potential as soon as many of the issues are fixed. But I am fed up with that “if Unity can’t do xyz, trying to do xyz must be wrong in the first place” mentality that is so typical of fanboys. If all you can say is how my otherwise working and valid model is the problem and Unity is doing it right (by messing it up), don’t even bother to answer.

13

No - on the contrary they worked as intended for there specific use, but when considering to take them into another environment for additional interactive use, they were similar to your CAD model. And had similar issues with other engines as well, but instead of complaining about a particular engines shortcomings, we just worked it out, and now have the benefit of deploying to numerous platforms not just PC or console, with very little effort.

Enjoy shopping around for another alternative. Good luck with your first person shooter car application - if settling on UE. :roll_eyes: