According to one of the developers, the issue is less severe in unity 2017 vs 2019 for some reason… but… I am working with point cloud data which is typically a file that looks like this
100200300 (it means that there are this many lines of code following) (ASCII BASED - typical and very large)
123.456(float), 123.496(float), 290.23(float), 11 or 12 bit scalar value, R (char), G(char), B(char)
163.42(float), 125.754(float), 103.45(float),11 or 12 bit scalar value, R (char), G(char), B(char)
Binary Based:(a typical but fairly reasonably sized)
00000000101001001001001001001001010(not seperator, but imagine there is)010010100100100101 1001001001010010 10010101010010010 (ETC ETC, No new line endings because binary, its just a long structure file that requires knowing its organization).
** sometimes this float data needs to be written as double float data, and we both know this is bad for unity so have to do extra conversions/math/processing.
**Keep in mind this is a small file for my work. 5gb… meh. most of them are 12-25gb files that need processed.
So first of all there are 2 methods to visualize this data (which is very much similar to mesh data without the lines and faces and indexing… its just vertex3 data as float, float, float, (char/short), char, char, char)
- Converting it into a binary format
- Converting it into a mesh format (mesh filter)
There are 2 converters that come with this particular asset written in C# (I dont know if it is IL2CPP) as most of the C# code is available and runs as is I am assuming not,also assuming that you are referring to the program being compiled into C++ etc once ready for client.
I’ve written a basic C++ Program that performs this same task (similar, not exact because I have to do a lot of extra coding to finish it which I intend to but instead of adding it all to 1 file I created 5 files and parse it out which in theory would take longer), and it does the same task in 1/10th of the time (generally).
Eventually I will need to make a similar version to be run by my clients and most of the competition is running this in C++ or even C (
). Because these data files in ASCII are about 5gb, and 1.x gb in Binary… I need performance to process the massive amounts of data. Keep in mind that these again are small file sizes… which means that customers receiving it would be spending an extra 20 minutes waiting for a single file which could be done in 3 or 4 minutes.
I need a way to integrate it into Unity not only for development/testing… but once compiled I know that it will work and I can go back in make changes and updates and release from all in one place instead of having to create separate instances and files and protocols to do something that would be perfectly easy to do in Unity…
Not to mention… if you could C++ script… you would have a development platform/interface to develop GUIs for C++… that would be a game changer in and of itself. No more WXWIDGETS… (PLEASE).