Hi - back to experimenting with Unity again.
So, now I need find out how to best support more image file formats than the default PNG/JPG, and I’m wondering what the most efficient approach is.
The best (but not very good) way I’ve found so far:
Parse the image file (either using the default .NET codecs, or custom ones). Most decoders will create System.Drawing.Bitmap objects.
Create a Texture2D based on the Bitmap format (size/color)
Create a Color array, write pixels from Bitmap into array, and apply using Texture2D.SetPixels()
This will require an extra copy of the pixel data in RAM (one for the Bitmap, one for the Color array), and is quite inefficient with all the memory allocation that will be going on (and the extra loop for copying data from Bitmap to array).
I’m also guessing that the Color array is a lot less efficient than a plain byte array would be.
Any alternatives to this?
The optimal solution would probably be if a Texture2D could be constructed simply using a Stream and a Decoder, and the next best would be using a Bitmap, but I haven’t found any way to implement that.
Related to this: how does Unity handle device lost events? Are textures automatically reloaded only if they are using the Asset pipeline, or are “custom-generated” textures handled too?
If so, do they have a copy in RAM?
If so, can I somehow get a pointer to those directly so I don’t have to use those clunky Color[ ] arrays?
Both of those would mean that Unity engine has a dependency on System.Drawing assembly, which we don’t want to add (e.g. Unity Web Player ships only with mscorlib). I’m not even sure System.Drawing is implemented in Mono (haven’t checked).
So yeah, your option is to manually decode the image, construct Color array and put that into texture. Yes, floating point color array is not the most efficient, but have you checked whether it actually slows you down? We could add Color32 (for example, with a byte per channel) versions in the future, if people figure there’s a bottleneck.
Depends on the platform. On D3D9, yes, all textures are in “managed” pool, that means probably D3D has a system memory copy (again, you can never know; it can be different on XP vs. Vista, on different hardware etc.). On OS X, I think we use APPLE_client_storage that points directly to Unity’s copy, so the driver should have no extra copies (again, it’s a PC-like platform, you can never know what’s really going on). On iPhone and Wii there are no device loss events by design, I think.
Right - I’m so used to thinking 100% .NET that I didn’t even consider that. A good choice to keep the player’s size minimal.
The creation of all the Color structs does take a fair amount of time. If possible, I’d prefer (the somewhat obscure) int[ ] approach for the RGBA format, with no extra per-pixel object instantiation. (Color32 doesn’t seem to have any public methods/properties?)
On the whole it probably isn’t that bad, it just felt a bit strange and made me wonder if there might be a more efficient way.
But Color is a struct, hence an array of colors is a single continuous blob of memory that’s allocated once. Yes, in C# you do “new Color”, but that does not actually allocate memory for value types (like structs).
I was guessing there’s some kind of initialization or validation code in the Color struct? Raw loop/initialization time removed, the actual setting of
ColorArray[×] = new Color()
is 7-8 times slower than
intArray[×] = val
(Note: the conversion from int to float needed for Color isn’t included. And: timed in .NET, not mono)
Maybe I’m just missing something. But anyway it’s no biggie, I’m mostly trying to get a feel for the inner workings of the engine and the API’s possibilities - I appreciate your putting the time into it!
I realize now that the lack of pointer-based access to the texture actually is a bit of a problem, since a lot of image manipulation APIs require them.
My own pixel access library is built around them, and I was also hoping to use AGG (anti-grain geometry) for some tasks.
While it is possible to use the “fixed” statement to temporarily pin down a pointer to the Color[ ] array, that pointer can’t be passed around to other methods AFAIK.
Is there any chance that the pointer to the unmanaged memory area will be exposed in a later version? I realize it is a security concern for the web plugin, but for the standalone it should be manageable.
Immensely sweet - thanks!
For the web player, maybe I can resort to the much slower, but still workable solution of copying the data to a System.Drawing.Bitmap, then using its BitmapData.Scan0 to get a pointer.
…or are “unsafe” assemblies disallowed altogether in the web player?