I’ve tried many different approaches, but couldn’t load a standard image file (cubemap panorama JPG) at runtime as cubemap. It’s easy to do in editor, just import as cubemap, drag’n’drop to scene and voila - you have a new skybox. Which takes around 16 times more space than the original JPG. And I have several skyboxes in my app.
To avoid producing a monstrously big app, I wanted to load the cubemap at runtime. But there’s no runtime function to load a Cubemap image.
I’ve tried loading it with Texture2D.LoadImage, which works fine, but then couldn’t convert it to Cubemap. It’s weird that Cubemap object doesn’t have LoadImage function, or any kind of conversion function.
So, is it possible in Unity 2023.1 (built-in pipeline) to load a cubemap from JPG at runtime?
The Cubemap class has SetPixels/SetPixelData that allow you to feed data to each individual face of the cube map. You can use this to load the cube map from any JPG/PNG image at runtime.
Thank you your answer, but I was trying to avoid doing that. Cubemaps often use the equirectangular projection (single-image), like so:
Which means the loaded texture doesn’t have 6 sides. So I guess one would have to do the following:
Preprocess the image and convert it to 6-sided cross-cube or in-line version. This would cause loss of quality or increase in size, depending on the format.
Add a runtime component that converts the image to 6-sides on the fly. This might cause scene-loading delays, depending on the implementation.
Considering that Unity already has those routines, and I assume they are very optimized but not publicly exported, it seems kinda redundant that every developer that wants to save on space has to make their own unique version. It’s not quite in the spirit of reusable coding.
Converting from equirectangular to cubemap is fairly easy: apply the texture to an aptly uv-unwrapped sphere, place a camera at its center and call RenderToCubemap.
Is your goal to reduce the memory footprint of your application?
Then you can also use AssetBundle, which is very useful because it can load Cubemap, many assets and even Prefab at runtime.
(However, its usage is a bit strange, so you will need patience to adopt it).
Thanks, that sounds like an useful feature. But I’m actually trying to work around the failure of Unity to use JPEG files as textures in their native form. Along with the failure of desktop graphics cards to implement the ASTC texture compression, which is offering compression similar to JPEG. Either of the two would be very useful for modern desktop games and 3D apps.
I was always curious why are games so enormously huge. Turns out, mostly because of textures. And by that I mean chronic lack of proper texture compression. Only mobile platforms seem to care about that problem, but it’s a problem on desktop systems and consoles as well. In the meantime, I’m trying to find a workaround for my projects…
You also have the addressables system, a higher-level system built on top of asset bundles with the intent of automatically managing things like asset dependencies and versioning.
JPEG is designed to reduce file size, with little focus on compression/decompression speed or access efficiency. GPUs need to be able to have fast random access to a texture’s contents, so they typically use block-based compression algorithms like ASTC or DXT that can be decompressed on demand (meaning that to access a specific texel, only the block where it resides needs to be decompressed).
No existing GPU can deal with JPEG directly as in its general form it just doesn’t meet the needs of a GPU, so it makes zero sense for Unity to support it (except when importing assets into the project!).
ASTC works, behaves and performs very different to JPEG, not sure why you’re comparing both.
Compression has been a solved problem for quite some time, there’s tons of efficient compression algorithms for GPUs and Unity supports most of them: BC4/5/6/7, ASTC, ETC, DXT, etc. Note that support varies across GPUs, so Unity will do its best selecting a runtime compression format.
Ah, I see, I understand what you want to do, but I think it is too hard road.
Converting JPEGs to textures is a very expensive process, and I am afraid that if you go ahead with this policy, you will end up with a “monstrously slow startup app”.
I don’t expect GPUs to work with JPEG in real-time. I expect JPEGs to be a delivery method, which is then converted to a GPU-friendly form (or simply left uncompressed). So that game can be small.
ASTC can actually achieve pretty good compression with relatively good quality AND it works directly with GPUs. But only with mobile GPUs. For a reason unknown to human species it doesn’t work with desktop GPUs. Hence, desktop games have to upload/download/install enormous amounts of data, contributing to global warming, coral bleaching, and extinction of polar bears.
Exactly so. But I don’t intend to use that for all textures, just a few critical ones. Which are currently occupying a lot of space, but are only used once per map. Loading a map takes some time anyway, so a few seconds more will not cause much problems. JPEG format was invented when computers were much slower than today, so it loads pretty fast. I don’t even need to convert it to DXT or anything, it can be uncompressed in VRAM.
This is quite easy to do, use ImageConversion.LoadImage. It accepts both png and jpeg files and converts them to textures at runtime. You can use it to (for example) download jpeg files from a server and convert them to a useable texture, to avoid shipping large textures with the game.
Note that both addressables and asset bundles solve the issue of shipping huge games and adding content after deployment in a more general way: they compress all bundled assets using LZMA, can take care of stuff other than textures, and have automatic local caching to avoid re-downloading things unnecessarily. You can use these ready made solutions, or roll your own stuff specifically for textures if you want to have finer control over quality vs compression ratio. I’ve used all 3 methods in the past (download/cache jpeg byte array & load image, use raw asset bundles, use addressables), and imho addressables yield the best effort/benefit by far, though they can be difficult to get running at first.
I have no problems loading JPG (or PNG or EXR) images. I have problems converting those images to a Cubemap instance, WHEN the original image is equirectangular single-image texture.
Meaning, I can load a JPG file that uses 6-sided format (either as 6 separate images, or a single very wide image) and then create an empty Cubemap and copy pixels side by side.
However, there’s no way to create a Cubemap instance and then assign it an equirectangular texture at runtime, without applying a relatively slow conversion formula. Which should certainly be implemented as a GPU shader. Hmmm, maybe somebody actually made such a converter…
LZMA is a lossless compression algorithm comparable to PNG compresion, which can’t reduce the size as much as lossy JPEG compression (high 90% quality). There are textures where PNG or even EXR cannot be avoided (those with gentle gradients) but there are many textures (such as ground textures, or cloudy skybox) where JPEG compression artifacts are invisible, while compression gain is extreme.
I think Unity should just export the cubemap conversion functions that are already present in the engine, and let the developers choose which ratio of speed vs compression they wanna use. In my particular case I only need to load one very big texture every few minutes. So the time spent on JPEG decompression (cca. 1 second) is really negligible compared to the saved space (32 MB as standard texture vs 2 MB JPEG).
Multiply that by 50 and you get 1.6 GB of DXTC textures vs 0.1 GB of JPEGs. Quite a difference.
I mean, isn’t it enough to just call Camera.RenderToCubemap, rendering a sphere with equirectangular UVs? It will run 100% on the GPU, and take care of resampling for you. No need to copy pixels in the CPU or do any manual conversion from polar to cartesian coordinates.
Once you have a properly unwrapped mesh it’s just one line of code, I don’t think it gets any easier (or any faster) than that. Only downside is that this is an approximation, but you can have a very dense mesh that gets you arbitrarily close to the exact solution.
Should you need it there’s also RenderTexture.ConvertToEquirect which performs the opposite conversion: takes a cube map and outputs an equirectangular image.
Ah, I see what you mean. To apply the panorama to a dense sphere and then render to cubemap. I could do that, but to avoid any mesh projection artifacts I have decided to do something else:
Manually import all the cubemaps to Unity Editor as top quality uncompressed textures.
Write a script that will convert them to 6-sided JPEGs.
Waste of time on preprocessing, but at least the runtime will be fast. Also, in my experience, 6-sided cubemaps take even less space than equirectangular.
