How does the iPhone handle hardware support for photos such as those in the users photo library or those used in other non-unity apps?
Are they expanded to the next power of two before being used, like textures in Unity, or is hardware support handled some other way for non-texture images (photos)?
I also asked this question once and didn’t get a satisfactory answer. Everyone will just tell you it has to be ^2 or that they don’t know, but you should make it ^2 anyway.
Power of two is required for generating cube mip-maps and certain compression algorithms. OpenGL used to require this and older graphics cards, but it’s not a general requirement to display an image, as many people believe.
I know that OpenGLES 2.0 allows non ^2 textures (unless they are cube or mip-mapped), not sure about 1.1; they might be required for textures. And I know that CoreAnimation uses OpenGL, but I would be very surprised if all fullscreen images needed to be scaled up to 512x512; I can’t imagine that is the case; there are extensions for pre OpenGL 2 that allow non-^2.
I have seen no reference to ^2 requirements in CoreAnimation, or that memory will get eaten up if you try to look at a non-^2 image. And CGImage and NSImage have no documented ^2 requirements. Also, if you create a ^2 texture from a non-^2 and put it on a generated quad, you will almost always experience blurring due to the roundoff errors in calculating the uvs. I don’t see any blurring of images photos on the iphone os.
So my guess is that, no, they are not required. But, like I said, I never received a concise answer to this question. If anyone knows for sure about how images are displayed, I would also be interested in an answer. It is always good to know what your memory footprint really is and what is happening to your images under the hood.
I was having a few memory issues when I was putting the finishing touches to Vertigo. This was, in part, because I was using non ^2 textures for the backdrops on the front-end. I had 2 choices when it came to fixing this:
Make the textures 256x256 then streeeetch them out to the 480x320 screen, or…
make them 512x512 and have empty space around the edges of the image.
Obviously I chose the second one, and the images compressed spectacularly.
As I understand it, Unity doesn’t actually compress non-power of 2 textures and it does make a hell of a difference in doing the compression.
If you want them to be compressed, they are required.
–Eric
@eric5h5
I would not say compression in general, but cube mip-maps as well as some compression (as I stated in the previous post). In fact, in unity even uncompressed textures are forced to pot. Also, JPG certainly does not require pot, but is viewable on the iphone; what is happening to the image data? How is it displayed?
I am aware of where pot is required, but not so clear on where it is not required; not in unity, but on the hardware; as per the original poster’s question.
@s7arbvck
Unity will certainly enforce power of two (not sure about GUITextures though, no one ever answered that one for me). You are notified of this with compression, but it happens anyway when using uncompressed textures. Make a non-square quad and smack a non pot texture on it and see what happens in memory. It forces the texture to the next pot and calculates your uvs, resulting blurring from the float roundoff error.
A good solution for fullscreen images that avoid 512x512 for each screen is to create a kind of “letterbox”; That way you can fit 2 screens on a single 512 texture with no quality loss if you don’t change the “bars” surrounding the letterbox. The bars don’t have to be black, they can be anything really, as long as it is the same for each screen.
That way, with or without compression, you reduce your memory for screens by nearly 100%.
Hardware (at least pre 3Gs) do require PowerOf2 textures. Always.
Really? Because I’m not using them (I didn’t want to compress the images anyhow) and they seem to display fine on my ipod touch.
What kind of problems would they cause? Random crashing?
@tonyd
if you do not specify a pot, it will be forced to the next pot behind the scenes.
@ReJ
So, on iphone, images in the photo library are either forced to pot before display? Or are they not using the hardware?
I’m just interested academically, I assume pot for all of my textures/images. Although, I am pretty sure that like you said, OpenGLES 2 (3Gs) does not require pot. Will unity support this? It might be a nice feature for GUITextures.
No they are either pow2 or will not work at all.
there is no “non hardware” mode
Dreamora could you please link to a source that supports this statement?
ReJ: The original question dealt with images, such as those in the user’s photo library, and how they are handled. Certainly the iPhone provides hardware support for those without any power-of-2 requirement. I assume your answer is limited to images used as textures on a 3d object in the 3d opengl writing surface provided by Unity.
Are the API’s to handle non-PO2 images simply not available to the opengl writing surface in any 3D app (Unity or otherwise) or is it just that they are not available through Unity?
What if a Unity developer wants to build a 3D app in Unity that also has an image (read photo) library component to display 2d non-PO2 images without having to create a PO2 version with alpha for each image?
Thanks
Its a hardware restriction.
Anything thats handled by the 3D hardware (-> all you see in unity, unity does not offer any cocoa functionality) has to comply to these requirements.
I would expect that the Cocoa based apps itself does this too, otherwise you couldn’t zoom that smoothly if it was all cpu driven.
What they likely do is take the picture and cut it into pow2 pieces that then are shown onscreen for you to interact with.
That also would be the way you do it, as the maximum texture size on the iphone is 1024x1024, additionally it only has 22mb of available graphics memory (24mb in total - the back front buffer)
If you work with regular applications, you just have the benefit that Cocoa does a lot of the dirty work for you.
If you want to develop games, you have to do the work yourself to do it the way best suiting your game and its available resources at the time.
Hm, I’m pretty sure that 24mb video memory was only for the 3G, all others are unified memory. Also, the 1024 limit is not for the 3Gs, but 2048.
And, yes, it is a hardware restriction, but only inasmuch as the hardware support for OpenGL goes, so it’s really an OpenGLES restriction. OpenGLES2 does not restrict images to pot.
Do you have any links to whitepapers, etc. on the pot restriction, even for photolibrary images, or camera images (not pot)? Just out of curiosity.
Well, you just pad (not scale) image to be POT and when rendering setup your UVs to use only a part of a texture where the actual image resides.
We do exactly that for GUITextures!
All iPhone / iPod Touch devices (regardless of generation) have unified memory. However GPU (MBX) can access only 24Mb at a time. Using more than 24Mb per frame is possible, however it will incur a significant cost on a GART side.
It is not really that much of an OpenGL ES 1.1 restriction. Non-POT texture support often is exposed via extensions, if present in hardware.
POT is a hardware restriction for MBX (or could be a driver restriction in the “best” case). SGX does not have such restriction.
Well, I would say that the MBX was designed to support OpenGL ES 1.1, not the other way around. Of course, openGL uses extensions to take advantage of new hardware features (PVRTC for example), but I’m going to stick with my statement.
We are primarily concerned with OpenGLES features, not hardware features directly.
OpenGLES2 does not require an extension to support non-pot, and the SGX supports this. Either way, chicken or egg, it doesn’t really matter.
In any case, it is nice to finally have a credible source state that all images are padded to power of two, even when viewing images in the photo library. That was really the original poster’s question.
I imagine there are a lot of apps out there that take up more memory than their developers planned for. I am quite used to texture atlases to use image real estate efficiently, but what about the non-game developers out there? That must be quite a surprise to realize that every 320x480 UIImage background is being padded to 512x512! That is a significant difference!
Something still doesn’t sit right with this explanation though… I’ll do some research.
Anyway, Thanks!