Optimising 2d sprite game for ipad

[BillyMFT]

hi all,

I’m working on a donkey kong, LCD style game in Unity that will be published for iPad. It’s been going great but as I get to the end I’m starting to notice some lag so I am looking for ways to optimise it.

You can see an small section of the game here: http://www.lcdemakes.com/transparentexample.png

The colour is a flat background image (png), but the other parts are made up of multiple transparent pngs. So in that pic there is the

chest
enemy
hero body
hero attack arms
//etc

There are hundreds in the game, and they get turned on and off to create the frame by frame animations. Just to be clear, the images transparent pngs - solid black for the shapes and then fully transparent - no gradients or anything.

I’ve tried a range of shaders for the sprites but this is the latest

Shader "Sprite" {
Properties {
    _MainTex ("Base (RGB) Trans (A)", 2D) = "white" {}
}

SubShader {
    Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
//    LOD 100

    ZWrite Off
    Blend SrcAlpha OneMinusSrcAlpha 
    Lighting Off

    Pass {
        SetTexture [_MainTex] { combine texture } 
    }
	}
}

Forum member Jessy suggested the shader should be multiplicative but whenever I add that blending to any of my shaders the images turn to solid black and I don’t know enough about shaders to determine the problem.

So, question 1: what would be the most efficient shader to use for this type of sprite? Am I doing it completely the wrong way?

Question 2: given that nothing actually moves in my scene (things just get turned on and off), would there be anything to be gained by setting everything to static? It seemed logical to me but I tried it and it didn’t seem to help.

Question 3: each image is retina res, scaled down to 50% in the scene. We did this so they didn’t have to be scaled over 100% on retina but is this the right approach?

Question 4: I have around 40 sfx in the game. All short, and most played regularly. At the moment I have a sound controller that attaches 40 audio source components and assigns an audio clip to each one, ready to be played. This made it easy to set up their volume and would allow for different combinations of sounds to be played at once, which happens constantly in this sort of game. Is this a bad way to do it? The sounds are a mixture of aiff and mp3 - is one much better than the other when publishing to iPad? The aiffs are all set to compressed. Commonly played sounds are set to decompress on load, less common sounds are compressed in memory.

Question 5: Aside from the main game screen, the scene is made up of a grid of 9 (3x3) big planes, each with a 2048x1536px jpg on it. We did this so that when buttons are clicked (like instructions) we can just zoom the camera over to that part of the grid rather than load a new scene. It looks nice but this seems to be chewing up a bit of memory as well and I often get memory warnings during the pan. Is there a more efficient way to do this effect? Is this likely to impact of the smoothness of the game if these planes are all off camera while the game is being played?

Anyway sorry about asking so many questions. This is my first proper iOS game made in Unity and I haven’t being able to get very far with the profiling tools in Unity and xcode.

Thanks in advance
Bill

[Jessy]

@BillyMFT

Just to be clear, the images transparent pngs - solid black for the shapes and then fully transparent - no gradients or anything.

That’s not what I see in your image. Antialiasing for compositing purposes uses intermediate values.

Your shader is fixed function; you should be using GLSL these days (write it in Cg if you can get it to compile; Unity has been making Cg difficult lately in my experience, in that things won’t compile for unpredictable reasons; YMMV).

@BillyMFT

So, question 1: what would be the most efficient shader to use for this type of sprite? Am I doing it completely the wrong way?

If your sprites are just black, then you can use a multiplicative shader. You don’t use transparency; you use white for no effect, and black for full effect, the same as in Photoshop with multiplicative blending. You need alpha blending if colors are involved.

@BillyMFT

Question 2: given that nothing actually moves in my scene (things just get turned on and off), would there be anything to be gained by setting everything to static?

Can’t offer any advice on this; not familiar with it.

@BillyMFT

Question 3: each image is retina res, scaled down to 50% in the scene. We did this so they didn’t have to be scaled over 100% on retina but is this the right approach?

I don’t know what you mean by “scaled down”. The right approach is probably to use a lower quality setting on non-retina, if only for using “Half Res” textures.

@BillyMFT

Question 4: I have around 40 sfx in the game. All short, and most played regularly. At the moment I have a sound controller that attaches 40 audio source components and assigns an audio clip to each one, ready to be played. This made it easy to set up their volume and would allow for different combinations of sounds to be played at once, which happens constantly in this sort of game. Is this a bad way to do it?

Seems reasonable to me.

@BillyMFT

The sounds are a mixture of aiff and mp3 - is one much better than the other when publishing to iPad?

Each has their uses; use AIFF for SFX unless you’re too low on memory.

@BillyMFT

The aiffs are all set to compressed.

That doesn’t make any sense; AIFF is an uncompressed format.

@BillyMFT

Question 5: Aside from the main game screen, the scene is made up of a grid of 9 (3x3) big planes, each with a 2048x1536px jpg on it.

There should never be a reason for you to be using jpegs unless that’s the best source format you had.

@BillyMFT

Is there a more efficient way to do this effect?

I don’t understand what you’re doing, with words alone.

[BillyMFT]

Thanks Jesse. Comments below.

@Jessy

That’s not what I see in your image. Antialiasing for compositing purposes uses intermediate values.

That’s true. Do you still think it requires a multiplicative shader?

@Jessy

Your shader is fixed function; you should be using GLSL these days (write it in Cg if you can get it to compile; Unity has been making Cg difficult lately in my experience, in that things won’t compile for unpredictable reasons; YMMV).

I’ve always just taken other shaders and made (very) minor tweaks. It’s a pretty steep learning curve.

@Jessy

If your sprites are just black, then you can use a multiplicative shader. You don’t use transparency; you use white for no effect, and black for full effect, the same as in Photoshop with multiplicative blending. You need alpha blending if colors are involved.

OK. So would a I see a big performance boost if I converted all my transparent background pngs and gave them white background and used a multiplicative shader?

@Jessy

I don’t know what you mean by “scaled down”. The right approach is probably to use a lower quality setting on non-retina, if only for using “Half Res” textures.

Scaled down probably isn’t the right term. I’m using an orthographic camera with its “size” set to perfectly frame a plane with an x scale of 102.4 and z scale of 76.8 (ie 10% of a standard ipad screen res in pixels). As 2048 x 1536 px texture is then applied to the plane. So it my mind that is scaled down 50%. Target res in the player settings is set “default device res” so hopefully this approach is OK.

@Jessy

That doesn’t make any sense; AIFF is an uncompressed format.

True, but once in Unity you can select Native or compressed for the AIFF file. My understanding is that native will make the binary bigger but compressed will use a little more memory to play back.

@Jessy

There should never be a reason for you to be using jpegs unless that’s the best source format you had.

That interesting. I just chose jog because the nature of the image meant it was a much smaller file size than the png version. Would you suggest making it a png anyway?

@Jessy

I don’t understand what you’re doing, with words alone.

It’s like the camera is pointing straight at a wall, and moving left and right and up and down to reveal different sections of the wall.

Sorry if anything isn’t clear - 330 am here.

Thanks again

[Jessy]

@BillyMFT

That’s true. Do you still think it requires a multiplicative shader?

Multiplicative blending is equivalent to alpha blending when your sprite is black:

Alpha blending, given two colors X and Y, and alpha channel A:
Result = AX + (1-A)Y

Given X = 0 (black), Result = (1-A)Y

“1-A” is the sprite on a white background, when the sprite is black. As such, no need for separate RGB and A. Whether this saves memory is dependent on what texture format you’re using. I don’t know how to best store the data via PVRTC for best visual results; it seems to change with revisions of the compressor.

@BillyMFT

OK. So would a I see a big performance boost if I converted all my transparent background pngs and gave them white background and used a multiplicative shader?

Depends on the format. It you were already using PVRTC, probably not.

@BillyMFT

Scaled down probably isn’t the right term. I’m using an orthographic camera with its “size” set to perfectly frame a plane with an x scale of 102.4 and z scale of 76.8 (ie 10% of a standard ipad screen res in pixels). As 2048 x 1536 px texture is then applied to the plane. So it my mind that is scaled down 50%.

To me, it sounds like no scaling is occurring. It’ll just look better on a retina display.

@BillyMFT

True, but once in Unity you can select Native or compressed for the AIFF file. My understanding is that native will make the binary bigger but compressed will use a little more memory to play back.

“Compressed” turns an AIFF into an MP3. What memory is used for an MP3 is dependent on “Load type”.

@BillyMFT

That interesting. I just chose jog because the nature of the image meant it was a much smaller file size than the png version. Would you suggest making it a png anyway?

Of course JPEG is smaller; it’s lossy. If you need to resort to it because you’re out of hard drive space, then you just need a bigger hard drive.

@BillyMFT

It’s like the camera is pointing straight at a wall, and moving left and right and up and down to reveal different sections of the wall.

I got that. It’s important to know what’s on the wall. Without seeing it, I don’t think you can do much better than you’re doing, in terms of memory, although there might be an easier way to model the behaviour.

[BillyMFT]

Thanks very much Jessy - your pointers have helped a lot.

[hippocoder]

You should profile it. Even if it means downloading unity trial to try out the profiler with deep profile. Compressed audio is a massive cpu hog, use decompress on load for short clips. Also this depends on how many are being played back at the same time. Jessy’s tips on shaders will serve you well although the fixed function ones ought be as fast. There’s some super fast 2D ones in my signature which are done in cg.

I do recommend unity pro for the profiler though, it’s an absolute god send.

[BillyMFT]

Thanks hippo. I do have Unity Pro but not a lot of experience with the profiler - certainly something I need to spend more time on.

Yeah I have changed all sounds to decompress on load now, although I am still suspicious of the audio causing some lag.

Jessy’s mention of PVRTC helped me optimise the texture settings. I had them on Alpha 8 but have now changed them to RGBA Compressed PVRTC 2 bits. Still using the Sprite shader in my original post for now.

Thanks again

[BillyMFT]

Hi Hippo,

Would you mind having a look at some of the profiler data? I’ve uploaded some screenshots here: http://www.lcdemakes.com/profiler-data.png

Is there anything that stands out as being a major issue? I have narrowed down the lag to fairly specific point but there is a lot happening at the point so I’m not exactly sure what is causing the lag. For example, when the player attacks an enemy

the attacks hands of the player turn on for a second (active on and off)
the enemy flashes and then hides (active on and off)
a short sound plays (decompress on load)
a small amount of data is saved to an sqlite db
a textmesh object is updated with the new score

Does the profiler data point to any of those in particular?

Thanks in advance
Bill

[hippocoder]

Can’t see it - sorry I’m late. Feel free to PM if I need a poke.

[BillyMFT]

no worries mate. Got it all running well with a few texture format and audio tweaks as well as some code optimisation - javascript arrays were killing it.

Thanks for your help