Our game runs fine on almost all the devices we’ve tested it on, except for ones with a PowerVR SGX gpu. The specific device we’re worried about is the Kindle Fire HD 7" (3rd Gen) (KFSOWI) (SGX 544)
We’ve also had issues on the Nexus S and Galaxy Nexus (SGX 540)
Symptoms:
Game runs at 60fps, for about 30 seconds, then the framerate slowly decreases to ~1 fps, at which point, a single black frame is drawn, the framerate goes back to 60fps, and the same pattern repeats. (About every 40 seconds)
Debugging:
I’ve tried to do some debugging of the issue - using PVRTune, when the framerate drops, the SPM counter increases to 100% until the black frame happens, at which point it drops back to 0 and the fps goes back to 60.
Theory:
What I think is happening is the amount in the parameter buffer keeps increasing, until the gpu crashes, SGX hardware recovery kicks in, and the gpu restarts, emptying the buffer again?
That assumption might be wrong, but anyway, any ideas as to how I’d fix this, or what might cause these symptoms?
I’m not sure how / what would cause the parameter buffer to slowly increase over frames and overflow, tri counts etc aren’t increasing, and this is only an issue on the SGX gpus.
About the only reason why the paramter buffer could keep on growing “forver” is if you don’t ever clear a frame. Or somehow else manage to confuse the driver in that you have an “never ending” amount of geometry.
Do you clear/discard all your render textures each frame? That would probably be the first thing I’d look into.
>the SPM counter increases to 100% until the black frame happens, at which point it drops back to 0
The SPM counter in PVRTune is a binary state, i.e. a value of 0% means SPM hasn’t happened and a value of 100% means it has. Any values in between this when the counter is graphed are interpolated from the true/false 0%/100% values. If you are seeing 100% SPM over a number of frames, this means that SPM is occurring in each of those frames.
>What I think is happening is the amount in the parameter buffer keeps increasing, until the gpu crashes, SGX hardware recovery kicks in, and the gpu restarts, emptying the buffer again?
When SPM occurs, the SGX hardware should handle the event cleanly with no crashes or hardware resets. If you’re seeing hardware reset events, then it could be caused by an SPM driver bug or by an unrelated issue (e.g. referencing outside of the bounds of vertex buffer data).
>Game runs at 60fps, for about 30 seconds, then the framerate slowly decreases to ~1 fps, at which point, a single black frame is drawn, the framerate goes back to 60fps, and the same pattern repeats. (About every 40 seconds)
Possibly unrelated, but a saw a similar problem on SGX a few months ago. There is an optimization in the SGX driver where FBO renders are only kicked when one of the FBO’s attachments are referenced by another render. This enables the driver to batch work efficiently. In the game I was analysing they were submitting draws to an FBO, but never referencing any of the FBO’s attachments in main frame buffer render. Following this investigation, we found a bug in the SGX driver where unused FBO data will continue to consume Parameter Buffer space until the Parameter Buffer space runs out. We’ve since fixed this by forcing unreferenced FBOs to render as excessive PB space is used. Applications can avoid the issue entirely by removing redundant FBO renders from their rendering pipeline.
Thanks, Aras and Joe! Was looking into the RenderTargets last night, and your experiences sound very much like what I found, by forcing all the rendertargets to rendered onto some quads where they can’t be seen, this issue was resolved!
Pretty strange fix, but seems to be working for now, thanks for the help!