Grab the unity screens bitmap data

Unity Engine
,
1

Hi, i am trying to “capture” Unitys screen and use the bitmap data to create an UIImage in xCode.

While i had some success, it´s not good enough. Here is the one that works, when Anti Aliasing is NOT used. If that is used, you only get an error and a black image.

  1. in unity have some script like this:
public function xCodeCaptureScreenshot(filename : String) {
	Application.CaptureScreenshot(filename);
}

You can call that from within Unity, i am calling it from xCode like this:

- (IBAction) shareOneImage:(id)sender {
	
	NSString *screenShotFileName = @"unityScreenshot.png";
	UnitySendMessage("xcodeReciever", "xCodeCaptureScreenshot", [screenShotFileName UTF8String]);
	
	NSString *docDir = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject];
	NSString *uniquePath = [docDir stringByAppendingPathComponent: screenShotFileName];
	
	UIImage *theImage  = [UIImage imageWithContentsOfFile:uniquePath ];
}

Ok, this works, the png is written to the documents directory. The file is saved asynchronous, so actually you need some delay befor you can read the file. If you just use this script, you will only see a part of the image, or nothing, or get an error, depending on your devices speed i guess.

By the way, on iPhone4 you only get a 480x360 image, so thats not so nice. This and the fact, that this doesn´t work with the new MSAA feature - you only will see a OpenGLES error 0x0502 in PresentSurface:651 Error, makes it somewhat unusable for me.

So my next attempt was the normal objective C way:

- (UIImage*) makeUnityScreenShot {
	
	UIGraphicsBeginImageContextWithOptions(theUnityView.bounds.size, theUnityView.opaque, 0.0);
	[theUnityView.layer renderInContext:UIGraphicsGetCurrentContext()];
	
	UIImage * viewImage = UIGraphicsGetImageFromCurrentImageContext();
	
	UIGraphicsEndImageContext();
	return viewImage;
}

Well, from this i get a black image. Apparently UIGraphicsGetImageFromCurrentImageContext() doesnt work with EAGLview.
But maybe i just have a small issue in this and somebody could give me hint.

Last thing i can think of is to somehow read out the buffers. But i have no clue how to do this.

A normal device Screenshot is also not wanted, cause i have GUI elements on top of unity which i don´t want to have inside the resulting image.

Any help is welcome, i am running out of time with my schedule…:shock:

2

I love it when i find a solution right afterwards i porst a problem:

In case you want to grab unitys EAGLviews bitmapdata in XCode, you can do it like this (found this on stackoverflow):

(You need to have a reference to the eagleView)

- (UIImage*) makeUnityScreenShot {
	int s = 1;
    UIScreen* screen = [ UIScreen mainScreen ];
    if ( [ screen respondsToSelector:@selector(scale) ] )
        s = (int) [ screen scale ];
	
    const int w = theUnityView.frame.size.width;
    const int h = theUnityView.frame.size.height;
    const NSInteger myDataLength = w * h * 4 * s * s;
    // allocate array and read pixels into it.
    GLubyte *buffer = (GLubyte *) malloc(myDataLength);
    glReadPixels(0, 0, w*s, h*s, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
    // gl renders "upside down" so swap top to bottom into new array.
    // there's gotta be a better way, but this works.
    GLubyte *buffer2 = (GLubyte *) malloc(myDataLength);
    for(int y = 0; y < h*s; y++)
    {
        memcpy( buffer2 + (h*s - 1 - y) * w * 4 * s, buffer + (y * 4 * w * s), w * 4 * s );
    }
    free(buffer); // work with the flipped buffer, so get rid of the original one.
	
    // make data provider with data.
    CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, buffer2, myDataLength, NULL);
    // prep the ingredients
    int bitsPerComponent = 8;
    int bitsPerPixel = 32;
    int bytesPerRow = 4 * w * s;
    CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
    CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
    CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;
    // make the cgimage
    CGImageRef imageRef = CGImageCreate(w*s, h*s, bitsPerComponent, bitsPerPixel, bytesPerRow, colorSpaceRef, bitmapInfo, provider, NULL, NO, renderingIntent);
    // then make the uiimage from that
    UIImage *myImage = [ UIImage imageWithCGImage:imageRef scale:s orientation:UIImageOrientationUp ];
   
    CGImageRelease( imageRef );
	return myImage;
    // buffer2 will be destroyed once myImage is autoreleased.
}

So far i didn´t test this with Anti-Aliasing turned on. Will be next step.

3

Arrrgggghhhh, doesn´t work with Antialiasing. Black Image again!

4

Anti aliasing is a tricky beast. You actually have to resolve into a non-anti aliased buffer and read from that. glReadPixels won’t return anything when anti aliasing is enabled.

5

thanks for your quick response. But my Brain is dead nor (Did you ever see the movie Braindead? You should).
And unless somebody posts working code i am lost.

I just wrote an Email to another developer i think has more knowledge than i do about that (cause i have no knowledge about buffers and c++ at all). In case he thinks he can handle that i am going to hire him.

But in case not, i am still happy about a working solution. I have to be done with this project in 10 days. Big Customer, little money.
Saying, in case somebode has a solution i can pay by hour, but the costs must not be exagerated. Otherwise i need to drop anti aliasing.

Some hint i read was to change the quality settings by script (in unity) to a level not showing MSAA. I still need to check that out. But all this checking and trial and error costs to much time, which i am running out of.
Anyways, i will retire now… BRAINDEAD

6

One of my plugins (Etcetera Plugin) handles screenshots with or without MSAA. It’s pretty cheap so it may be an option for you if you dont want to dig any further.

7

Thank you, i just bought your package. Since i already have a custom setup, i had to go through the resulting files and cut and paste the necessary stuff into my project. I hope you don´t mind.
I guess i made my setup overly complicated at the desparate attempt to get the EAGLview into a viecontroller view, showing landscape only with rotation between the two landscape modes.
This was in conflict with your rotation methode so i had to hack that as well.

I found one strange error in the resulting image - some parts of the model using a shader with a sphere reflection map on it appear completly white. Which is even more strange, because i have such a reflection on other parts to, and those are not white.

Anyways, i am trying to drop you a private message, maybe you have seen such a behaviour before.

8

A behaviour I had with your Eceteraplugin Screenshot function, it’s not taking the actual picture with MSAA on, its taking 1 thats some milliseconds old.
I found out, because, i made a hideGUI: boolean followed by a yield WaitForEndOfFrame(); before MSAA that helped to kick my GUI out for the screenshot, when activating MSAA i need to wait for some more frames for the same effect.
Maybe you could explain why or what happens, then i could work around more precisely :wink:

9

@marjan, I responded to your DM.

@gamma.psh, with MSAA on you will end up with a bit of lag on the screenshot and it won’t be with MSAA. There is a lot going on when capturing the screenshot:

  • the entire screen must be resolved into a non-anti aliased buffer
  • the pixels need to be read and stored in a buffer
  • an image is created from the pixels and rotated appropriately based on the orientation of the device
  • the image is written to disk
  • the entire screen is resolved back into an anti aliased buffer