Image Sprite from Byte Array

Unity Engine
1

Hey,

I want to connect an android application and a unity game via socket. The android application should send a Bitmap as a Byte Array to the Server Socket in unity, and this Image should be displayed on an Image Game Object.
The sending part ist working fine, with the right amount of bytes arriving at the socket. However the displaying of said image is having some issues.

The Byte Array is generatet in Android Java like this:

ByteArrayOutputStream bos = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.PNG, 0, bos);
byte[] array = bos.toByteArray();

(unfortunately I couldn’t fint any information in the Android Documentation about the exact format of this compression, but PNG should be equivalent to RGBA32, right?)

The received byte array is the processed in Unity C#:

Texture2D bmp = new Texture2D(imageWidth, imageHeight, TextureFormat.RGBA32, false);
bmp.LoadRawTextureData(socketsServer.imageByteArray);  //this is line 23

Vector2 pivot = new Vector2(0.5f, 0.5f);
Rect tRect = new Rect(0, 0, imageWidth, imageHeight);
Sprite newSprite = Sprite.Create(bmp, tRect, pivot);
gameObject.GetComponent<Image>().overrideSprite = newSprite;

And this is a part of the Byte Array that is received:
Byte Array

When running the Script an error pops up:

What could be the issue? Usually the problem would be a missing TextureFormat / Mipmap boolean, which is not the case here.

Thank you very much!
Julian

2

It also doesn’t work by setting the TextureFormat to something else, like RGB24 for example?

3

Yes, I have tried a few different TextureFormats, like ARGB32, ARGB4444, RGBA444, RGB24, DXT5 and RGB565, however none of them have worked so far.

4

how many bytes you get from receive? (there’s also one trailing comma at the end…or that is just from print?)

5

The exact amout varies greatly, it’s usually anywhere between 10,000 and 250,000. But where would that trailing comma be? If you mean somewhere in the Byte Array it is probably from printing the array, which is too long for the unity console and gets truncated.

6

is it same amount of bytes that the texture needs?

7

Well, I’m not perfectily sure how exactly png compression works, the dimensions of the testure stay the same. However due to the nature of the image, which is drawn by touch on the android device the amount of transparent and opaque pixels varies, depending on how much of the canvas was drawn on.

8

Okay, I have deleted two of my answers, since the issue with the byte array not being complete is fixed, however the not enough data provided error still persisits even when the byte array is complete and valid.

9

actually that looks like png header data, you only want to send the actual image data without headers.

137,80,78,71 > ‰,P,N,G

10

Good idea, but unfortunately it’s still not working. I removed the first 8 bytes of the data, but all 12 formats I tested (DXT5, DXT1, RGBA32, RGBA444, ARGB32, ARGB4444, RGB24, RGB565, ATC_RGBA8, BGRA32, RGBAFloat, RGBAHalf) and no format specified at all, do not work and still produce the same error as before.

This would be the image data sent with the header (now in hex format): https://pastebin.com/y6MMgZRD
And this is a different image without the header: https://pastebin.com/Up1CntmJ

11

header has more data than 8 bytes, not sure how to get raw bitmap data in java though…
http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html

also try sending small manually created array first, to see if sending/receiving works.

1 Like
12

You have definitely pushed me in the right direction, I have now changed my Java Byte Array generation to

ByteBuffer byteBuffer = ByteBuffer.allocate(bmp.getByteCount());
bmp.copyPixelsToBuffer(byteBuffer);
byte[] array = byteBuffer.array();

Which eliminates the error I had all the time with the not enough data provided. However, The decoding still doesn’t seem to be working, as when I iterate through the roughly 60 Texture formats, all of them either generate a solid color, a repeating pattern or crash unity entirely.

Just as additional information: the Byte array is now about 3.5M bytes.

EDIT: Well, technically the question I started this thread on is answered, so I should probably start a new thread for this new issue. Thanks!