Problem:
I have to send one CameraView (from Camera it self, Screen or something else) over Network or over an videocard output (HDMI/USB3.0/…) so fast as it can, maybe with a static framerate. I have to simulate a camera and have to send it to another PC with a framerate equal or greater then 25 fps.
I’m a student from germany an i write my diploma in Unity3d right now an it is the first time with Unity, so i had read some examples in the internet, but i have a problem with my code. So i have 3 general questions to you and at first here are my ideas how it can be possible:
1. have to take pixel from the Camera or Screen. This following Script is attached to a Camera:
best resolution is 640x480 i have to do.
void OnPostRender()
{
Texture2D tex = new Texture2D(...);
tex.ReadPixel(...);
screenShot.Apply();
byte[] bytes = screenShot.EncodeToJPG();
}
2. Encode the Picture (Server uses OpenCV to read this bytes)
some idea:
-
Encoding to PNG/JPEG/… (easy but take a lot of time)
-
get only the Color32[ ] of the RenderTexture like:
-
encoding by my self in threads
3. sending Data over Network/VideoOut(Ethernet/HDMI)
idea:
- sending packages over Ethernet via UDP (fast, low package size)
- sending packages and use the Unity Transport Layer API (i don’t now so much about, i have to read more about) https://docs.unity3d.com/Manual/UNetUsingTransport.html
- read my input devices and take a “videoGrapperCard” to put the screen out of HDMI to the other Device which is read the video input.
- simulate my screen as a IPCamera over Network (ethernet)
Questions:
to 1.:
So the Problem is, when i start the Application, so my RAM Memory goes up very fast unti it is full an abort.
so what is the Problem of this function? I have read some users use RenderTexture and this Color, it is better and faster? is there a better way to get the picture without loose much of the perfomance?
what is better and faster:
- to read Pixel from the Screen or
- or render the camera by my self ?
to 2.:
there is a way to get the RAW information and encode it by my self. so we test the encoding functions
EncodingToJPG();
EncodingToPNG();
only JPG was possible with UDP to get a small size package.
Are the better variants? or realy do it by my self from code-examples from www!?
to 3.:
so we test it only over UDP with 640x480 Pixel(RGB24) and EncodingToJPG. is works, but only for view Secounds, because the RAM Memory goes up very fast with the funktion Texture2D.ReadPixel() and then the application abort cause of full RAM.
Is there a faster way to get the Information to the other Device?
What kind of tutorials i have read:
and some more …
So it was a lot of text and questions. I hope anybody can help me a at some points or have a package for all to solve my problem.
I thank you so much for reading about my problem and enjoy your day.
PS: I have to use all code in free-verison.