I am using Unity as the back-end game engine that sends camera views to a C# application that simulates a control console. The C# console can send information back to change camera movements and switch active cameras. The console will have 8 320x240 and 2 640x480 views of the game running in Unity. I am looking for some information on a general approach I should take to achieve this. Here’s what I’ve tried so far along with the results:
Approach 1:
I have 10 cameras in Unity rendering to texture. The RenderTextures are sent via UPD to the C# app. Since the test was done with 128x128 RenderTextures everything works fine and the game runs at 20-30 fps. Since UPD as a packet limit I can’t really go much higher than 128x128. I could continue with this method by sending multiple images to represent one frame and then stitch them back together in the C# app. I think I will start hitting performance issue when I tray to get 8 320x240 and 2 640x480.
Approach 2:
I have the same Unity game sending out RenderImages via TCP. The game starts to run really slow when I send out one 512x512 image.
TCP Code to send out images:
using UnityEngine;
using System.Collections;
using System;
using System.Net;
using System.Net.Sockets;
using System.IO;
using System.Threading;
public class TCP_ImageSend : MonoBehaviour {
public RenderTexture sendRenderTextureMain;
private Texture2D sendImage2DMain;
byte[] data = new byte[0];
int sent;
IPEndPoint ipep = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9050);
Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
void Start ()
{
sendImage2DMain = new Texture2D(sendRenderTextureMain.width, sendRenderTextureMain.height, TextureFormat.RGB24, false);
try
{
server.Connect(ipep);
}
catch (SocketException e)
{
Debug.Log(e.ToString());
}
}
void Update () {
RenderTexture.active = sendRenderTextureMain;
sendImage2DMain.ReadPixels(new Rect(0, 0, 640, 480), 0, 0);
sendImage2DMain.Apply();
sent = SendVarData(server, sendImage2DMain.EncodeToPNG());
}
private static int SendVarData(Socket s, byte[] data)
{
int total = 0;
int size = data.Length;
int dataleft = size;
int sent;
byte[] datasize = new byte[0];
datasize = BitConverter.GetBytes(size);
sent = s.Send(datasize);
while (total < size)
{
sent = s.Send(data, total, dataleft, SocketFlags.None);
total += sent;
dataleft -= sent;
}
return total;
}
}
Approach 3:
Make the game networked so that each instance of the game can take the load of rendering to texture and sending out network streams. Networking the games like this I would assume would be and alternative to threading.
The main issues I’m running into are performance due to the time it takes to send the images stream through the network. If I understand it correctly, Unity can not put the RenderTexure and network streams onto a different thread.
Is there something I need to do with the code to optimize the image processing and network streaming?
I’m generally trying to see if I’m heading down the right path. How do I get 8 320x240 and 2 640x480 camera views out of Unity without the game running at 2fps? Perhaps there are better ways and approaches for what I’m trying to do.
Hello. i am trying to do the same, can you please share the code that you have developed. it would help me a lot.
– rajanerve