I have a external C# client and I am trying to send a WriteableBitmap over a socket and catch the image with my server which is in unity but I don’t know how to rebuild a WriteableBitmap in the server. Can anyone point me to the right direction?
Thanks.
Server: ```csharp
**try {
listener = new TcpListener(IPAddress.Parse(conHost), conPort);
listener.Start();
print("Server started!");
print("Waiting for a client...");
while (keepListening) {
Thread.Sleep(20);
TcpClient client = listener.AcceptTcpClient();
NetworkStream netStream = null;
try {
byte[] buffer;
// Obteniendo una referencia al Stream de datos de la conexión.
netStream = client.GetStream();
// Definiendo el buffer de entrada y salida.
buffer = new byte[client.ReceiveBufferSize];
// Guardamos la cantidad de bytes leidos.
int bytesRead = netStream.Read(buffer, 0, Convert.ToInt32(client.ReceiveBufferSize));
// CONVERT THE BUFFER TO A WRITEABLEBITMAP...
client.Close();
} catch (Exception ex) {
print(ex.ToString());
} finally {
if (netStream != null) {
netStream.Close();
}
}
}
} catch (Exception e) {
print("Error: " + e.ToString());
}**
** **Client:** **csharp
** byte data = new byte[8294400];
myWriteableBitmap.CopyPixels(data, myWriteableBitmap.BackBufferStride, 0);
ClientSocket.sendMessage(data);
///
/// Sends a message
///
///
public static void sendMessage(object msg)
{
byte message = (byte) msg;
TcpClient tcpClient;
byte buffer = new byte[256];
try
{
tcpClient = new TcpClient();
tcpClient.Connect(new IPEndPoint(IPAddress.Parse(conHost), conPort));
try
{
Console.WriteLine("Sending message...");
Send(tcpClient.Client, message, 0, message.Length, 5000);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
catch (SocketException e)
{
if (e.SocketErrorCode == SocketError.ConnectionRefused) {
Console.WriteLine("Conexión a servidor rehusada. Puede que el servidor no esté disponible.");
}
else
{
Console.WriteLine("Socket error:" + e);
}
}
}
private static void Send(Socket socket, byte[] buffer, int offset, int size, int timeout)
{
int startTickCount = Environment.TickCount;
int sent = 0; // how many bytes is already sent
do
{
if (Environment.TickCount > startTickCount + timeout)
throw new Exception("Timeout.");
try
{
sent += socket.Send(buffer, offset + sent, size - sent, SocketFlags.None);
}
catch (SocketException ex)
{
if (ex.SocketErrorCode == SocketError.WouldBlock ||
ex.SocketErrorCode == SocketError.IOPending ||
ex.SocketErrorCode == SocketError.NoBufferSpaceAvailable)
{
// socket buffer is probably full, wait and try again
Thread.Sleep(30);
}
else
throw ex; // any serious error occurr
}
} while (sent < size);
}**
``` I can use another image type/object/format to send it over sockets as long as I can convert my WriteableBitmap. I need to use it as a Texture2D in unity by the way…
You should be able to use the Texture2D.SetPixels function. Sending images through networking does seem like a bad idea, most would agree with me. Why not pass the image to a php page that uploads the image, then the server uses the WWW class to grab it. But… SetPixels should work for you if you are going to do it that way. You could also convert it to a byte[ ] array and send that to the server.
Thanks but the method “setPixels” of Texture2D has a Color array as argument so… Should I convert my byte array buffer to a Color array? In that case, how can I convert it?
Thanks.
EDIT:
I did convert my WriteableBitmap to a System.Drawing.Bitmap and tried to convert it to a byte array as you point me in your links and I’m getting the next error: A generic error occurred in GDI+
The error points me to the red line:
public static byte[] ImageToByte(Image img)
{
byte[] byteArray = new byte[0];
using (MemoryStream stream = new MemoryStream())
{
[COLOR=#ff0000]img.Save(stream, ImageFormat.Png);[/COLOR]
stream.Close();
byteArray = stream.ToArray();
}
return byteArray;
}
You may need System.Drawing.Imaging.ImageFormat as well as System.Drawing, or even System. (when i say namespace, i meant the imports at the start of the file, the using System; stuff)
I have all the right imports though. I will copy my entire Client code for you.
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Drawing.Imaging;
using System.Windows.Interop;
using System.Windows.Media.Imaging;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Web.Script.Serialization;
namespace SiberosNestle.Connectivity
{
class ClientSocket
{
private static string conHost = "127.0.0.1";
private static int conPort = 8888;
/// <summary>
/// Sends a message
/// </summary>
/// <param name="message"></param>
public static void sendMessage(object msg)
{
byte[] message = (byte[]) msg;
TcpClient tcpClient;
byte[] buffer = new byte[256];
try
{
tcpClient = new TcpClient();
tcpClient.Connect(new IPEndPoint(IPAddress.Parse(conHost), conPort));
try
{
Console.WriteLine("Sending message...");
Send(tcpClient.Client, message, 0, message.Length, 5000);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
catch (SocketException e)
{
if (e.SocketErrorCode == SocketError.ConnectionRefused) {
Console.WriteLine("Conexión a servidor rehusada. Puede que el servidor no esté disponible.");
}
else
{
Console.WriteLine("Socket error:" + e);
}
}
}
private static void Send(Socket socket, byte[] buffer, int offset, int size, int timeout)
{
int startTickCount = Environment.TickCount;
int sent = 0; // how many bytes is already sent
do
{
if (Environment.TickCount > startTickCount + timeout)
throw new Exception("Timeout.");
try
{
sent += socket.Send(buffer, offset + sent, size - sent, SocketFlags.None);
}
catch (SocketException ex)
{
if (ex.SocketErrorCode == SocketError.WouldBlock ||
ex.SocketErrorCode == SocketError.IOPending ||
ex.SocketErrorCode == SocketError.NoBufferSpaceAvailable)
{
// socket buffer is probably full, wait and try again
Thread.Sleep(30);
}
else
throw ex; // any serious error occurr
}
} while (sent < size);
}
}
public class ClientHelper
{
private static MainWindow kWnd = null;
private static bool doSend = false;
public static void initClient(Object kinectWnd)
{
// Si el objeto es una instancia de tipo MainWindow...
if ( kinectWnd is MainWindow ) {
kWnd = (MainWindow) kinectWnd;
while (kWnd.keepConnecting())
{
if (doSend) {
byte[] data = null;
Bitmap bm = null;
kWnd.Dispatcher.Invoke(new System.Action(() =>
{
bm = BitmapFromWriteableBitmap(kWnd.colorBitmap);
if (bm != null)
{
data = ImageToByte(bm);
ClientSocket.sendMessage(data);
}
else
{
Console.WriteLine("Null bitmap.");
}
}), null);
doSend = false;
}
}
}
return;
}
public static void sendData()
{
ClientHelper.doSend = true;
}
public static Bitmap BitmapFromWriteableBitmap(WriteableBitmap writeBmp)
{
Bitmap bmp;
using (MemoryStream outStream = new MemoryStream())
{
BitmapEncoder enc = new BmpBitmapEncoder();
enc.Frames.Add(BitmapFrame.Create((BitmapSource)writeBmp));
enc.Save(outStream);
bmp = new Bitmap(outStream);
}
return bmp;
}
public static byte[] ImageToByte(Image img)
{
byte[] byteArray = null;
using (MemoryStream stream = new MemoryStream())
{
img.Save(stream, ImageFormat.Png);
byteArray = stream.ToArray();
}
return byteArray;
}
public static byte[] ImageToByte2(Image img)
{
ImageConverter converter = new ImageConverter();
return (byte[])converter.ConvertTo(img, typeof(byte[]));
}
}
}
EDIT: Ok. Now it’s fixed the part of sending the bitmap. But know I don’t know how to convert it in the server side which is in unity and seems like I can’t use System.Drawing assembly…
Any ideas? I will update the code that solves my problems in the client side here:
Bitmap bm = null;
using (MemoryStream outStream = new MemoryStream())
{
BitmapEncoder enc = new BmpBitmapEncoder();
enc.Frames.Add( BitmapFrame.Create( (BitmapSource) kWnd.colorBitmap ) );
enc.Save(outStream);
bm = new Bitmap(outStream);
using (MemoryStream stream = new MemoryStream())
{
bm.Save(stream, ImageFormat.Png);
data = stream.ToArray();
ClientSocket.sendMessage(data);
}
}
The major problem here is that I can’t use a Bitmap (from System.Drawing namesapce/assembly) in the unity code.The client sends the bitmap perfectly, but in unity if I can’t create a Bitmap object… what can I do?
Just note that my client is outside unity, and my server is in unity.It is a little uncommon but it is like that.
Now. I see your code but I don’t know how can I load the bitmap into the Texture2D. I don’t see any reference to the variable “bmp”.
Is it maybe a way to use a buffer of pixels, load them into the Texture2D, and then load my bitmap’s pixels info into that?
Edit: Thank you for all your help. I appreciate it. Edit 2: I realize that I was using another thread to not block the entire application while listening for a TCP Client but Unity is not threadsafe and I can’t call unity core methods from another thread… I saw this old post: .net tcp networking help - Unity Engine - Unity Discussions And I tried to make my call to AcceptTcpClient() inside a coroutine but it is still blocking… Any ideas? ```csharp
**using UnityEngine;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Drawing;
public class Server : MonoBehaviour {
public GameObject testPlane;
private string conHost = "127.0.0.1";
private int conPort = 8888;
private TcpListener listener = null;
private volatile bool keepListening = true;
void Start() {
createListener ();
}
/// <summary>
/// Creates the server.
/// </summary>
public void createListener() {
try {
listener = new TcpListener(IPAddress.Parse(conHost), conPort);
listener.Start();
print("Server started!");
print("Waiting for a connection...");
// Starting a listening coroutine.
StartCoroutine("listen");
} catch (Exception e) {
print("Error: " + e.ToString());
}
}
IEnumerator listen() {
while(keepListening) {
TcpClient client = listener.AcceptTcpClient();
NetworkStream netStream = null;
try {
byte[] buffer;
Bitmap bmp;
using (var ms = new MemoryStream(buffer))
{
bmp = new Bitmap(ms);
}
int w = 50;
int h = 50;
Texture2D texture = new Texture2D(w, h, TextureFormat.RGB24, false);
byte[] data = new byte[w * h * 3];
int pixel = 0;
for (int y = 0; y < h; y++) {
for (int x = 0; x < w; x++) {
if (y % 2 == 0) {
data[pixel] = 255;
data[pixel+1] = 0;
data[pixel+2] = 0;
} else {
data[pixel] = 0;
data[pixel+1] = 0;
data[pixel+2] = 255;
}
pixel += 3;
}
}
texture.LoadRawTextureData(data);
texture.Apply();
testPlane.renderer.material.mainTexture = texture;
print("Bitmap arrived.");
client.Close();
} catch (Exception ex) {
print(ex.ToString());
} finally {
if (netStream != null) {
netStream.Close();
}
}
}
yield return new WaitForSeconds(.1f);
}
void OnApplicationQuit() {
keepListening = false;
if (listener != null) {
// This would stop the AcceptTcpClient method by trhowing an exception. This makes it finish faster.
listener.Stop ();
}
}
Then just pass method as a variable and execute it from the main thread using method(), or whatever the variable is with (). This allows you to define functions etc within the other thread, but actually execute it from whatever thread you want. Perhaps make a static class, where you can add the action to a generic list, then when the main thread is ready to execute the function, iterate through the list and execute it.
I really don’t care if I use unity’s networking or plain sockets, but I think if I use unity’s networking then I am forced to have the client and the server inside unity. I can’t do that.
Edit: I’m going to use my server as is. I think I understood the way to load the pixel data from my bitmap to the Texture2D but I can’t show the entire image. I just see a short line in the bottom of the plane. Another thing is… how can I do this in a Canvas panel instead of a plane. I am trying many things and no one is working. My new bitmap to Texture2D copy code: ```csharp
**int w = 512;
int h = 512;
Texture2D texture = new Texture2D(w, h, TextureFormat.RGB24, false);
// Lock the bitmap's bits.
Rectangle rect = new Rectangle(0, 0, b.Width, b.Height);
System.Drawing.Imaging.BitmapData bData = b.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite, b.PixelFormat);
// Get the address of the first line.
IntPtr ptr = bData.Scan0;
// Declare an array to hold the bytes of the bitmap.
int bytes = Math.Abs(bData.Stride) * b.Height;
byte[] rgbValues = new byte[bytes];
// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);
texture.LoadRawTextureData(rgbValues);
texture.Apply();
colorWrapper.material.mainTexture = texture;**