I am transmitting real-time tracking information of a physical object from a server (desktop computer) that sends data at 40hz to hololens (the client) using UWP TCP sockets. The hololens uses primitive game objects to visualize the tracking data. I profiled my application in hololens and found the GPU and CPU to be within limits and the frame rate to be at a constant 60 FPS. The visualization is smooth but seems to slightly lag behind the actual movements of the physical object. I assumed that the problem might be that I am visualizing cached incoming data, i.e I am using data that is slightly older than the real-time data. Since I am not an expert in networking, I did not have the right keywords to google my situation. I would like to know if my assumption is right and if there are other things that I should consider to minimize this lag. And if my assumption is right, which UWP socket function should I modify to remove cache. Any help would be greatly appreciated. Thanks.
TL;DR: TCP is a poor choice for your application, and your code implementation of it could have inefficiencies.
There’s lots of variables. Sockets don’t implement caching on their own, but your implementation of them certainly could.
Also, TCP is not designed to be a fast real time protocol. By default TCP waits 200 ms before sending to wait for partial packets to fill up (so data is sent up to 12 frames late at 60 FPS). It is possible to disable that, but there are other features of TCP that aren’t great for real time, such as how it deals with out of order packets and dropped packets. If a packet is dropped you probably don’t care in your application, because the data is old as soon as it is lost, replaced by new data already being sent. But TCP blocks everything and waits for the sender to timeout, and then resend the old data before any new data can be processed.
The size of the data you are sending is also important. For something real time you should consider UDP which has none of that kind of higher level functionality built in.
The speed at which data is read from the NIC in your network receiver thread and then moved and processed on Unity’s main thread can also be a cause for delay. Your implementation may be taking too long somewhere in there. Hard to guess without code.
Hi ,
Did you find any way to resolve this ?
I have the exact same problem , however I built the system using udp socket to share the hand of user across network on an holo 2
To prove that my system is not at fault I tried a simple experiment ,
I have 1 server and 2 clients, 1 of the client is on Holo 2 and the other on windows standalone.
When both are connected they can see each other avatar as well as their own (The computer simulate hand with the MicrosoftRealityToolkit)
Inside the hololens when the user move his hand, we can see the Pc client receiving it instantly with a really small delay
however the Holo recv his own move way later (0.5s delay approximatively)
Same experiment with the pc moving his avatar, It will recv his own move instantly , however the holo will once again recv it with a delay.
It’s really strange because all the packets are recved, all the movements are well restored they just have a big delay on holo. I suppose that the networks card is at fault and buffer the received msgs, but Im not knowledgeable enough on network stuff to really understand what’s happening there .