Hands Tracking (Oculus Quest) directly in Unity Editor

I’ve been playing around with Oculus Quest hand tracking which is truly mind-boggling!

If you’re finding lack of hand-related data and visuals in Scene View at runtime annoying have a look at a tool I’ve put together.

https://github.com/handzlikchris/Unity.QuestRemoteHandTracking

Can’t make it work. Is it possible to upload an example project with everything set up?

When I run the scene on the Quest it freezes on startup. Tried to run adb logcat, but didn’t reveal much. I did make rules for the firewall to allow inbound connections on both TCP/UDP.

Hi,

Have you tried running the scene without HandTracking components? Generally on send error it’ll wait so it shouldn’t freeze your Quest. Unless there’s something not related that does that.

Yes, when I remove the HandTracking components it works. I will re-importing everything and test :slight_smile:

Any news on this? my application freezes after the loading screen aswell.

Let me check guys, I can only do that tomorrow though

1 Like

Hi,

I tried to recreate that with no luck, even when I dropped off WiFi with Quest or specified invalid IP address.

The only thing I’ve noticed is it gets bit slow with invalid network setup. Probably due to exceptions being thrown when it tries to send messages.

  1. Bit daft question but when you say that scene runs it means that you can actually use your hands there in train scene (without remote-tracking component turned on)?
    Just want to double check that your quest is on beta channel (with hand tracking) as without it it’d error when using new APIs.

  2. Unlikely but maybe quest version causes issues. What version are you on? Is it same as the one I’m testing?
    Version: 12.0.0.226.469.188362039
    Runtime Version: 12.0.0.226.469.188362025

  3. Other than than is there anything in logcat?

Generally if it can’t connect you’ll see some errors like
01-15 09:57:33.507: I/Unity(5016): On client connect exception System.Net.Sockets.SocketException (0x80004005): No route to host

01-15 09:57:27.508: E/Unity(5016): Keep alive error, System.InvalidOperationException: The operation is not allowed on non-connected sockets.

If the setup is fine you’ll see some data being sent
01-15 09:59:38.406: I/Unity(5191): UDP Connected hands data endpoint
01-15 09:59:44.151: I/Unity(5191): Message sent (size: 2948)

Hello Chris,

I have the same problem:
The headset freezes at load time.
But actually when I don’t use it for a little while and I take it again the scene works for few seconds (with hand tracking on the Quest side) and then freeze again.
On the computer (Editor) nothing happens, and its hard to debug anyways since the quest is freezing.

Here is the quest log :
cd C:\Program Files\Unity\Hub\Editor\2019.2.6f1\Editor\Data\PlaybackEngines\AndroidPlayer\SDK\platform-tools
adb logcat -v brief

Complete adb logcat (brief) in the text file
Here I paste a part, similar to yours :

I/VrApi  ( 5468): FPS=72,Prd=45ms,Tear=0,Early=72,Stale=0,VSnc=1,Lat=1,Fov=0,CPU4/GPU=2/2,1651/414MHz,OC=FF,TA=0/0/0,SP=N/N/N,Mem=1804MHz,Free=1678MB,PSM=0,PLS=0,Temp=27.0C/0.0C,TW=1.61ms,App=0.42ms,GD=0.00ms,CPU&GPU=1.70ms,LCnt=1,GPU%=0.15,CPU%=0.02(W0.05),DSF=1.00
E/Unity   ( 5468): SocketException: The requested address is not valid in this context
E/Unity   ( 5468):   at System.Net.Sockets.Socket.Bind (System.Net.EndPoint localEP) [0x00043] in <0e1afc3ce03d4e2ab478056348d6a9e9>:0
E/Unity   ( 5468):   at Assets.RemoteHandsTracking.HandsDataReceiver.Start () [0x00032] in <d2c4b3c0a45248e7a331cb3544dfe565>:0
E/Unity   ( 5468):
E/Unity   ( 5468): (Filename: <0e1afc3ce03d4e2ab478056348d6a9e9> Line: 0)
E/Unity   ( 5468):

What I don’t get, is why is freezing, at least I could understand that the data could not be send through the Network so nothing happens in the Editor.
But that the Oculus Quest freezes, its like overreacting ^^

Hope this helps,
Don’t hesitate to ask more info if you need.

Cheers :slight_smile:

5428731–552612–Oculus Quest adb logcat brief.txt (672 KB)

Hi Satolas,

Thanks, that helps.

I’ve recreated that and it’ll be down to wrong IP in HandsDataSender. Or if you’re certain that’s correct IP then Quest can not reach, eg. due to FW / network setup.

I’ll push a fix later this afternoon so it doesn’t freeze when that happens - but that freeze is not cause of your issue.