My question is if anyone has been able to get a working Microsoft azure face API working within Unity? Not sure how to include all of the correct plugins for it.
This is for a project what is working with a rehabilitation robot for stroke victims and children with cerebral palsy.
I have been designing a log on system with MySQL integrated into Unity’s C# scripts to currently manually log the users in, however this limits the users to have to have another person(doctor or nurse) to help them log on.
So as of this I hope to redesign my login system with facially recognition, this will automatically sign in the users. For this I am planning on using Microsoft Azure’s face API.
It should be pretty straightforward once you have the SDK installed. Your Unity project will need network access as the actual face recognition happen in Azure.
You’ll get a confidence interval (0 to 1) floating value of how likely they two images are to be the same person. Your application will need to determine an acceptable threshold value given your security requirements.
I have been able to save faces to a face group. I can check a face to that group to see if the user is in the face group.
I have also been able to store data to the users face such as there ID and name for when they register.
I now need to try redo the scripts but within Unity. I will need to install the SDK, and try get the project to have network access. I might already have this as I have it using MySQL with a locally hosted database for users.
Good luck! Sounds like a really cool project. I’d love to learn more about it when you have something up and running. I think Microsoft’s PR department might be interested too.
I’ve been doing a lot of Azure programming lately. You shouldn’t need any Azure SDKs to do this, you can issue all queries as REST API calls. That’s good news because the Azure SDKs tend to need relatively current .NET Framework versions which would be a big problem with Unity.
Furthermore, you should be able to use Unity’s own Networking namespace to issue your requests, no need to reference the .NET HTTP classes. Here’s a REST client I wrote awhile back using Unity’s namespace and Json.Net.
You’ll need to make some changes, I owned the server and client so I cheated on the REST rules and ran everything as an HTTP PUT, but that should be relatively easy. The only gotcha I’ve found with Unity is that it’ll URL-encode POST message bodies. If I remember right, the trick was to set up the call as a PUT, then change the method to POST just before sending. (Search the forums, I posted something about it awhile back.)
The earlier comment about Microsoft likely being interested in your project is probably dead-on. The New Friendly Microsoft is highly responsive on github if you can find the right group. It has been pretty interesting having the option of going straight to the real .NET devs with questions and issues. I’d give that a shot.
using UnityEngine;
using UnityEngine.Networking;
using System.Collections;
using System.Collections.Generic;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
public class REST
{
// TODO - use defines or config to select URI
private const string URI = "http://localhost:50000/BlackHandService";
public UnityWebRequest request = null;
/// <summary>
/// Builds a UnityWebRequest object. Caller should then "yield return call.request.Send()"
/// to send the request to the server. All operations must use HTTP PUT because UnityWebRequest
/// will URL-encode POST message bodies and message bodies on HTTP GET are a gray area.
/// </summary>
/// <param name="uriParameters">service, action, and additional parameters to list in the URI</param>
/// <param name="payload">null or object(s) to serialize in the request</param>
/// <param name="addToken">if true, GameData.token will be added to requestBody</param>
public REST(object[] uriParameters, JObject payload = null, bool addToken = true, string contentType = "application/json; charset=utf8")
{
if (payload == null) payload = new JObject();
if (addToken) REST.PayloadAddObject(payload, "LoginToken", GameData.token);
string body = payload.ToString();
// shouldn't ever be zero-length but the sorely-broken UnityWebRequest won't permit an empty PUT/POST body (WTF?)
if (body.Length == 0) body = " ";
string uri = URI;
foreach(object p in uriParameters)
{
uri = uri + "/" + p.ToString();
}
request = UnityWebRequest.Put(uri, body);
if(contentType.Length > 0) request.SetRequestHeader("Content-Type", contentType);
}
/// <summary>
/// This creates and adds objects to a JObject for REST request payloads.
/// This way the caller doesn't need to manage JObject syntax.
/// </summary>
/// <param name="payload">reference to a JObject to update</param>
/// <param name="key">name of the new data to store</param>
/// <param name="value">the new data to store</param>
public static void PayloadAddObject(JObject payload, string key, object value)
{
// Without this JObject.FromObject throws an error.
if (value.GetType().Equals(typeof(string)))
{
payload.Add(new JProperty(key, (string)value));
}
else
{
payload.Add(new JProperty(key, JObject.FromObject(value)));
}
}
/// <summary>
/// Read the response body as a raw string.
/// </summary>
public string GetResponseBody()
{
return System.Text.Encoding.UTF8.GetString(request.downloadHandler.data);
}
/// <summary>
/// Read the response body as a JSON JObject.
/// </summary>
public JObject GetResponseJObject()
{
return JsonUtility.FromJson<JObject>(GetResponseBody());
}
/// <summary>
/// Test this after rest.request.Send() returns.
/// </summary>
public bool isError()
{
return (request.isError || request.responseCode != 200L);
}
/// <summary>
/// When isError() is true, use this to produce a (somewhat) friendly error message.
/// </summary>
public string error()
{
string err = request.error;
if (err == null)
{
err = GetResponseBody();
if (err == null || err.Length == 0) err = "Server returned " + decodeResponseCode();
}
return err;
}
/// <summary>
/// UnityWebRequest rather stupidly eats the HTTP status description so
/// this helper function turns the numeric error into a slightly more
/// useful message.
/// </summary>
private string decodeResponseCode()
{
switch (request.responseCode)
{
case 400L:
return "400: Invalid Request";
case 401L:
return "401: Unauthorized; Login Required";
case 500L:
return "500: Internal Error";
default:
return "HTTP response code " + request.responseCode.ToString();
}
}
}
I have ended up complying the application for the facial recognition into a separate executable files so that the menu system can find them within a local folder to use them when needed. The information is sent through temp .txt files.
I haven’t been able to get Microsoft to agree to any kind of deal. I may just have to pay up.
We are planning on redesigning the Robot system for the end of Easter.
I’m working on face matching(Recognising) by using Microsoft Azur Api. I succeed to show face matching with Microsoft Azur Api with image grab by url, in nodejs (javascript).
But i’m facing the problem how to take image from my local machine and sent that image to Microsoft Api in nodejs (javascript).
@Archsan can you please expain how you did ’ face matching(Recognising) by using Microsoft Azur Api. I succeed to show face matching with Microsoft Azur Api with image grab by url, in nodejs (javascript). ’