How to use Live Capture to build PC applications

(I am sorry for my poor English ability)

I loaded the Live Capture package with Unity 2021.2, and look forward to using Live Capture to build PC applications.
I hope to use the Face Capture App in IOS to control the PC applications built by Unity.

The CompanionAppServer class is private, I set it to public to create a server from a custom script. So I copied Live Capture as a custom package and changed the CompanionAppServer class.

The following is my test code, which is added to the empty GameObject in the scene.

using UnityEngine;
using Unity.LiveCapture.CompanionApp;
using Unity.LiveCapture;

public class TstServ : MonoBehaviour
{
    CompanionAppServer newSer;
    void Start()
    {

        var servMag = ServerManager.Instance;
        newSer = (CompanionAppServer)servMag.CreateServer(typeof(CompanionAppServer));

        newSer.AutoStartOnPlay = false;

        newSer.Port = 8090;

        newSer.StartServer();
    }
    private void OnDestroy()
    {
        if (newSer)
        {

            ServerManager.Instance.DestroyServer(newSer);
            Debug.Log("Clean Server");
        }
    }
    private void OnGUI()
    {
        if (newSer)
        {
            GUILayout.Label("PORT:" + newSer.Port.ToString());
            GUILayout.Label("Running:" + newSer.IsRunning.ToString());
        }
        else {
            GUI.color = Color.red;
            GUILayout.Label("Server Not Found!");
            GUILayout.Label("PORT:--");
            GUILayout.Label("Running:--");
        }
    }
}

It can run normally in the editor .
but in the built PC application, although Log shows that the server is already running, the server cannot be found in the Face Capture App.

What i want to ask is
1. Is the Face Capture App designed to only link with Unity Editor?
2. Is the Live Capture package allowed to be built as a standalone PC application?
3. I don't know much about network deployment, can anyone give me some reference to modify the Live Capture package so that it can be used on the built PC application?

Above is my question, thanks for the help.

1 Like

@Astion
I can give some pointers regarding this.

  1. Yes, the app is only designed to work with the editor. It is theoretically possibly to use for other cases, but this is not officially supported at this time.
  2. Technically you can include the Live Capture package in a standalone build, but it is not officially supported at this time as there is still much work left to properly support this.

That all said, if you really just want to communicate with app yourself, it is possible to use the CompanionAppServer and FaceClient classes directly by making them public.

In your code above, you'll need to call CompanionAppServer.OnUpdate() in Update(), which is probably why it is not working. Otherwise, it looks fine to me at a quick glance.

3 Likes

[quote=“ScottSewellUnity”, post:2, topic: 863323]
@Astion
I can give some pointers regarding this.

  1. Yes, the app is only designed to work with the editor. It is theoretically possibly to use for other cases, but this is not officially supported at this time.
  2. Technically you can include the Live Capture package in a standalone build, but it is not officially supported at this time as there is still much work left to properly support this.

That all said, if you really just want to communicate with app yourself, it is possible to use the CompanionAppServer and FaceClient classes directly by making them public.

In your code above, you’ll need to call CompanionAppServer.OnUpdate() in Update(), which is probably why it is not working. Otherwise, it looks fine to me at a quick glance.
[/quote]
It’s awesome
After testing it does work.
Calling CompanionAppServer.OnUpdate() in Update() allows my packaged application to connect to IOS FaceCapture

Thank you so much for helping me

How do you set the CompanionAppServer class to public?

[quote=“DanRP”, post:4, topic: 863323]
How do you set the CompanionAppServer class to public?
[/quote]
I moved the LiveCapture package to my local project, and then added the “public” keyword to the CompanionAppServer class

1 Like

This is exactly what I had to do. Wish i found this thread earlier. But Yes copying the live-capture package from the packagecache into my assets and then making some classes public let me run the server in a standalone windows build.

I'm trying to use this test code and have followed the steps of moving the LiveCapture package and adding public to the CompanionAppServer class but I am running into this error when I build and connect the Face Capture app. Does the FaceClient class need to be public as well? When I try to make it public it creates inconsistent accessibilities.

8707839--1176294--LiveCaptureError.txt (5.57 KB)

AMAZING! Thanks!. I leave the updated code:

using UnityEngine;
using Unity.LiveCapture.CompanionApp;
using Unity.LiveCapture;
public class TstServ : MonoBehaviour
{
public CompanionAppServer newSer;
void Start()
{
var servMag = ServerManager.Instance;
newSer = (CompanionAppServer)servMag.CreateServer(typeof(CompanionAppServer));
newSer.AutoStartOnPlay = false;
newSer.Port = 8090;
newSer.StartServer();
}
private void OnDestroy()
{
if (newSer)
{
ServerManager.Instance.DestroyServer(newSer);
Debug.Log("Clean Server");
}
}
private void OnGUI()
{
if (newSer)
{
GUILayout.Label("PORT:" + newSer.Port.ToString());
GUILayout.Label("Running:" + newSer.IsRunning.ToString());
}
else
{
GUI.color = Color.red;
GUILayout.Label("Server Not Found!");
GUILayout.Label("PORT:--");
GUILayout.Label("Running:--");
}
}
void Update()
{
newSer.OnUpdate();
}
}

hi, when i try to run my project using all the steps above, it seems to show errors, can someone help me?

Due to an update in the Live Capture API after version 3.0, the solution is to change the previous ServerManager to ConnectionManager.


Dont forget to move the live-capture folder from the library to the assets folder to u can modify the code and dont show errors.

Hi, when I copy the contents of that folder there are quite a few errors (eg.: Assets\Resources\Live Capture\Runtime\Mocap\MocapDevice.cs(405,33): error CS0115: 'MocapDevice.OnDestroy()': no suitable method found to override), any advice on how to resolve this?

EDIT #1: So I tried only copying the CompanionAppServer file, then assembly references are missing. I tried to add assembly references but as I add more, more and more errors pop up.

EDIT #2: After handling all assembly references, I got stuck with classes being inaccessible due to protection levels. So I tried importing the entire Live Capture folder, fixing assembly references and now even more classes are inaccessible.
How is it that inside the package folder they can access eachother just fine, as soon as they are copied its utter chaos?

EDIT #3: Utilizing

I have successfully custom imported the package. When doing this with already existing packages, make sure to:
- copy everything from the existing package into an external folder, rename package.json
- remove the original package
- use this video to re-import

Additionally DestroyServer and CreateServer needs to be replaced with DestroyConnection and CreateConnection, respectively.

upon reimporting the package into the assets folder, i get ~70 errors telling me different types of namespaces cannot be found. how can I deal with this? Thanks


bump, i got the same issue