Upgrade to Sentis 1.4

AI & Machine Learning AI Inference Engine
,
1

Is there a way to upgrade from Sentis v1.3 to v1.4 without removing the package from Package manager?

2

You can update the “<UnityProject>\Packages\manifest.json” file on the com.unity.sentis line with “1.4.0-pre.3”.

3

Yes, it works. Thank you!
But I have few more questions in this regard, if you don’t mind:

  1. Why doesn’t package upgrade work in the Package manager UI, by seeing a new version and pressing a button?
  2. Why are there (so many) breaking changes? Will this continue in the next updates?
  3. Why were the sentis-files exported so far invalidated, and required to be re-exported? If this continues in the subsequent releases, it would render the sentis-files hardly usable in production apps.
5
  1. This may be because it is a pre-release version so you may not see an upgrade button. But it is possible to update in the package manager like this.
  • Click on the plus:
    image
    Click on “Install package by name” Then you can type in “com.unity.sentis” and it should give the latest version. Or you can specify the version number 1.4.0-pre.3

  1. Apologies for the breaking changes. 1.4.0 was a major update to the API. This was felt necessary to bring a more unified approach to the API and allows for speed ups and prevention of accidental memory leaks. For those coming from using torch and python it should also be more familiar. We will be pleased to hear any feedback you have on this and are here to help with code upgrades. There are (probably!) not expected to be any other major updates to the API except for small improvements.

  2. The technical reason is Sentis files have changed the serialization format of the protobuf in order that they are future proof and able to be upgraded automatically. From 1.4.0 onwards they should update automatically to the latest version. Unfortunately it does mean, that to update to 1.4.0 you will need to re-serialize the onnx files using the button in the inspector or else download new versions from Hugging Face.

Sorry for the inconvenience as we knew this big update would cause issues for some people. Of course you’re welcome to continue using the 1.3.0-pre.3 version for any applications you have currently made but we would recommend updating to 1.4.0 for future projects to get the improvements.

6

Thank you for the heads up! It sounds reasonable. The auto-update of sentis-files would be a great feature.
Here’s one final request. When you have a minute, can you please help me translate the following code snippet to the new API:

var output = _sentisModel.outputs[0];

_sentisModel.layers.Add(new Unity.Sentis.Layers.ReduceMax("max0", new[] { output }, false));
_sentisModel.layers.Add(new Unity.Sentis.Layers.ReduceMin("min0", new[] { output }, false));
_sentisModel.layers.Add(new Unity.Sentis.Layers.Sub("new_output", "max0", "min0"));

_sentisModel.outputs = new List<string>() { "new_output" };
7

Hi yes. The way the new API works is you can create a new model with modified outputs like this. Here is the new code assuming your original model had two inputs:

 var _sentisModelWithNewOutput = Functional.Compile(
            (input1, input2) =>
            {
                var output = _sentisModel.Forward(input1, input2)[0];
                return Functional.ReduceMax(output, -1) - Functional.ReduceMin(output, -1);
            },
            (_sentisModel.inputs[0], _sentisModel.inputs[1])
        );

The ReduceMax is just on the last dimension but you can add more to the array such as new[]{-1,-2,..}. The tuple of inputs at the end are give the types of the input (e.g. float/int and shape), you can get them from the previous model or define them using the InputDef class.

There are a couple of more changes you need to make.

(1) Make sure your worker now references the new modified model:

        var worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, _sentisModelWithNewOutput);

When creating a model with functional.compile the inputs are automatically named like this:

        worker.Execute(new Dictionary<string, Tensor>() { { "input_0", input0 }, { "input_1", input1 } });

        var output = worker.PeekOutput("output_0") as TensorFloat;

        //This used to be MakeReadable()
        output.CompleteOperationsAndDownload();

Hope that helps.