How to obtain multiple outputs from the original model within Functional.Compile()?

Unity Engine
, , , ,
1

Hello,

Just had a quick question about the Functional library, especially when compiling a model with Functional methods. I have the following code:

FunctionalTensor[] ModelWithPrePostProcessing(FunctionalTensor[] inputs)
        {   
            // Define mean and standard deviation for model input
            FunctionalTensor mean = FunctionalTensor.FromTensor(TensorFloat.AllocZeros(new TensorShape(3,modelInputHeight,modelInputWidth)));
            mean[0] = FunctionalTensor.FromTensor(new TensorFloat(103.53f));
            mean[1] = FunctionalTensor.FromTensor(new TensorFloat(116.28f));
            mean[2] = FunctionalTensor.FromTensor(new TensorFloat(123.675f));

            FunctionalTensor std = FunctionalTensor.FromTensor(TensorFloat.AllocZeros(new TensorShape(3,modelInputHeight,modelInputWidth)));
            std[0] = FunctionalTensor.FromTensor(new TensorFloat(57.375f));
            std[1] = FunctionalTensor.FromTensor(new TensorFloat(57.12f));
            std[2] = FunctionalTensor.FromTensor(new TensorFloat(58.395f));

            FunctionalTensor meanAndStdInput =  (inputs[0] - mean)/std;
            FunctionalTensor[] outputs = model.Forward(meanAndStdInput);
        
            return outputs;
        }

The original model (being used on line 15 model.Forward() ), originally returns 3 outputs, as shown in the netron graph below:

Screenshot from 2024-07-30 17-06-20

However, I am stumped as to how I can get obtain all the three outputs (dets,labels and masks) within this function and how do I return these outputs from the compiled model too?

Currently, it just returns the first output dets with the size [1,41,5]. I am unable to access the other two outputs.

Thanks in advance.

2

You’re actually very close!

FunctionalTensor[] outputs = model.Forward(meanAndStdInput);

output has 3 values

dets = outputs[0];
labels = outputs[1];
masks = output[2];

Now after you compile the model, you’ll see that the model will have 3 outputs. (“output_0”, “output_1”, “output_2”)
You can use these names to query the worker to get the results

// Compile a new model from the source model using the functional API to transform the inputs.
m_RuntimeModel = Functional.Compile(ModelWithPrePostProcessing);
// Create worker to run the model.
m_Worker = WorkerFactory.CreateWorker(backendType, m_RuntimeModel);

m_Worker.Execute(input);

var dets = m_Worker.PeekOutput("output_0") as TensorFloat;
var labels = m_Worker.PeekOutput("output_1") as TensorInt;
var masks = m_Worker.PeekOutput("output_2") as TensorFloat;