How to batch input for a model

Unity Engine
, ,
1

Hi,
How can I concat input for the model using sentis?

For example

TensorShape color_shape = new TensorShape(1, 3, 480, 640);

TensorFloat color_tensor_0 = TextureConverter.ToTensor(color_data_0, channels: 3);
color_tensor_0.Reshape(color_shape);
TensorFloat color_tensor_1 = TextureConverter.ToTensor(color_data_1, channels: 3);
color_tensor_1.Reshape(color_shape);

Here the shapes of color_tensor_0 andcolor_tensor_1 are (1, 3, 480, 640). how can I concat them together to create a TensorFloat with shape (2, 3, 480, 640)?

2

You have two options:
1* use functional API to add a concat layer for your model
https://docs.unity3d.com/Packages/com.unity.sentis@2.0/manual/create-a-new-model.html

var graph = new FunctionalGraph();
var x = graph.AddInput<float>(new TensorShape(1, 3, 480, 640));
var y = graph.AddInput<float>(new TensorShape(1, 3, 480, 640));
var concat = Functional.Concat(new [] {x,y}, 0);

model = graph.Compile(Functional.Forward(concat));

2* write a compute shader that concats the two tensors
https://docs.unity3d.com/Packages/com.unity.sentis@2.0/manual/access-tensor-data-directly.html
Samples\Use a compute buffer

You can use either, but we can also add an option to concat a of images to a single tensor

3

Thank you for your suggestions! I might directly modify my model to accept more input, concatenate them, and then export a new ONNX model.

4

Oh by the way, I think it would be great if sentis could support more operations on TensorFloat!