There was a memory leak during multiple forward inference in the same frame

AI Beta Sentis
1

My model is autoregressive, but there was a memory leak during multiple forward inferences in the same frame. How to solve it? Thanks in advance!

The code is below:

public TensorFloat sample(TensorFloat original_data, TensorFloat x_t)
{
for (int i = 15; i >= 0; i–)
{
Dictionary<string, Tensor> inputTensors = new Dictionary<string, Tensor>()
{
{ “original_data”, original_data},
{ “x_t”, x_t },
};
worker.Execute(inputTensors);
x_t = worker.PeekOutput(“output”) as TensorFloat;
}
return x_t;
}

Unity report is:

image
image1207×538 97.5 KB

2

So I assume that you’ve allocated the original x_t right?
Then you assign the output of your model to the x_t you own.
So you have a dangling reference and haven’t disposed the original input tensor.

Also make sure to dispose the worker.

tensors with PeekOutput don’t need to be disposed unless you takeownership of them

3 
Dictionary<string, Tensor> inputTensors = new Dictionary<string, Tensor>()
{
{ “original_data”, original_data},
{ “x_t”, x_t },
};
worker.Execute(inputTensors);
output = worker.PeekOutput(“output”) as TensorFloat;
x_t.Dispose();
output.TakeOwnership();
x_t = output;

this is the way to do it