I’m running a network every frame with a fairly large output tensor ((56x80x80) - I mention this because the leak is much more apparent with large outputs), and found that outputTensor.MakeReadable() steadily increases my program’s memory use each frame. When I remove this line the memory use is constant, however I need to be able to do further calculations on the cpu.
I initialize the output tensor with the “using TensorFloat …” statement as in the examples. The only difference is just the network and having worker.Execute() and worker.PeekOutput() in the Update() call.
Thanks!
1 Like
I agree with you. Even a most simple code like this (which does nothing) definitely leaks memory.
using UnityEngine;
using Unity.Sentis;
public class TestScript : MonoBehaviour
{
[SerializeField] private ModelAsset asset = null;
private Model model = null;
private IWorker worker = null;
private void Start()
{
model = ModelLoader.Load(asset);
worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, model);
}
private void Update()
{
var width = model.inputs[0].shape[3].value;
var height = model.inputs[0].shape[2].value;
var channels = model.inputs[0].shape[1].value;
var texture = new Texture2D(width, height, TextureFormat.RGBA32, false);
var input = TextureConverter.ToTensor(texture, width, height, channels);
worker.Execute(input);
var output = worker.PeekOutput();
output.MakeReadable();
/* post process */
input.Dispose();
output.Dispose();
Destroy(texture);
}
private void OnDestroy()
{
worker?.Dispose();
worker = null;
}
}
- Unity 2022.3.11f1
- Sentis 1.2.0-exp.2
Disposing the output without taking ownership of it is a no go, it’ll mess up the allocator and force us to need to re-allocate it every frame.
Can you try to not Dispose of it?
Also do post a picture of the profiler window ok? help us narrow down the issue to either gpu or cpu excessive allocations
But we’ll investigate this issue it sounds fishy
My code is similar to Sugiura’s:
using UnityEngine;
using Unity.Sentis;
public class RunNetwork : MonoBehaviour
{
Texture2D tex;
public ModelAsset modelAsset;
Model runtimeModel;
IWorker worker;
void Start()
{
runtimeModel = ModelLoader.Load(modelAsset);
worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, runtimeModel);
}
void Update()
{
tex = new Texture2D(640, 640, TextureFormat.RGB24, false);
using TensorFloat inputTensor = TextureConverter.ToTensor(tex, width:640, height:640, channels:3);
worker.Execute(inputTensor);
using TensorFloat outputTensor = worker.PeekOutput("output2") as TensorFloat; // output w/ shape (1, 56, 1, 80, 80)
outputTensor.MakeReadable(); // commenting this out stops memory increase
Destroy(tex);
}
void OnDisable()
{
worker.Dispose();
}
}
Memory profiler pic and comparison of snapshots about 30 seconds apart:
Getting rid of the “using TensorFloat” and taking ownership of the output tensor after calling PeekOutput() didn’t seem to fix the issue.
Unity 2022.3.12f1
Sentis 1.2.0-exp.2
Mac M1
1 Like
I learned that Tensor.DeepCopy() can solve this problem.
So, it seems to be working without memory leak for now.
Thanks,
//var output = worker.PeekOutput();
var output = worker.PeekOutput().DeepCopy();
I had missed the existence of the Tensor.TakeOwnership() method. It works perfectly! 
var output = worker.PeekOutput();
output.TakeOwnership();
output.MakeReadable();
/* post process */
output.Dispose();
1 Like
Sorry I’m late to the party and haven’t checked your code.
Glad you got a workaround.
What your code is doing is manually telling our allocator to not handle the output.
Which you are making readable. So dumping the compute buffer and converting that to cpu data. And then finally disposing of the tensor.
It still points to some weird behaviour with our allocator in which probably we don’t recover the computebuffer or something, causing in too much allocations, or not enough re-use… causing a memory increase
I’ll investigate and fix the issue.
Thank you for raising the issue
2 Likes
This is internally known as issue 209
