SAM2 encoder has error occurred with GPU inference

Unity Engine
,
1

I exported ONNX model (sam2_hiera_tiny_encoder.onnx) using the notebook in the following repository to implement inference for SAM2 with Unity Sentis.

I had written the following simple minimum code to test inference with encoder model.
This test code works without error when the worker backend type is CPU.
But, an error occurs when worker backend type is GPU.

using Unity.Sentis;
using UnityEngine;
using Unity.Sentis;

public class Script : MonoBehaviour
{
    [SerializeField] private ModelAsset model_asset;

    void Start()
    {
        // load model
        var model = ModelLoader.Load(model_asset);

        // create worker with gpu backend type
        // var worker = WorkerFactory.CreateWorker(BackendType.CPU, model); // ok
        var worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, model); // error

        // dummy input (input shape is 1x3x1024x1024)
        var input_texture = new Texture2D(1024, 1024, TextureFormat.ARGB32, false);
        var input_tensor = TextureConverter.ToTensor(input_texture, 1024, 1024, 3);

        // worker execute
        worker.Execute(input_tensor);

        // peek output
        var high_res_feats_0 = worker.PeekOutput("high_res_feats_0");
        var high_res_feats_1 = worker.PeekOutput("high_res_feats_1");
        var image_embed = worker.PeekOutput("image_embed");

        // show output tensor shape
        Debug.Log(high_res_feats_0.shape);
        Debug.Log(high_res_feats_1.shape);
        Debug.Log(image_embed.shape);

        // dispose worker
        worker.Dispose();
    }
}

The error message when worker backend type is GPU is here.
Is this a bug?

Thread group count is above the maximum allowed limit. Maximum allowed thread group count is 65535.
UnityEngine.ComputeShader:Dispatch (int,int,int,int)
Unity.Sentis.ComputeHelper:Dispatch (Unity.Sentis.ComputeFunction,int,int,int) (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Backends/GPUCompute/ComputeHelper.cs:149)
Unity.Sentis.GPUComputeBackend:GlobalAverageVariancePool (Unity.Sentis.TensorFloat,Unity.Sentis.TensorFloat,int) (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Backends/GPUCompute/GPUCompute.cs:1013)
Unity.Sentis.GPUComputeBackend:LayerNormalization (Unity.Sentis.TensorFloat,Unity.Sentis.TensorFloat,Unity.Sentis.TensorFloat,Unity.Sentis.TensorFloat,single) (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Backends/GPUCompute/GPUCompute.cs:1229)
Unity.Sentis.Layers.LayerNormalization:Execute (Unity.Sentis.ExecutionContext) (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Layers/Layer.Normalization.cs:160)
Unity.Sentis.GenericWorker:Execute () (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Backends/GenericWorker.cs:137)
Unity.Sentis.GenericWorker:Execute (Unity.Sentis.Tensor) (at ./Library/PackageCache/com.unity.sentis/Runtime/Core/Backends/GenericWorker.cs:106)
BugReport:Start () (at Assets/BugReport.cs:17)

gpucompute
gpucompute856×697 119 KB

My environment is here.

  • OS : Windows 11
  • Unity Editor : 6000.0.3f1
  • Unity Sentis : 1.6.0-pre.1
2

Thanks, we take note.
One of our kernel does not handle big inputs and runs out of threads.
We can patch that in a future release, but in the meantime try using smaller inputs

1 Like
3

Thanks, I will wait and look forward to the fix update. :+1:

4

Hi, it’s been 1 year, the issue is still there. Are there plans to solve this soon? Thanks.

5

Do you still need this?