Issues with Sentis v1.1.0

Unity Engine
,
1

Thank you for the Sentis update! And sorry to say this, but I have issues with v1.1.0:

  • the shape of model outputs in Inspector are almost always ‘?’. This was not like this before.
  • more importantly, the inference results are different than before (and also different than ORT outputs), at least for some models. Here is one sample model, will provide you more later.
1 Like
2

I was also getting odd results. But then I did a “reimport all” and seemed to fix them.

But I am getting some extra outputs in one model. (Can’t remember if they were there before):

screenshoty
screenshoty441×676 29.7 KB

Still works but I don’t remember those “key.#” and “value.#” outputs being there before :thinking: A clue is when I click on the model asset without the inspector window open I get an error:

ArgumentException: An item with the same key has already been added. Key: 
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) (at <1c8569827291471e9db0dcd976e97952>:0)
Unity.Sentis.ModelLoader.LoadModelDesc (System.IO.Stream stream, Unity.Sentis.Model& model) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/ModelLoader.cs:147)
Unity.Sentis.ModelLoader.LoadModelDesc (Unity.Sentis.ModelAsset modelAsset, Unity.Sentis.Model& model) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/ModelLoader.cs:59)
Unity.Sentis.Editor.ModelAssetEditor.CreateInspectorGUI () (at <5762c911b23d4eaeaedb54029f24e979>:0)

PS.
In the layers list, it would be nice to be able to make it searchable. Or alternatively to be able to copy and paste the data into a notepad to search it in there. For example I wanted to search it to see if for example “key.59” appeared in the layers.

1 Like
4

Ok thanks for providing the model, we’ll take a look at it.

  • shape of the output, compared to 1.0 to speedup UI display we don’t do full shape inference for the UI elements.
    So more output shapes will be unknown
  • inference result, yes re-import the model to be sure. if it is still wrong we’ll fix it
  • searchable UI, yes we could do that in a future release
1 Like
5

Thank you for reporting this!

  • Unfortunately I could not reproduce the error when clicking on a model asset without the inspector window open. If you know the exact steps or a project that reproduces the issue, you can upload it using SendSafely: Unity

  • For searchable UI, I’ve added a task for it, so hopefully it will make it in a future release

6

@liutaurasvysniauskas_unity Which model are you referring and what issue could you not reproduce?

7

I’m referring to yoonitee’s message

ArgumentException: An item with the same key has already been added. Key: 
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <1c8569827291471e9db0dcd976e97952>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) (at <1c8569827291471e9db0dcd976e97952>:0)
Unity.Sentis.ModelLoader.LoadModelDesc (System.IO.Stream stream, Unity.Sentis.Model& model) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/ModelLoader.cs:147)
Unity.Sentis.ModelLoader.LoadModelDesc (Unity.Sentis.ModelAsset modelAsset, Unity.Sentis.Model& model) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/ModelLoader.cs:59)
Unity.Sentis.Editor.ModelAssetEditor.CreateInspectorGUI () (at <5762c911b23d4eaeaedb54029f24e979>:0)
8

Is there any update on the wrong output of the model in the first post?

9

Yes I’m taking a look at this now. Will let you know when I have more information or need to know anything more.

1 Like
10

Ok we believe we found the issue, until we publish a fix you will have to uncheck ‘Optimize Model’ in the importer. Can you verify this fixes the incorrect inference?

11

Just to clarify roumenf’s issue was a different one to mine I think. Sorry, should have started a different thread. I’ll see if I can post the model that caused the issue.

12

When the model optimization is turned off, I get this error:

InvalidOperationException: Tensor data cannot be read from, use .MakeReadable() to allow reading from tensor.
Unity.Sentis.Tensor.ToReadOnlySpan[T] () (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/Tensor.cs:249)
Unity.Sentis.Layers.Clip.Execute (Unity.Sentis.Tensor[] inputs, Unity.Sentis.ExecutionContext ctx) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/Layers/Layer.Math.cs:140)
Unity.Sentis.GenericWorker+<StartManualSchedule>d__33.MoveNext () (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/Backends/GenericWorker.cs:234)
Unity.Sentis.GenericWorker.Execute () (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/Backends/GenericWorker.cs:167)
Unity.Sentis.GenericWorker.Execute (Unity.Sentis.Tensor input) (at Library/PackageCache/com.unity.sentis@1.1.0-exp.2/Runtime/Core/Backends/GenericWorker.cs:159)

The input tensor is converted from a texture with ‘TextureConverter.ToTensor()’.

13

Ah thanks for reporting this issue, I can see what the problem is, unfortunately our testing didn’t uncover this. This is known as issue 113 internally and we will try and address it soon.

1 Like
14

@gilescoope Here is one more model that doesn’t infer well, if you’d like to take a look.

15

Thanks for these models, I can replicate your issues with inference on GPU for both these models. I will raise bugs internally and get back to you when we have a fix.

2 Likes
16

Good afternoon,

I found another example that may be another data point. I have a YOLOv8 model that is showing output discrepancies between Sentis v1.0 and v1.1. The imported model is also showing “??” for some of the inputs when re-imported from 1.0 to 1.1.

I’ve attached an example script and model HERE, the debug log for v1.0 outputs 4 detections (correct) while the v1.1 outputs 24 (incorrect).

Thanks

17

The ?? in the UI is normal, we disabled deep shape inference on the UI to speedup display of the inspector.
If you have numerical precision issues then we can investigate

18

I suppose the YOLO-model also has precision issues, as the other two before. By the way, is it feasible to have a button in the model UI that can call the deep shape inference (on user’s request) and refine the output (or input) shapes?

19

That’s a good idea, we can add that in the UI

20

We just published Sentis 1.1.1-exp.2 which should fix the above inference issues along with the UI crashing issue. You will have to manually reimport affected models after you update Sentis.

Please let us know how this goes.

21

After some testing, my YOLOv8 model seems to be working as expected now with latest update.

Much appreciated!

1 Like