ONNX If operator unsupported

Unity Engine
, ,
1

Hi,

I’m trying to bring some custom trained object detection models (yolov5_s and rtmdet_tiny) into Unity but I’m facing the issue that Sentis doesn’t support the If operator in the converted .onnx formats.

I’m using mmyolo to train and convert the models.

I was wondering if someone could help me figure out a way forward. Are people using some other architecture for object detection? A different way of converting that results in a compatible .onnx file?

2

Having an If in your model is most likely due to some bad export.
Fixing input shapes can help remove it.

I’ve seen many yolov5 or v8 that do not have if inside so it is possible.

Usually ifs are mostly handling shape logic. In this case you can override the if layer in our importer (cf Samples\Add a custom layer) and perform the math operated by the if layer there

1 Like
3

Thanks for the ideas, I’ll take a closer look!

4

If you share your model I can probably guide you in the right direction

5

Actually, that would be great. You can find a sample model I trained based on the same rtmdet architecture here:

https://drive.google.com/drive/folders/18ABwue3sefMqnopSAk70EMDPgIbSxU7X?usp=share_link

It’s based on rtmdet_tiny_fast_1xb12-40e_cat.py config from the mmyolo repo https://github.com/open-mmlab/mmyolo/tree/main/configs/rtmdet

The model expects an image in resolution (640, 640)

6

Couldn’t put this third link in the previous post. I basically just followed the mmyolo documentation with a custom dataset I prepared
https://mmyolo.readthedocs.io/en/dev/get_started/15_minutes_object_detection.html#training

I was in the process of debugging the model, but didn’t have too much time today.

I did find out the last operators in the .onnx model were

[..., 'Shape', 'Gather', 'Constant', 'Equal', 'If', 'Constant', 'Greater', 'Cast', 'ReduceSum', 'Cast']

That’s the only If operator and it’s the end part.

7

image
image696×756 19.2 KB

You can see what the if does.
It checks the output of Equal (which is coming from shape->gather[index:-1]) and if it’s true then it squeezed the shape else gives identity

so it’s doing
X.shape[-1] == 1 ? X.squeeze : X

  • one you don’t need to do this with an if it’s too heavy. so you can just remove it from your model
  • two you can overload the if operator (cf custom layer sample) to do just that
1 Like
8

I ran into the same problem when using custom trained model , I used onnxsim , it simplified the model to remove ‘if’ operators, Hope this helps someone.