Jets TTS on Sentis (AI Inference Engine)

Unity Engine
, , ,
1

Hello everyone,

It’s been a while that I’ve been trying to train my own TTS model to run in Unity.
Unity has this Jets TTS example available on HuggingFace and it works fine.
So I started to work on my own model. I managed to train my Jets TTS model using Espnet and I can export it to ONNX using espnet_onnx.

When I import it to Unity, Sentis (now AI Inference Engine) simply rejects it as it has If Operators. If I try remove the If Operators, I still get different errors.

Apparently, there’s an specific way to export the model to ONNX that Unity can handle.
I looked everywhere. I tried to export in different ways, but simply got stuck.

Does anyone happen to know something to help me here ?

Thanks,

Diego

1 Like
2

Unfortunately we do not currently support the If operator (Supported ONNX operators | Inference Engine | 2.3.0) so that’s likely what you hit initially. We are looking to improve the coverage of operators on the roadmap, so we’ll take on board.

So, you’re right to try removing the Ifs. Could you share the new errors you’re seeing?

3

Hi Simon,

After removing the If operators and trying to import, Unity gives me this on the Console:
AssertionException: RankError: incorrect rank, expecting 2, got 4 Assertion failure. Values are not equal. Expected: 4 == 2 UnityEngine.Assertions.Assert.Fail (System.String message, System.String userMessage)

After some digging, I managed to flatten the ranks to 2 since that’s what Sentis supposedly expects.
The model gets imported, but once I try to run an inference I get an error like this:
AssertionException: TensorShape.ValueError: operands could not be broadcast together with shapes (5, 256), (1, 256, 256) Assertion failure. Value was False

So it feels the whole model is really different from the original model on HuggingFace.
When you check the model on Netron, there’s a torch_jit, which implies it could have been traced and that leads to my orignal question. The current model on HuggingFace must have gone through some specific torchscript to export it in a way that Sentis can read.

Thanks again,

Diego