Cast Tensor to TensorFloat

Unity Engine
,
1

Hey there :wave: ,

First of all, I found the provided examples and samples very clear thanks to the people who wrote them :hugs:

I’m currently testing Sentis with a Sentence Similarity Hugging Face Model. The inference is working correctly (I wrote the Tokenizer in C#) which is a first win :joy:

My problem is in the post-processing, this model takes sentences [“I love dogs”, “I love cats”] and embed. The idea is to calculate the cos similarity of these embeds.

But first, I need to do a MeanPool of the embeds. And the first step of this MeanPool is doing the equivalent of torch.unsqueeze(-1).

Given this Unsqueeze method does not exist in Ops what I’ve done instead was using Expand but the problem is that this returns a Tensor not a TensorFloat. And the next step is a MatMul between two TensorFloat. So I need to cast my Tensor to a TensorFloat.

I don’t find how to cast a Tensor to TensorFloat in Sentis. Or methods in Tensor to transform a Tensor to a TensorFloat.

Thanks for your help :hugs:

2

Few things,

  • Unsqueeze exists indirectly you can do tensor.ShallowReshape(tensor.shape.Unsqueeze(-1)); or ops.Reshape(tensor, tensor.shape.Unsqueeze(-1)). But thanks for pointing it out, we’ll add it to the api.
  • MeanPool you can use ops.ReduceMean/ops.AveragePool
  • Tensor is a abstract class that is implemented either by TensorInt/TensorFloat.
    If you know your tensor is a TensorFloat (tensor.dataType == Float) then you can simply cast it.
    tensor as TensorFloat
2 Likes
3

Can to share the tokenizer? I’ve been working on text to speech and have not done that part. Model didn't import: ljspeech-jets-onnx - #21 by mrwellmann

4

Thanks for the help I’m testing it today :hugs:

5

I’m finishing cleaning the code and I plan to share the tokenizer for a tutorial this week :smile:

2 Likes
6

Is this the tutorial Building a smart Robot AI using Hugging Face 🤗 and Unity you where talking about?

I haven’t had much time to fix my tokenizer but with some help i got lately I think I’m almost there.

7

No it’s not this one. I’ve just published the tutorial: Create an AI Robot NPC using Hugging Face Transformers 🤗 and Unity Sentis

And the Sharp Transformers library: GitHub - huggingface/sharp-transformers: A Unity plugin for using Transformers models in Unity.

8

Thanks! Looks like something for my next free weekend :grin:

1 Like
9

It sounds like you’re making great progress with your tokenizer! While this isn’t the tutorial you mentioned, I’d love to hear more about your project. Building a smart robot AI using Hugging Face :hugs: and Unity sounds fascinating. If you need any assistance or have questions along the way, feel free to reach out. Keep up the excellent work!

10

I’m kind of done with the tokenizer. And by now its also kind of outdated there is a https://discussions.unity.com/t/suggestions there small text-to-speech model on Hugging Face now.

1 Like