Problem getting started with Sentis

Unity Engine
, , , ,
1

Hello!

I am trying to include an AI model that predicts an angle for character animation, but I seem to have problems with the API’s.

So I basically coded everything in keras and now learned that Sentis only accepts ONNX files. There was a package called keras2onnx who used to do the trick, but it isn’t supported anymore. It is now part of tf2onnx, which I have tried using, but still doesn’t seem to work due to an error related to my input layer. I have tried fixing that too, but the problem seems to be underdocumented too.

Do you have any alternatives I might have missed? My last solution would be to recode my AI in pytorch and try my hand at converting pth to onnx, which seems to be embedded into pytorch this time, but there has to be a simpler way. Do you have any idea how to get past this?

Thanks in advance!

2

Yeah, I noticed it too that tf2onnx isn’t the best these days.
I’d say to go with pytorch for now.
But for sure that bumps up the prio for us to do better workflow coming from tensorflow

3

It would be very useful to add compatibility with other formats like .keras or .h5. If it were feasable until the end of August, it would be very useful for my purpose. Thanks for replying!

4

What’s your keras model?

5

The model is very basic as I was trying out different architectures for my internship:

def get_RNN_model(shape): # RNN
    model = Sequential()
    model.add(Embedding(input_dim=1000, output_dim=64))
    model.add(GRU(256, return_sequences=True))
    model.add(SimpleRNN(128))
    model.add(Dense(1))

    model.compile(optimizer   = 'adam',
                    loss      = 'mse',              # MSE used in DOI = {10.1111/cgf.14628} 
                    metrics   = ['mae', 'msle'])    # MSLE used in DOI = {10.1111/cgf.14635}
    return model

I have used chatGPT to translate it into pytorch, since I haven’t used it enough to know how to create a pytorch project and it seems immediate. My problem is that I need to read two separate pickle files (for train+test and validation) and then to process them with some more mathematical transformations, but trying this using pytorch is pretty confusing.

6

Given your problem space and scope, I’d suggest you move to torch