MNIST example fails to recognize a lot of digits (any onnx mnist model)

just running this example, fails to recognize digits.

digit_9
digit_6
digit 8v2

Classifying 6 as a five, 8 and 9 as something else.
I tried various onnx models from here
(version 1.2, 1.3, 1.9) But they all produce wrong result.

After checking with breakpoints, I can see that tensor is correct, 1x1x28x28 but the softmax doesn’t seem to make any difference to the output (result values are below and above 1)

Is there is a bug in the way models are handled?
Also, is Softmax layer is working?

yes, the 7 from the example does work, but the images I’ve pasted are missed quite badly: the values in result are not close to the anticipated ones.

Using Sentis 1.2.0-exp.2 and unity 2023.1.11f1

I get the same result in ONNXRuntime, so it’s just the model’s error. Softmax is working, you probably missed the mathematical constant
image

Thank you!
Do you mean I need some kind of bias term for the softmax?

The thing is, when inspecting the already softmaxed values from the Unity example, the values are greater than 1.

The Softmax should produce a distribution of values which add to 1, hence I think there might be a bug with it. Have a look:

I think you’re applying the Softmax incorrectly.
This works for me:

// Add softmax layer to end of model instead of non-softmaxed output
string softmaxOutputName = "Softmax_Output";
runtimeModel.AddLayer(new Softmax(softmaxOutputName, runtimeModel.outputs[0]));
runtimeModel.outputs[0] = softmaxOutputName;
2 Likes

That solved it. The issue indeed was due to adding a layer but not specifying it as output. Thanks!

1 Like