Microsoft-Llama ONNX fails to import, what does this error message mean?

AI Beta Sentis
1

I attempted to import an off-the-shelf Llama ONNX model by Microsoft.
LlamaV2_7B_FT_float16.onnx

I am seeing this import error. What does it mean? Some sort of not supported operation or opset?

Asset import failed, "Assets/Models/llama/ONNX/LlamaV2_7B_FT_float16.onnx" > InvalidOperationException: Sequence contains no matching element
System.Linq.Enumerable.Single[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) (at <8b4edcb4eeea4daaa8ad5fcc9d6d39f1>:0)
Unity.Sentis.ONNX.ONNXConstantsLoader.LoadConstant (Onnx.TensorProto tensorProto, System.IO.FileStream weightStream) (at /Users/username/Downloads/package 2/Runtime/ONNX/ONNXConstantsLoader.cs:27)
2

Eh, I asked Bing Chat. Probably a complex number operation I guess based on another thread and Bing’s response.

This Unity Sentis asset import error could mean that the ONNX model file you are trying to import has some issues with its tensor data or its operators. According to the Sentis documentation, Sentis can import most ONNX model files with an opset version between 7 and 15, but it does not support the following:

  • Models that use tensors with more than 8 dimensions.
  • Sparse input tensors or constants.
  • String tensors.
  • Complex number tensors.

Sentis also converts some tensor data types like bools to floats or ints, which might increase the memory your model uses. Additionally, some operators might not be supported by Sentis, and you might need to add a custom layer to implement them.

The error message you got indicates that the ONNXConstantsLoader class could not find a matching element in the tensorProto parameter, which is a TensorProto object that represents a tensor in the ONNX model. This could be because the tensor data is not compatible with Sentis, or the tensor name is not specified correctly.

To fix this error, you could try the following steps:

  • Check the opset version of your ONNX model file and make sure it is within the range supported by Sentis.
  • Check the tensor data types and dimensions of your ONNX model file and make sure they are compatible with Sentis.
  • Check the tensor names and operators of your ONNX model file and make sure they are consistent and supported by Sentis.
  • If you have any custom operators in your ONNX model file, make sure you have added a custom layer to implement them in Sentis.
  • Reimport the ONNX model file and see if the error persists.
3

The appendix (“_float16”) of the file name indicates that the model may use half precision floats. I guess unity sentis currently doesn’t support half precision values.

4

Can you try with 1.3? I’ve fixed this issue for llama. The problem was that we didn’t handle external weights that were split up in different files