Microsoft-Llama ONNX fails to import, what does this error message mean?

I attempted to import an off-the-shelf Llama ONNX model by Microsoft.
LlamaV2_7B_FT_float16.onnx

I am seeing this import error. What does it mean? Some sort of not supported operation or opset?

Asset import failed, "Assets/Models/llama/ONNX/LlamaV2_7B_FT_float16.onnx" > InvalidOperationException: Sequence contains no matching element
System.Linq.Enumerable.Single[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) (at <8b4edcb4eeea4daaa8ad5fcc9d6d39f1>:0)
Unity.Sentis.ONNX.ONNXConstantsLoader.LoadConstant (Onnx.TensorProto tensorProto, System.IO.FileStream weightStream) (at /Users/username/Downloads/package 2/Runtime/ONNX/ONNXConstantsLoader.cs:27)

Eh, I asked Bing Chat. Probably a complex number operation I guess based on another thread and Bing’s response.

This Unity Sentis asset import error could mean that the ONNX model file you are trying to import has some issues with its tensor data or its operators. According to the Sentis documentation, Sentis can import most ONNX model files with an opset version between 7 and 15, but it does not support the following:

  • Models that use tensors with more than 8 dimensions.
  • Sparse input tensors or constants.
  • String tensors.
  • Complex number tensors.

Sentis also converts some tensor data types like bools to floats or ints, which might increase the memory your model uses. Additionally, some operators might not be supported by Sentis, and you might need to add a custom layer to implement them.

The error message you got indicates that the ONNXConstantsLoader class could not find a matching element in the tensorProto parameter, which is a TensorProto object that represents a tensor in the ONNX model. This could be because the tensor data is not compatible with Sentis, or the tensor name is not specified correctly.

To fix this error, you could try the following steps:

  • Check the opset version of your ONNX model file and make sure it is within the range supported by Sentis.
  • Check the tensor data types and dimensions of your ONNX model file and make sure they are compatible with Sentis.
  • Check the tensor names and operators of your ONNX model file and make sure they are consistent and supported by Sentis.
  • If you have any custom operators in your ONNX model file, make sure you have added a custom layer to implement them in Sentis.
  • Reimport the ONNX model file and see if the error persists.

The appendix (“_float16”) of the file name indicates that the model may use half precision floats. I guess unity sentis currently doesn’t support half precision values.

Can you try with 1.3? I’ve fixed this issue for llama. The problem was that we didn’t handle external weights that were split up in different files

1 Like

It seems to be importing. It hasn’t finished yet.
The first system I tried it on it gave a out of memory error and crashed.
The system I am running it on now is much beefier, so will see. It did pause already and said to close other applications because it ran out of memory.

I am guessing the out of memory errors relate to this and maybe the float16 issue.Taking over 8 minutes and counting to import a 1.6GB onnx model - #6 by yoonitee

Here’s the error message from the first system. A Windows Lenovo laptop with 16GB RAM and Nvidia Geforce RTX 2060.

2nd system froze up, so I killed the process
After re-opening the project, it looks like it may have imported successfully. I’ll test and find out.

On my other system, I’m going to try out float_32 instead.

2nd system was unable to import, so 16GB is not enough RAM for either of those models I guess.