Dynamically loading Onnx file and setting as the model for an Agent

Unity Engine
, ,
1

I have a .onnx file from a trained model that’s stored on the application persistentpath. I want to load it at runtime and set it as the model for an Agent. Now, the Agent expects a NNModel in its SetModel method. This is what I’ve tried

var modelPath = NNHelper.GetOnnxPath(Species, selectedBrain.Brain);
var modelData = File.ReadAllBytes(modelPath);
NNModel model = ScriptableObject.CreateInstance<NNModel>();
model.modelData = ScriptableObject.CreateInstance<NNModelData>();
model.modelData.Value = modelData;
GetComponent<FishAgent>().SetModel("Fishy", model);

This results in the NotSupportedException: Format version not supported: 242 error since Barracuda doesn’t support Onnx files apparently. Online I’ve read some people have used the ONNXConverter, however, that converts the byte[ ] to a Model, where a byte[ ] is required for the modelData.Value.

Any suggestions?

2

Fixed it. If anyone wants to know, take a look at the ONNXModelImporter.

3

Thanks a bunch, man. Your hint came in clutch for me.

But to kick the can a few meters down the streets and save people some time and confusion, I’ll give a more specific answer for the folks in the future:

Specifically, look at the implementation of the OnImportAsset(AssetImportContext ctx) method in the ONNXModelImporter class and do what they did there.

I came up with this:

NNModel LoadNNModel(string modelPath, string modelName)
{
    var converter = new ONNXModelConverter(true);
    Model model = converter.Convert(modelPath);
    NNModelData modelData = ScriptableObject.CreateInstance<NNModelData>();
    using (var memoryStream = new MemoryStream())
    using (var writer = new BinaryWriter(memoryStream))
    {
        ModelWriter.Save(writer, model);
        modelData.Value = memoryStream.ToArray();
    }
    modelData.name = "Data";
    modelData.hideFlags = HideFlags.HideInHierarchy;
    NNModel result = ScriptableObject.CreateInstance<NNModel>();
    result.modelData = modelData;
    result.name = modelName;
    return result;
}

:

1 Like
4

It may depend on the version of ML-Agents and Barracuda you are using, but I was able to update onnx files for ML-Agents dynamically with the following code. The versions I used were ml-agents=v2.0.1 and barracuda=2.0.

using Unity.MLAgents;
using Unity.MLAgents.Policies;
using Unity.Barracuda;

// Difficulty Levels
public enum BattleMode
{
   Easy,
   Medium,
   Hard
}

// Unity.Barracuda.NNModel class
public NNModel easyBattleBrain = "BossBattle-easy.onnx";
public NNModel mediumBattleBrain = "BossBattle-medium.onnx";
public NNModel hardBattleBrain = "BossBattle-hard.onnx";

// Should match with Agent's Behavior Name in Unity's UI
public string behaviorName = "BossBattle";

public void setDifficulty(BattleMode mode, Agent agent)
{
   switch (mode)
   {
       case BattleMode.Easy:
           agent.SetModel(behaviorName, easyBattleBrain);
           break;
       case BattleMode.Medium:
           agent.SetModel(behaviorName, mediumBattleBrain);
           break;
       case BattleMode.Hard:
           agent.SetModel(behaviorName, hardBattleBrain);
           break;
       default:
           break;
   }
}

We have also included a tutorial for the ML-Agents demo we developed at this URL. Please feel free to use it as a reference.
I hope this information is helpful.

1 Like