*.onnx file load script at runtime.

jhlee0918 · February 18, 2022, 12:09pm

Hi, I have some problem using MLagents.

The onnx file was created through mlagents.
and that file drag&drop to the agent(behavior param) , it works without any problem.
I want to load and replace this onnx file at runtime(in script) , but the following message appears.

Is there any other way to solve this problem?

Thank you for read this…

[my code]

byte[] modelValues = File.ReadAllBytes(_modelPath);
Unity.Barracuda.NNModel nnmodel = ScriptableObject.CreateInstance<Unity.Barracuda.NNModel>();
nnmodel.modelData = ScriptableObject.CreateInstance<Unity.Barracuda.NNModelData>();
nnmodel.modelData.Value = modelValues;
nnmodel.name = "sa_agent";

rl_env.GetComponent<shooting_env>().AgentsList[0].Agent.GetComponent<BehaviorParameters>().Model = nnmodel;

[Message]
System.NotSupportedException: Format version not supported: 244
at Unity.Barracuda.ModelLoader+d__11.MoveNext () [0x00106] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:223
at Unity.Barracuda.ModelLoader.Load (System.IO.BinaryReader fileReader, System.Boolean verbose, System.Boolean applyPatching, System.Boolean skipWeights) [0x0002d] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:200
at Unity.Barracuda.ModelLoader.Load (System.Byte[ ] stream, System.Boolean verbose, System.Boolean skipWeights) [0x00001] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:140
at Unity.Barracuda.ModelLoader.Load (Unity.Barracuda.NNModel model, System.Boolean verbose, System.Boolean skipWeights) [0x00001] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:31
at Unity.MLAgents.Inference.ModelRunner…ctor (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.Int32 seed) [0x00085] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Inference\ModelRunner.cs:72
at Unity.MLAgents.Academy.GetOrCreateModelRunner (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice) [0x00036] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Academy.cs:620
at Unity.MLAgents.Policies.BarracudaPolicy…ctor (Unity.MLAgents.Actuators.ActionSpec actionSpec, System.Collections.Generic.IList`1[T] actuators, Unity.Barracuda.NNModel model, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.String behaviorName) [0x00008] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BarracudaPolicy.cs:84
at Unity.MLAgents.Policies.BehaviorParameters.GeneratePolicy (Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Actuators.ActuatorManager actuatorManager) [0x000b9] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:240
at Unity.MLAgents.Agent.ReloadPolicy () [0x00023] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Agent.cs:648
at Unity.MLAgents.Policies.BehaviorParameters.UpdateAgentPolicy () [0x00016] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:276
at Unity.MLAgents.Policies.BehaviorParameters.set_Model (Unity.Barracuda.NNModel value) [0x00008] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:108
at mapEvolvWithRL_SceneManager_controller.Initialized (System.String _loadMapFilePath, System.String _modelPath) [0x001af] in D:\project_junhyeoklee\GA_map_evolution\Assets\Scripts\SA\mapEvolvWithRL_SceneManager_controller.cs:337
UnityEngine.Debug:Log (object)
mapEvolvWithRL_SceneManager_controller:Initialized (string,string) (at Assets/Scripts/SA/mapEvolvWithRL_SceneManager_controller.cs:342)
mapEvolvWithRL_SceneManager_controller:Start () (at Assets/Scripts/SA/mapEvolvWithRL_SceneManager_controller.cs:194)

mbaske · February 18, 2022, 6:25pm

Maybe try loading the onnx model as an asset

and then assign it to the agent via SetModel()
https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/api/Unity.MLAgents.Agent.html#Unity_MLAgents_Agent_SetModel_System_String_NNModel_Unity_MLAgents_Policies_InferenceDevice_

ChillX · February 18, 2022, 10:53pm

This is a snippet of code from how I"m doing this.

if (CXMLTrainingManager.Instance == null)
{
    //Running in inference mode. Get the appropriate brain for each behavior based on type
    switch (AIType)
    {
        case Enum_AIType.Player:
            AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
            break;
        case Enum_AIType.Enemy:
            AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
            break;
        case Enum_AIType.NPC:
            AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
            break;
    }
    AIBehavior.BehaviorType = BehaviorType.InferenceOnly;
}
else
{
    // Training mode so null the NN model if training that agent. Else get the appropriate brain for each behavior based on type
    switch (AIType)
    {
        case Enum_AIType.Player:
            if (CXMLTrainingManager.Instance.Env_TrainPlayer) { AIBehavior.Model = null; }
            else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
            break;
        case Enum_AIType.Enemy:
            if (CXMLTrainingManager.Instance.Env_TrainEnemy) { AIBehavior.Model = null; }
            else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
            break;
        case Enum_AIType.NPC:
            if (CXMLTrainingManager.Instance.Env_TrainNPC) { AIBehavior.Model = null; }
            else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
            break;
    }
    if (ManualControl)
    {
        AIBehavior.BehaviorType = Unity.MLAgents.Policies.BehaviorType.HeuristicOnly;
    }
    else if (AIBehavior.Model == null)
    {
        AIBehavior.BehaviorType = BehaviorType.Default;
    }
    else
    {
        AIBehavior.BehaviorType = BehaviorType.InferenceOnly;
    }
}

This code is where I hold references to the various OMNX files.

        [SerializeField]
        private NNModel m_AIBrainPlayer;
        public NNModel AIBrainPlayer { get { return m_AIBrainPlayer; } }

        [SerializeField]
        private NNModel m_AIBrainEnemy;
        public NNModel AIBrainEnemy { get { return m_AIBrainEnemy; } }

        [SerializeField]
        private NNModel m_AIBrainBackgroundNPC;
        public NNModel AIBrainBackgroundNPC { get { return m_AIBrainBackgroundNPC; } }

        public NNModel AI_GetBrain(CXMLAgentBase AgentInstance)
        {
            switch (AgentInstance.AIType)
            {
                case Enum_AIType.Player:
                    return AIBrainPlayer;
                case Enum_AIType.Enemy:
                    return AIBrainEnemy;
                case Enum_AIType.NPC:
                    return AIBrainBackgroundNPC;
                default:
                    Debug.LogError(@"Unknown AI Type in GameManager.AI_GetBrain");
                    return null;
            }
        }

jhlee0918 · February 21, 2022, 11:15am

@mbaske Thank you. I will try.

@ChillX Finally I’m trying something new, I’ll refer to your code later. thanks for letting me know.

jhlee0918 · February 22, 2022, 5:13am

@mbaske Thanks for comment this issue
re-write my code follow your comment. but can work it

Unity.Barracuda.NNModel nnmodel = (Unity.Barracuda.NNModel)AssetDatabase.LoadAssetAtPath("Assets/Models/v1.4.14/basic_agent.onnx", typeof(Unity.Barracuda.NNModel)); //nnmodel = null ... 

Agent.SetModel("basic_agent", nnmodel);

I put it directly into the agent in the Unity editor, there is no problem.
It calls using scripts are not possible.

jhlee0918 · February 22, 2022, 11:27am

After few hours late … I solved.
@mbaske gave me hint, I was setting the asset’s path incorrectly. Thank you.

Topic		Replies	Views
Dynamically loading Onnx file and setting as the model for an Agent Unity Engine ML-Agents , Question , com_unity_ml-agents	3	1592	April 28, 2023
"the external ONNX file data" to NNModel Instance Unity Engine ML-Agents , com_unity_ml-agents	12	3478	August 28, 2023
Can't load generated .onnx file into Behavior Parameters with 3DBall example Unity Engine ML-Agents , Question , com_unity_ml-agents	1	651	May 8, 2021
Can't seem to export model to .ONNX instead of .nn Unity Engine ML-Agents , com_unity_ml-agents	12	3009	October 22, 2020
Barracuda/BehaviorParameters.cs Errors when importing mlagents package Unity Engine ML-Agents , com_unity_ml-agents , Bug	1	256	April 28, 2024

*.onnx file load script at runtime.

Related topics