Access to neural network weights

unity_nBMnid_GOQvclw · March 21, 2020, 10:18am

I want to train agents in the FoodCollector-like environment with a genetic algorithm.
I want each agent to inherit the neural network weight parameter from the parent when they are born, so I think I need to access to the network and assign the weight.
How can I access to weight parameters of ML-Agents Agent?

Thanks!

celion_unity · March 23, 2020, 4:54pm

Hi,
There are some methods to support this in the TFPolicy class:

github.com

Unity-Technologies/ml-agents/blob/0.15.0/ml-agents/mlagents/trainers/policy/tf_policy.py#L131-L151


      
          def get_weights(self):
              with self.graph.as_default():
                  _vars = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
                  values = [v.eval(session=self.sess) for v in _vars]
                  return values
          
          def init_load_weights(self):
              with self.graph.as_default():
                  _vars = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
                  values = [v.eval(session=self.sess) for v in _vars]
                  for var, value in zip(_vars, values):
                      assign_ph = tf.placeholder(var.dtype, shape=value.shape)
                      self.assign_phs.append(assign_ph)
                      self.assign_ops.append(tf.assign(var, assign_ph))
          
          def load_weights(self, values):
              with self.graph.as_default():
                  feed_dict = {}
                  for assign_ph, value in zip(self.assign_phs, values):
                      feed_dict[assign_ph] = value
                  self.sess.run(self.assign_ops, feed_dict=feed_dict)

unity_nBMnid_GOQvclw · March 24, 2020, 4:31pm

Thank you very much!
Are there such methods on the Unity side (C#) ?

celion_unity · March 24, 2020, 6:09pm

I’m not exactly sure what’s available on the C# side. We import the models with Barracuda, which has access for iterating through the layers:
https://docs.unity3d.com/Packages/com.unity.barracuda@0.6/manual/index.html#introspecting-barracuda-models
and you should be able to access the weights from the layers:
https://docs.unity3d.com/Packages/com.unity.barracuda@0.6/api/Barracuda.Layer.html
but I believe these are read-only.

If you have feature requests for Barracuda, they best place to ask is on their github issues: Issues · Unity-Technologies/barracuda-release · GitHub

unity_nBMnid_GOQvclw · March 25, 2020, 4:09pm

Thank you very much for your kind reply!
Now I’m trying to implement NN using KelpNet (GitHub - harujoh/KelpNet: Pure C# machine learning framework) and train it by GA on C#,
but I’ll try your methods if I have a chance.

Looking at the code a bit, the weight variable is at
Agent.m_Brain : BarracudaPolicy
.m_ModelRunner : ModelRunner (protected)
.barracudaModel : Barracuda.Model (not a member variable)
.layers : List<Barracuda.Layer>
→ Layer.weights : float[ ]