I have been manually normalizing all my vector observations and wasn’t using the normalization hyperparamter. I tried flipping on normalization in my config file and noticed a big impact on training so I’m wondering if it’s normalizing my ray sensors which weren’t before but I would’ve assumed these would be normalized already.
I believe when normalize is set to true all network inputs are normalized on the python side regardless of if they are already in the -1,1 space. This will result in worse performance than when you know the bounds of the inputs and can normalize it yourself.
Yes but I am talking about the RaySensor 2d Component that comes packaged with unity ML Agents.
I assume that they are in fact normalized but I am just puzzled how I am getting better results from using the python normalization versus doing my own manual normalizing.
At a cursory glance I’m not seeing any normalization of the observations at the sensor level - ml-agents/com.unity.ml-agents/Runtime/Sensors/RayPerceptionSensor.cs at develop · Unity-Technologies/ml-agents · GitHub
Could still be happening but I haven’t been able to find it.
I’m curious why they would not normalize these using the ray distance. I think I must be getting a large benefit from that.
https://forum.unity.com/threads/how-to-get-value-of-distance-between-agent-obstacles-with-using-rayperceptionsensor3d.1043338/ @Luke-Houlihan in this thread it appears that the sensor output is normalized. I only have a few simple 2d vectors to normalize, I have tested that they are not going past 1, and yet I am getting significantly better results from using the normalization hyperparamter than manually normalizing my vectors.
most of my bugs came from wrong normalization. If you want to use it without auto-norm than debug everything very carefully.
regarding your question: I am pretty sure they normalize it.
in my experience normalized by hand vs auto norm gives no real performance difference. The only thing that is better with manually is to achieve better generalization (if the agent doesnt go in Training to the end of the map, it will break while gaming when used there)
@GamerLordMat I think you are correct. I think I am getting a boost on the first half of the level from the auto-normalization because the bounds of the vectors are more accurate for that portion of the level and I am then getting an observation space blowup on the second half of the map when it finally reaches it. The problem is I am struggling to even get to the second half of the map where my manual normalization would help out when I am using manual normalization.