Does an agent understand location of tagged objects when using Ray Percept Sensor 3D

I have an environment that has two types of walls. Walls tagged “GoodWall” that give positive rewards when hit and walls tagged “BadWall” that give negative rewards, my question is when using ray sensor 3D if im correct the rays give the agent the tag on an object, but is there something else I need to pass in observations for the agent to learn to move towards goodwalls and avoid badwalls? Cause right now even when the rays hit the objects tagged good walls multiple times it still does not move towards it

https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/api/Unity.MLAgents.Sensors.RayPerceptionOutput.RayOutput.html

it can determine tag index and hit distance but not direction