Movement with Velocity

Unity Engine
, ,
1

In most of the example agents are using similar movement style -discrete rotation and forward with adding force. Even though it is a great way to move, it is not the best solution for my environment. My agent needs to be agile and cancelling forces sometimes result with death. How can I directly use velocities ? Also how can I use them as continuous actions ? (not only 0 and 1)

Here is the code that I’m using below What changes can be made ?

   public void MoveAgent(float[] act)
    {
        var dirToGo = Vector3.zero;
        var rotateDir = Vector3.zero;

        var rotateAxis = (int)act[0];

        var direction = (int)act[1];

        switch (rotateAxis)
        {
            case 0:
                rotateDir = -transform.up;
                break;
            case 1:
                rotateDir = transform.up;
                break;
            case 2:
                rotateDir = Vector3.zero;
                break;
        }
        switch (direction)
        {
            case 0:
                dirToGo = transform.right;
                break;
            case 1:
                dirToGo = -transform.right;
                break;
        }

        rb.AddForce(dirToGo * moveSpeed, ForceMode.VelocityChange);
        transform.Rotate(rotateDir, Time.fixedDeltaTime * turnSpeed);

        if (rb.velocity.sqrMagnitude > 25f) // slow it down
        {
            rb.velocity *= 0.95f;
        }
    }
2

To use switch from discrete to continuous action, you just have to go in Behavior Parameters, vector action, space type, select Continuous. With a space size of 3 you could try something like that:

Rb.velocity = new Vector3(vectorAction[0] * speed, 0f, vectorAction[1] * speed);
transform.Rotate(Vector3.up, vectorAction[2] * rotationSpeed);

if you want the agent to be agile, than i suggest you to increase the movement possibility. My agents has 3 discrete actions with 3 branches each.
0 = [do nothing, move fwd, move backward]
1 = [do nothing, rotate left, rotate right]
2 = [do nothing, strife left, strife right]

by combining forward movement, strife and rotation, the agent can navigate very efficiently, but i’m moving it with math, not with physics, maybe you want a different solution

3

Can you share your knowledge about mathematically calculating these, maybe it could be better option. And how to code strive left,right ?

4

sure man, here you go.

that’s my heuristic function and OnActionReceived for a 3x3 discrete action size

public override void Heuristic(float[] actionsOut)
    {

        if(Input.GetKey("a")){
            actionsOut[0] = 1;
        }else if(Input.GetKey("d")){
            actionsOut[0] = 2;
        }else{
            actionsOut[0] = 0;
        }
if(Input.GetKey("w")){
            actionsOut[1] = 1;
        }else if(Input.GetKey("s")){
            actionsOut[1] = 2;
        }else{
            actionsOut[1] = 0;
        }

        if(Input.GetKey("q")){
            actionsOut[2] = 1;
        }else if(Input.GetKey("e")){
            actionsOut[2] = 2;
        }else{
            actionsOut[2] = 0;
        }
    }

public override void OnActionReceived(float[] vectorAction)
    {

        var rotation = (int)vectorAction[0];
        var fwdAxes = (int)vectorAction[1];
        var lateralAxes = (int)vectorAction[2];


            // vectorAction[0] 0 = no rotation, 1 = rot sx, 2 = rot rx
            switch(rotation)
            {
                case 1:
                    rb.transform.Rotate(0, -rotSpeed*Time.deltaTime, 0);
                    break;
                case 2:
                    rb.transform.Rotate(0, rotSpeed*Time.deltaTime, 0);
                    break;
            }

            // vectorAction[1] 0= no movement, 1= mov fwd, 2= mov bkw,
            switch(fwdAxes)
            {
                case 1:
                    rb.transform.Translate(0, 0, speed*Time.deltaTime, Space.Self);
                    break;
                case 2:
                    rb.transform.Translate(0, 0, -speed*Time.deltaTime, Space.Self);
                    break;
            }

            // vectorAction[2] 0= no movement, 1= strife left, 2= strife right
            switch(lateralAxes)
            {
                case 1:
                    rb.transform.Translate(-speed*Time.deltaTime, 0, 0, Space.Self);
                    break;
                case 2:
                    rb.transform.Translate(speed*Time.deltaTime, 0, 0, Space.Self);
                    break;
            }
    }
1 Like
5

Thanks for the info, I will try and come back

6

This works like charm but one question: Shouldn’t we also need to define case 0: for doing nothing ?

7

Glad it helped :slight_smile:
as far as i understand, if the case is not listed, it just doesn’t perform any of the listed action.
But i’ve noticed that if you don’t put case 0 in the heuristic function, you cannot properly control the agent (try for yourself, it’s a mess).
As usual, i might be wrong, can anyone deny/confirm we don’t need to define “do nothing” in the OnAcionReceived() method ?

8

Yeah that would be great if someone can confirm this, otherwise the actions will increase +1 for every branch and its definetly slows the process.Thanks for the movement code again.

9

well, i’m starting to think that since we mostly use “take actions between decisions”, if we do not define a case 0, the “last action taken” might not be changed to “do nothing”.
The agent goes left, keeps going left till the next decision, next decision is 0, case is not defined, agent keeps going left till next decision and so on.

actually it would be pretty easy to fix.
just add

case 0:
rb.transform.Rotate(0,0,0);

also, by just looking at the training process, if they now willingly stop, their movements should look a lot more jittery at the start.

10

Alright I was thinking the same. I tried the movement code and at some point they learned to go inside the outer walls of the environment and escape the area. Since we are not using physics that is a bit of a problem. Do you have a solution for this ? I increased the wall thickness a little bit but it didn’t help. Maybe some combination of using velocity can solve this. If I solve I will paste the code.

6164091--674232--upload_2020-8-3_19-32-48.png

6164091--674229--upload_2020-8-3_19-32-33.png

11

wow that’s weird, i haven’t had any escape problem. no idea what’s causing it.

btw, you asked for agility… :smile:

1 Like
12

i forgot to mention, if you are interest in physic based movements, search for “the nature of code” by Daniel Shiffman.
it goes from explaining the basics of using velocity, acceleration etc. to coding behaviors like seek, intercept, flocking, and so on. Really interesting stuff

13

Great source, I skimmed it, I think it is essential for these kind of simulation

1 Like