Rigidody.velocity = Direction * speed; How to get direction?

So I have a navigation script, I can’t use the nav mesh and nav mesh agent due to technical reasons. But that’s ok. Anyway I have a list of vector3s as waypoints that my sphere needs to pass through. I want it to roll so I need to use the rigidbody and physics engine. Anyway my code works, mostly, the only problem is that my sphere accelerates and decelerates when moving from waypoint to waypoint. I don’t want this I want the speed to remain constant. My scene just has the ball and a plane.Here’s my code I’ve attached to my sphere:

using System.Collections.Generic;

public class Nav : MonoBehaviour {

    public bool isNavigating;
    public List<Vector3> points;

    Rigidbody playerRB;
    int currentPoint, lastPoint;
    float speed = 5f;

    bool isMoving = false;

	// Use this for initialization
	void Start ()
        // set in instaniate script
        points = new List<Vector3>()
            new Vector3(5, 0, 1),
            new Vector3(5, 0, 5),
            new Vector3(8, 0, 5),
            new Vector3(8, 0, 8),
            new Vector3(14, 0, 8),
            new Vector3(14, 0, 14),
            new Vector3(20, 0, 14)

        isNavigating = true;

        // leave here
        currentPoint = 0;
        lastPoint = points.Count;
        playerRB = GetComponent<Rigidbody>();
	void FixedUpdate ()
        if (!isNavigating) return;

        if (new Vector3(Mathf.Round(transform.position.x), Mathf.Round(transform.position.y), Mathf.Round(transform.position.z))  == points[currentPoint])
            if (!(currentPoint + 1 > points.Count - 1))
                isNavigating = false;
                playerRB.velocity = Vector3.zero;

        // Navigate
        Vector3 direction = (points[currentPoint] - transform.position).normalized;
        playerRB.velocity = direction * speed;


If i got it right - you can divide Vector3 that stands for direction with it’s highest value so:

velocity = (direction /Mathf.Max(Mathf.Max(direction[0], direction[1]),direction[2]) )*speed;

it will give you speed of 1 in one direction and some lower values in other determined by ratio AKA constant speed