How to Manipulate the Build Up and Dispersal of Velocity

How would I make my character controller build up velocity faster and diminish that velocity faster? When you apply force by moving, it gradually builds until it reaches its max speed, and I want to make it to where you reach that speed faster. Also, when you release the movement key, it takes a while for all force to be removed. I am not using the Character Motor, script, by the way, I am using a custom movement script, but I am using the Character Controller Script. To move, I am using a method like this:

    if (controller.isGrounded) {
        moveDirection = Vector3(Input.GetAxis("Horizontal"), 0, Input.GetAxis("Vertical"));
        moveDirection = transform.TransformDirection(moveDirection);
        moveDirection *= speed;

Is anyone able to help with this? I need help for this project.

http://wiki.unity3d.com/index.php?title=RigidbodyFPSWalker