Gravity is not 0 when player is grounded

When the player is grounded, the downwards force is around -20. this is problematic because when the player starts falling, the fall speed will start from the -20 and not 0 (or preferably something like -2). (Im doing a 3d project.)

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class PlayerMovement : MonoBehaviour
{
    //MOVEMENT
    public float jumpSpeed = 10f;
    public float speed = 6f;
    public float gravity = 20f;
    public CharacterController characterController;

    private Vector3 moveDirection = Vector3.zero;
    private Vector3 moveGravity = Vector3.zero;

    void Start()
    {

    }


    void Update()
    {
        //MOVEMENT START    MOVEMENT START    MOVEMENT START    MOVEMENT START 
        
        float x = Input.GetAxisRaw("Horizontal");
        float z = Input.GetAxisRaw("Vertical");

        moveDirection = transform.right * x + transform.forward * z;
        moveDirection = Vector3.ClampMagnitude(moveDirection, 1f);
        moveDirection *= speed;
        
        if (Input.GetKeyDown(KeyCode.Space) && characterController.isGrounded)
        {
            moveGravity.y = jumpSpeed;
        }

        if (!characterController.isGrounded)
        {
            moveGravity.y -= gravity * Time.deltaTime;
        }

        characterController.Move((moveGravity+moveDirection) * Time.deltaTime);


        Debug.Log(moveGravity);

        //MOVEMENT END    MOVEMENT END    MOVEMENT END    MOVEMENT END

The gravity is not when your character is grounded because nowhere in your code do you set it to , alos gravity does not work like that, it is a constant force, but yo ucan do what you can fairly easily if you tell it what to do.

     if (characterController.isGrounded)
             gravity=0;
     else
             gravity=20;