Unity Camera Issue: Scroll up and down speed not consistent

I am making a 2d side scrolling game, the character can look up and down with w/up arrow and s/down arrow. The camera will scroll down/up a max of 40f. here is my code for the camera.

public class CameraFollow : MonoBehaviour
{
    // Start is called before the first frame update
    private float yOffset = 40f;
    private bool isUp;
    private bool isDown;
    private float counter = 0f;
    void Start()
    {
        
    }

    // Update is called once per frame
    public Transform player;
    void Update()
    {
        //if player is pressing down/up, pan down/up untill x distance 
        isUp = Input.GetKey(KeyCode.UpArrow) || Input.GetKey(KeyCode.W);
        isDown = Input.GetKey(KeyCode.DownArrow) || Input.GetKey(KeyCode.S);
        if (isUp) 
        {
            if (counter != 40f)
            {
                counter = counter + 0.5f;
            }
        }
        else if (isDown) 
        {
            if (counter != -40f) 
            {
                counter = counter - 0.5f;
            }
        }
        else
        {
            if (counter != 0f && counter > 0f)
            {
                counter = counter - 0.5f;
            }
            else if (counter != 0f && counter < 0f)
            {
                counter = counter + 0.5f;
            }
        }
        transform.position = new Vector3(player.position.x, player.position.y + yOffset + counter, -10);
    }
}

The problem is with the ±0.5f sometimes when testing the game that speed is fine for the pan up and down but sometimes when I test it is unbearably slow. I will change it to 2f and it seems fine again but then after a while it will become way too fast. I am looking for a solution to where the speed is consistent even if it requires me to rewrite CameraFollow completely.

This is probably because of variable frame rates. In unity, the amount of time between each call of Update can vary frame by frame, so when this is a short time, your camera will move quickly, and when it is a longer delay your camera will move more slowly.

To avoid this, whenever you do anything involving a speed in Update, you need to multiply it by Time.deltaTime (which is the length of time since the last update).

So in your case on every line where you change the value of the counter, you need something like this:

counter = counter + 0.5f * Time.deltaTime;

You will definitely need to tweak the 0.5f though!