Why is the secondary showing on play?

So I made a weaponChange script, and when I play it automatically switches to the secondary, without being able to switch back. Could someone tell me why that is?

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class weaponChange : MonoBehaviour {

	public GameObject primary; 
	public GameObject secondary;
	private bool isPrimary; 


	void Start () {
		primary.SetActive (true); 
		secondary.SetActive (false); 
	}
	
	void Update () {
		if (Input.GetButtonDown("Mouse ScrollWheel")) {
			isPrimary = !isPrimary; 

		}

		if (isPrimary == true) {
				secondary.SetActive(false); 
				primary.SetActive(true); 
		}

		if (isPrimary == false) {
				primary.SetActive(false); 
				secondary.SetActive(true); 
		}
	}
}

A couple things. When you have a movement axis, you can’t read it like it’s a button…or at least I wasn’t able to do so. You have to measure whether or not there was any movement. I put a little tolerance in the code just in case. Also, isPrimary starts out as false.

using UnityEngine;

public class weaponChange : MonoBehaviour
{
  public GameObject primary;
  public GameObject secondary;
  private bool isPrimary = true;

  void Start()
  {
    SetWeaponVisibility();
  }

  void SetWeaponVisibility()
  {
    primary.SetActive(isPrimary);
    secondary.SetActive(!isPrimary);
  }

  void Update()
  {
    if (Mathf.Abs(Input.GetAxis("Mouse ScrollWheel")) > 0.1)
      isPrimary = !isPrimary;

    SetWeaponVisibility();
 }
}