How to limit rigidbody rotation to one axis at a time?

I am working on a plat-former where a cube moves by adding force to a rigidbody and it rolls. However, after a bit of constant rotation along the the Z axis(moving along the X axis) it will start to roll in the X direction as well and fall off course. How would I make it rotate only one direction at a time? Below is the code for the movement.

using UnityEngine;
using System.Collections;

public class PlayerController : MonoBehaviour 
{
	public float speed = 20;
	public float jump = 20;
	
	void Start () 
	{
		
	}
	
	void Update () 
	{
		if(Input.GetKey(KeyCode.RightArrow))
		{
			rigidbody.AddForce(Vector3.right * speed);
		}
		if(Input.GetKey(KeyCode.LeftArrow))
		{
			rigidbody.AddForce(Vector3.left * speed);
		}
		if(Input.GetKey(KeyCode.UpArrow))
		{
			rigidbody.AddForce(Vector3.forward * speed);
		}
		if(Input.GetKey(KeyCode.DownArrow))
		{
			rigidbody.AddForce(Vector3.back * speed);
		}
	}
}

You could add and remove joints to the object in runtime?