missile guidance

the missile is homing good, however my model shakes like crazy on the way to its target, it just freaks out and starts flapping like crazy on the way to the target

 public float speed = 5f;
 private Rigidbody rb;    
 public Transform target;    
 public float rotateSpeed = 200f;
 // Start is called before the first frame update
 void Start()
     rb = GetComponent<Rigidbody>();

 // Update is called once per frame
 void Update()
     Vector3 direction = (Vector3)target.position - rb.position;


     Vector3.Cross(direction, transform.up);

     Vector3 rotateAmount = Vector3.Cross(direction, transform.up);

     rb.angularVelocity = -rotateAmount * rotateSpeed;

     rb.velocity = transform.up * speed;

It is probably better/ more expedient to figure out the reason but maybe a workaround is filtering it out. Maybe (and I know some missile expert in the forum is going to be screaming at their monitor) using a kalman filter to smooth it out?

Either that or by taking snapshots and then lerping between those values would average it out.

Example of Kalman