If more then 2 players join the Photon Server it gets glitchy?

So I’m testing the Photon Network and I have 2 clients open but when I open a third client, players are seen floating above the ground on the third client.
Why is that?

Because there is some syncing issue.
Try to reproduce in the provided samples. If these work (typically they do), check your code and fix things.

It was an issue with my smoothing script.

Is there a way to smooth player rotation and position without interrupting the sync?

Here is my code:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class NetworkCharacter : Photon.MonoBehaviour
{

    Vector3 realPosition = Vector3.zero;
    Quaternion realRotation = Quaternion.identity;
    // Start is called before the first frame update
    void Start()
    {
       
    }

    // Update is called once per frame
    void Update()
    {
        if(photonView.isMine)
        {

        } else
        {
            transform.position = Vector3.Lerp(transform.position, realPosition, 0.1f);
            transform.rotation = Quaternion.Lerp(transform.rotation, realRotation, 0.1f);
        }
    }

    public void OnPhotonSerializeView(PhotonStream stream, PhotonMessageInfo info)
    {
        if(stream.isWriting)
        {
            stream.SendNext(transform.position);
            stream.SendNext(transform.rotation);
        } else
        {
            realPosition = (Vector3)stream.ReceiveNext();
            realRotation = (Quaternion)stream.ReceiveNext();
        }
    }


}