Cant change client position via ServerRpc

Hey there, I got a lil problem.

So in my attempt to learn the Netcode API for Unity, I am running into this weird ServerRpc issue.

In the my following code, I have a hunter (catcher) and an unlimited amount of catchable Players.

The OwnerClientId of the catcher is saved as a NetworkVariable.

private void HandlePowerUp(){
    Vector3 moveDir = transform.forward;
    float moveDistance = speed * Time.deltaTime;
    m_HitDetect = Physics.BoxCast(transform.position, transform.localScale*0.5f, moveDir, out m_Hit, Quaternion.identity, moveDistance);
    if (m_HitDetect)
    {
        if (m_Hit.collider.gameObject.tag == "Power")
        {
            if(OwnerClientId != Game.Value.catcher){
                KillServerRpc(m_Hit.collider.gameObject);
                Debug.Log("I "+ OwnerClientId + " HAVE THE POWEEEEEEEEEEEEEEEEEEEEER!!!!!");
                Game.Value = new MyCustomData{
                    catcher = OwnerClientId,
                };
            }
        }

        if (m_Hit.collider.gameObject.tag == "Player")
        {
            TestServerRpc(OwnerClientId,Game.Value.catcher);
            if(OwnerClientId == Game.Value.catcher){
                if(IsHost) {
                    //This was only for trying to teleport directly, not using a ServerRpc
                    DieServerRpc(m_Hit.collider.gameObject, OwnerClientId);
                }
                else{
                    DieServerRpc(m_Hit.collider.gameObject, OwnerClientId);
                }

                Game.Value = new MyCustomData{
                        catcher = 10,
                    };
            }      
        }
    }
}

[ServerRpc]
public void DieServerRpc(NetworkObjectReference target, ulong talker){
    Debug.Log(talker + " suggests we Hitnkill!");
    NetworkObject targetObject = target;
    Vector3 startPos = new Vector3(0,0,0);
    targetObject.transform.position = startPos;
}

(Only involved the relevant parts of ma code)

So here, you (lets say you are the chaser) hit a player via a raycast, if your Id matches the one of the saved Chaser, you start the DieServerRpc (here the player you hit), that simply gets the targets gameobject and teleports it to the coordinates 0 0 0.

However,so far the only outcome was that Clients teleported the Host. The host, if he called the ServerRpc or directly tried to teleport the clients, never managed to achieve anything like that.

I have checked if targeting works, it always does, so im a lil lost.

Idk if this is like an administration issue,
So i could really use and would apreciate some help on this.

Already, Thanks to everyone who will answer to this.

(Also merry christmas everyone! hehe)