Movement jittery between client and server values

I’m trying to debug movement issues with my Netcode for Entities project.
Hopefully someone here can help me :slight_smile:

Here is a video showing the issue:

My player rapidly flickers between the Client (predicted) and Server positions.

The Server position is always behind the Client (predicted) position, seems normal.
But (I think) my cube is also flickering between rendering at the server position and the client position.

The project is expanded from the Networked Cube example.

Original post

This is the inspector on the player prefab:

I am trying to move the player with PhysicsVelocity instead of directly modifying the LocalTransform.

This is the system that applies movement to the cube.

[BurstCompile]
[UpdateInGroup(typeof(PredictedFixedStepSimulationSystemGroup))]
public partial struct CubeMovementSystem : ISystem {
    ComponentLookup<Ability> ability_lookup;

    [BurstCompile]
    public void OnCreate(ref SystemState state) {
        ability_lookup = SystemAPI.GetComponentLookup<Ability>(true);
    }

    [BurstCompile]
    public void OnUpdate(ref SystemState state) {
        foreach (
            var (
                transform,
                phys_velocity,
                input,
                movement,
                abilities
            ) in SystemAPI.Query<
                RefRW<LocalTransform>,
                RefRW<PhysicsVelocity>,
                RefRW<NetPlayerInput>,
                RefRW<Movement>,
                DynamicBuffer<Abilities>
            >().WithAll<GhostOwnerIsLocal>()
        ) {
            ability_lookup.Update(ref state);  // Update the state of ComponentLookups

            float speed = movement.ValueRO.speed;
            float speed_max = movement.ValueRO.speed_max;

            // Code related to dash ability removed for the sake of readability
            // All logic is conditional on input and bug happens without using dash

            // Enforce physics constraints
            ///////////////////////////////////////////////////////////////////
            phys_velocity.ValueRW.Linear.y = 0;
            transform.ValueRW.Rotation = quaternion.Euler(
                new float3(0, transform.ValueRO.Rotation.value.y, 0)
            );
            if (math.length(phys_velocity.ValueRO.Linear) <= math.EPSILON) {
                phys_velocity.ValueRW.Linear = float3.zero;
            }


            // Player input movement
            ///////////////////////////////////////////////////////////////////
            bool has_player_movement = input.ValueRO.move.Equals(float2.zero);
            if (!!!has_player_movement) {
                movement.ValueRW.Velocity.Linear.x += input.ValueRO.move.x * (speed * SystemAPI.Time.DeltaTime);
                movement.ValueRW.Velocity.Linear.z += input.ValueRO.move.y * (speed * SystemAPI.Time.DeltaTime);
            }


            // Enforce movement limits
            ///////////////////////////////////////////////////////////////////
            if (math.length(movement.ValueRW.Velocity.Linear) > speed_max) {
                movement.ValueRW.Velocity.Linear = math.normalizesafe(movement.ValueRO.Velocity.Linear) * speed_max;
            }


            // Apply movement to physics
            ///////////////////////////////////////////////////////////////////
            if (math.length(movement.ValueRW.Velocity.Linear) > math.EPSILON) {
                phys_velocity.ValueRW.Linear.x += movement.ValueRO.Velocity.Linear.x;
                phys_velocity.ValueRW.Linear.z += movement.ValueRO.Velocity.Linear.z;
            }
        }
    }
}

This is the component that holds the player input:

[GhostComponent(PrefabType=GhostPrefabType.AllPredicted)]
public struct NetPlayerInput : IInputComponentData {
    public float2 move;
    public InputEvent dash;
}

Component holding player-driven movement:

[GhostComponent(PrefabType=GhostPrefabType.AllPredicted)]
public struct Movement : IComponentData {
    public PhysicsVelocity Velocity;
    public float speed;
    public float speed_base;  // Allow modifying speed
    public float speed_max;
    public float speed_max_base;  // Allow modifying max speed
}

Camera follow system:

public partial class CameraSystem : SystemBase {
    protected override void OnUpdate() {
        float3 world_position = float3.zero;

        foreach (var (
            _,
            player,
            local_transform,
            local_to_world,
            entity
        ) in SystemAPI.Query<
            GhostOwnerIsLocal,
            RefRO<Player>,
            RefRO<LocalTransform>,
            RefRO<LocalToWorld>
        >().WithEntityAccess()) {
            // Debug.Log($"Local Position: {local_transform.ValueRO.Position}");
            world_position = local_transform.ValueRO.Position;
        }

        foreach (var (
            camera,
            camera_settings,
            local_to_world
        ) in SystemAPI.Query<
            MainCamera,
            RefRW<MainCameraSettings>,
            RefRW<LocalToWorld>
        >()) {
            camera.look_at = world_position;
            camera.Transform.LookAt(world_position);
            camera.Transform.position = SystemAPI.GetComponent<LocalToWorld>(camera_settings.ValueRO.Position).Position;
            // Debug.Log($"World Position: {world_position}");
        }
    }
}

Camera components:

public class MainCamera : IComponentData {
    public Transform Transform;
    public float3 look_at;
}

public struct MainCameraSettings : IComponentData {
    public Entity Pivot;
    public Entity Position;
}

The MainCameraSettings hold references to Entities that are initialized by a proxy component just like in the ECSSamples PhysicsSamples. They are Child GameObjects of the Player prefab with only a transform – then baked.

Package versions:
Entities: 1.0.16
Netcode for Entities: 1.0.17
Unity Physics: 1.0.16
Mathematics: 1.2.6

EDIT: In this case, the results were caused by the camera follow and was completely unrelated to the movement. See posts below.

Without the full picture, here’s some guesses and some steps to debug

First thing I’d look into is making sure your system runs before the physics simulation (check out NetcodeSamples’ physics sample)
[UpdateBefore(typeof(PhysicsInitializeGroup))]

From your video, playing frame by frame looks like there’s a discrepancy between your camera, your cube and the client debug mesh. Your camera and cube seem super smooth which is suspicious. What’s your setup for camera following vs your cube? Normally the debug mesh is supposed to draw every frame your ghost’s LocalToWorld. Are there extra entities used for rendering?
Quick sanity check: what does your movement component look like? Are its fields replicated with GhostField? (Any reason why you’re not just using phys velocity’s values there? why the extra velocity in your movement component?)
This shouldn’t cause your issue, but can you add a check for the Simulate tag please, that’ll make sure you simulate only when needed.
What package version are you using?

In general, to debug this, I’d add Debug.Logs and Debug.DrawLines to see if 1. your OnUpdate is executed the expected amount of times when resimulating and 2. if there’s anything blocking/preventing your entity’s movement. Debug.Drawlines with a lifetime of one frame should show you frame per frame all the resimulation steps that happened in that frame. Then screen record your scene view and step frame by frame.

Thanks for taking the time Samuel!

  • Added camera follow system

  • Added movement component

  • Not sure if/not replicated GhostFields

  • Added package versions

I have a separate player movement so that I can add external forces and limit player-driven movement forces.
I assume I’ll need to know both the physics velocity and the player-movement velocity to do this.

At the moment the code system just clamps all movement beyond the max_speed but that’s just because I have been stripping things away and simplifying to try and figure out what’s causing this issue.

I’ll add more Debugs to investigate and update here with results.

Upon further investigation, the issue has to be with the camera following, not the movement.

When disabling the camera-follow, the issue is gone.

The best results so far come from running the CameraSystem in PredictedFixedStep, but after CubeMovementSystem with these attributes:

[UpdateInGroup(typeof(PredictedFixedStepSimulationSystemGroup))]
[UpdateAfter(typeof(CubeMovementSystem))]

Now, when following, it’s mostly in-sync, but it jitters around 1-2 times per second.
I’ve tried recording a video but without the debug bounding boxes the jitters are so fast it’s hard to see in the video.

However, It’s not subtle and is very disruptive when actually moving around in play-mode.

What would be causing this issue?

Here’s the current CameraSystem:

[UpdateInGroup(typeof(PredictedFixedStepSimulationSystemGroup))]
[UpdateAfter(typeof(CubeMovementSystem))]
public partial class CameraSystem : SystemBase {
    protected override void OnUpdate() {
        // return; // DEBUG

        float3 player_world_position = float3.zero;

        foreach (var (
            _,
            player,
            local_transform,
            local_to_world,
            entity
        ) in SystemAPI.Query<
            GhostOwnerIsLocal,
            RefRO<Player>,
            RefRO<LocalTransform>,
            RefRO<LocalToWorld>
        >().WithEntityAccess()) {
            player_world_position = local_transform.ValueRO.Position;  // World position
        }

        foreach (var (
            camera,
            camera_settings,
            local_to_world
        ) in SystemAPI.Query<
            MainCamera,
            RefRW<MainCameraSettings>,
            RefRW<LocalToWorld>
        >()) {
            float3 player_camera_position = SystemAPI.GetComponent<LocalToWorld>(camera_settings.ValueRO.Position).Position;
            float3 current_position = camera.Transform.position;
            Debug.Log($"Camera Cur Pos: {current_position}");
            Debug.Log($"Camera New Pos: {player_camera_position}");

            camera.Transform.position = player_camera_position;
            camera.look_at = player_world_position;
            camera.Transform.LookAt(player_world_position);
        }
    }
}

Your Movement component has the GhostComponent attribute, but has no GhostFields, so it’s actually not replicated (if you look at your source generated folder, Movement should not be there). Make sure to mark your fields as [GhostField]. (It’s not super clear in our docs and I got caught by this too originally, I’ll add a task to fix this on our side).
So your Movement component won’t get rolled back when resimulating and won’t get synced with the server correctly. That could explain some of the discrepancy.
And again make sure your logic runs before Physics’ simulation group.