How to use RayCast in Physics together with Netcode?

I tried two options, one from the Laser Dot s 101 example, the other by simply going through all the suitable colliders, none registering an obvious collision. There is a client world on the stage, there is no server world (this is something like a lobby scene before connecting) Does anyone know how this can be achieved?

ClientDispBoxCollisionSystem.cs
using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Physics;
using Unity.Physics.Systems;
using Unity.Rendering;
using Unity.Transforms;
using UnityEngine;
using Collider = Unity.Physics.Collider;

[BurstCompile]
[UpdateAfter(typeof(FixedStepSimulationSystemGroup))]
[WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation)]
public partial struct ClientDispBoxCollisionSystem : ISystem
{
	[BurstCompile]
	public void OnCreate(ref SystemState state)
	{
		state.RequireForUpdate<CamTran>();
		state.RequireForUpdate<DispBox>();
		state.RequireForUpdate<PhysicsWorldSingleton>();
	}

	[BurstCompile]
	public void OnUpdate(ref SystemState state)
	{
		var ecb = new EntityCommandBuffer(Allocator.Temp);
		
		// Получаем данные о трансформации камеры
		var camTran = SystemAPI.GetSingleton<CamTran>();

		// Параметры луча: начало — позиция камеры, конец — 10 единиц вперед
		float rayLength = 5f;
		float3 rayStart = camTran.tran.Position;
		float3 rayEnd = camTran.tran.Position + camTran.tran.Forward() * rayLength;

		Utils.DrawLine(ref ecb, rayStart, rayEnd, Color.green);

		var raycastInput = new RaycastInput
		{
			// specify starting and end points of the raycast (which together imply direction)
			Start = rayStart,
			End = rayEnd,
			// don't forget to set a filter or else you'll get no hits!
			Filter = CollisionFilter.Default
		};

		// Перебираем все сущности с DispBox и PhysicsCollider
		foreach (var (dispBox, collider, entity) in SystemAPI.Query<RefRO<DispBox>, RefRO<PhysicsCollider>>().WithEntityAccess())
		{
			unsafe
			{
				Collider* colliderPtr = collider.ValueRO.ColliderPtr;

				colliderPtr->SetCollisionResponse(CollisionResponsePolicy.CollideRaiseCollisionEvents);
				if (colliderPtr->CastRay(raycastInput, out var hit))
				{
					Debug.Log("Collide");
					// Если луч попал в коллайдер, меняем цвет
					SystemAPI.SetComponent(entity, new URPMaterialPropertyBaseColor
					{
						Value = new float4(1f, 0f, 0f, 1f) // Красный
					});
				}
				else
				{
					Debug.Log("NoCollideUnsafe");
				}
			}
		}

		// to perform raycasts or other collision queries, we need the collision world
		var physicsWorld = SystemAPI.GetSingleton<PhysicsWorldSingleton>();
		if (physicsWorld.CollisionWorld.CastRay(raycastInput, out var closestHit))
		{
			//Debug.Log(closestHit.Entity.Index);
			// Получаем текущий компонент DispBox
			if (SystemAPI.HasComponent<DispBox>(closestHit.Entity))
			{
				// Устанавливаем красный цвет
				SystemAPI.SetComponent(closestHit.Entity, new URPMaterialPropertyBaseColor
				{
					Value = new float4(1, 0, 0, 1),
				});
			}
		}
		else
		{
			Debug.Log("NoCollideWorld");
		}

		ecb.Playback(state.EntityManager);
		ecb.Dispose();
	}
}

Physics run inside the prediction loop on the client.

Therefore, by default the ClientWorld does not allow running physics if there aren’t predicted ghost.
Furthermore, it is required that at least one entity with physics-velocity component exist, or lag-compensation is enabled.

You can change the default behaviour by:

  • Using the ClientTickRate.PredictionLoopUpdateMode and configure the prediction loop to always run (independently by the presence of predicted ghosts or not).
  • Setting up the NetCodePhysicsConfig.PhysicGroupRunMode and set that to AlwaysRun

You can do the first by using NetCodeConfig or by setting up manually the ClientTickRate.
For the second, you need to add the authoring to a sub-scene and configure it.

You can find more info by checking the XML Docs (or API docs) for

  • PredictionLoopUpdateMode
  • PhysicGroupRunMode

Alternatively, you can use also Multi-Physics world for this by setting up a CustomPhysicWorld group, that will run inside the FixedStepSystemGroup and target.
By setting your RigidBody authoring to use the proper PhysicsWorldIndex you can raycast in this world instead of the default one (that is usually used for prediction).

You can find docs for it un Unity.Physics, or Netcode and in our Netcode Samples

Well, I tried to make a separate physical world that would check the RayCast only with the DispBox.

ClientDispBoxSelectSystem.cs
using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Physics;
using Unity.Rendering;
using Unity.Transforms;
using UnityEngine;

[BurstCompile]
[UpdateAfter(typeof(FixedStepSimulationSystemGroup))]
[WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation)]
public partial struct ClientDispBoxSelectSystem : ISystem
{
	[BurstCompile]
	public readonly void OnCreate(ref SystemState state)
	{
		state.RequireForUpdate<CamTran>();

		state.RequireForUpdate<DispBox>();
	}

	[BurstCompile]
	public void OnUpdate(ref SystemState state)
	{
		var ecb = new EntityCommandBuffer(Allocator.Temp);


		// Получаем данные о трансформации камеры
		var camTran = SystemAPI.GetSingleton<CamTran>();
		float3 rayStart = camTran.tran.Position;
		float3 rayEnd = camTran.tran.Position + camTran.tran.Forward() * 5f;

		// Параметры луча
		var raycastInput = new RaycastInput
		{
			Start = rayStart,
			End = rayEnd,
			Filter = CollisionFilter.Default,
		};

		Utils.DrawLine(ref ecb, raycastInput.Start, raycastInput.End, Color.green);

		// Проверяем наличие компонента и получаем PhysicsWorld
		if (SystemAPI.HasSingleton<CustomPhysicsWorldSingleton>())
		{
			var customPhysicsWorld = SystemAPI.GetSingleton<CustomPhysicsWorldSingleton>().CustomPhysicsWorld;

			foreach (var body in customPhysicsWorld.CollisionWorld.StaticBodies)
			{
				Utils.DrawLine(ref ecb, camTran.tran.Position, body.WorldFromBody.pos, Color.cyan);
			}

			if (customPhysicsWorld.CollisionWorld.CastRay(raycastInput, out var hit))
			{
				Debug.LogWarning("Collide!!!!");

				if (SystemAPI.HasComponent<DispBox>(hit.Entity))
				{
					SystemAPI.SetComponent(hit.Entity, new URPMaterialPropertyBaseColor
					{
						Value = new float4(1f, 0f, 0f, 1f)
					});
				}
			}
			else
			{
				Debug.Log("No Collide");
			}
		}

		ecb.Playback(state.EntityManager);
		ecb.Dispose();
	}
}
ClientDispBoxRayCastSystem.cs
using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Physics;
using Unity.Transforms;
using UnityEngine;

[BurstCompile]
[UpdateInGroup(typeof(FixedStepSimulationSystemGroup))]
[WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation)]
public partial struct ClientDispBoxRayCastSystem : ISystem
{
	private PhysicsWorld customPhysicsWorld;
	private EntityQuery dispBoxes;
	private Simulation simulation;

	[BurstCompile]
	public void OnCreate(ref SystemState state)
	{
		customPhysicsWorld = new PhysicsWorld(0, 0, 0);
		dispBoxes = new EntityQueryBuilder(Allocator.Temp)
			.WithAll<PhysicsCollider>()
			.WithAll<LocalTransform>()
			.WithAll<DispBox>().Build(state.EntityManager);

		simulation = Simulation.Create();

		// Добавляем PhysicsStep, если его нет
		if (!SystemAPI.HasSingleton<PhysicsStep>())
		{			;
			state.EntityManager.AddComponentData(state.EntityManager.CreateEntity(), new PhysicsStep
			{
				SimulationType = SimulationType.UnityPhysics,
				Gravity = new float3(0, -9.81f, 0),
				SolverIterationCount = 4
			});
		}

		// Добавляем компонент сразу при создании системы
		state.EntityManager.AddComponentData(state.EntityManager.CreateEntity(), new CustomPhysicsWorldSingleton
		{
			CustomPhysicsWorld = customPhysicsWorld
		});
	}

	[BurstCompile]
	public void OnUpdate(ref SystemState state)
	{
		var entities = dispBoxes.ToEntityArray(Allocator.Temp);

		if (entities.Length > 0)
		{
			if (entities.Length != customPhysicsWorld.NumStaticBodies)
			{
				customPhysicsWorld.Reset(entities.Length, 0, 0);
			}

			var bodies = customPhysicsWorld.CollisionWorld.StaticBodies;
			for (int i = 0; i < entities.Length; i++)
			{
				var entity = entities[i];
				var collider = SystemAPI.GetComponent<PhysicsCollider>(entity);
				var transform = SystemAPI.GetComponent<LocalTransform>(entity);

				if (collider.IsValid)
				{
					bodies[i] = new RigidBody
					{
						Collider = collider.Value,
						WorldFromBody = new RigidTransform(transform.Rotation, transform.Position),
						Entity = entity
					};
				}
			}

			var physicsStep = SystemAPI.GetSingleton<PhysicsStep>();
			var stepInput = new SimulationStepInput
			{
				World = customPhysicsWorld,
				TimeStep = SystemAPI.Time.DeltaTime,
				Gravity = physicsStep.Gravity,
				NumSolverIterations = physicsStep.SolverIterationCount
			};

			unsafe
			{
				simulation.Step(stepInput);
			}

			SystemAPI.GetSingletonRW<CustomPhysicsWorldSingleton>()
				.ValueRW.CustomPhysicsWorld = customPhysicsWorld;
		}

		entities.Dispose();
	}

	public void OnDestroy(ref SystemState state)
	{
		simulation.Dispose();
		customPhysicsWorld.Dispose();
	}
}

// Компонент для хранения PhysicsWorld
public struct CustomPhysicsWorldSingleton : IComponentData
{
	public PhysicsWorld CustomPhysicsWorld;
}

The result is the same, nothing works.

It seems to me that somehow there are too many actions to just get the intersection of the ray with the box? In this scenario, I’d rather screw on some SharpDX (or something even more lightweight), spend at most two hours on it, and everything would work like clockwork (i’ve spent more than two days on this RayCast-Systems now).

Now I’m going to try the MultiPhysics approach, but I’m already losing hope that something will work.

UPDATE:@CMarastoni
I didn’t understand how to set up MultiPhysics, so I decided to try adding the NetCodePhysicsConfig component to the scene, as in this video with this timecode.
1:07:19


But I just don’t have that button. I don’t know what to do…

Then I tried to access this setting through the code, but it’s not there either…

UPDATE 2:
For a strange reason, the NCfE version is 1.3.6 was marked as the latest, and there was no suggestion to update it, although if you try to reinstall the package through the phrase “com.unity.netcode” and the version “1.4.0”, it will magically install… (yes, I was too lazy to watch ALL three and a half hours of the video initially, but in the end I still had to)


However, nothing is working at the moment anyway, and the number of bodies in physicsWorld is zero. It may be worth manually adding them there as in the ClientDispBoxRayCastSystem

I think I found an example similar to my problem in the examples in the official repository. But, apparently, they only demonstrated a hypothetical problem, but did not solve it in any way.

PhysicsSamples/ReleaseNotes

Assets/Tests/MultipleWorlds/ClientServer/ClientServer
  • Added Assets/Tests/MultipleWorlds/ClientServer/ClientServer scene as a simplified client-server sample, with game-critical ghost bodies (exist on both server and client) and client-only bodies that are simulated only locally to add to game’s visual appeal. Bodies in server physics world are simulated first (would be predicted based on server data in real use-cases) and then mirrored into the default physics world. Default physics world contains ghost and client-only bodies, where ghost bodies are driven by their server counterparts (game objects are linked via Server Entity setting in Drive Ghost Body Authoring script). DriveGhostBodySystem drives ghost bodies by position or velocity, the ratio (position vs velocity) is determined by First Order Gain setting in Drive Ghost Body Authoring script on the client ghost body game object. ServerPhysicsSystem is the key system that builds, simulates and exports server physics world. The demo shows different kinematic bars (client-only, dynamic and kinematic ghost) rotating and pushing various speheres around (client-only, dynamic and kinematic ghosts driven by position, velocity and both equally).

Correct me if I’m wrong, but how can create a Client-Server-sample if the project doesn’t even have the NetcodeForEntity package?

Sorry, I may need to ask some further question here:

  1. Is the client connected to the Server (in this lobby) or not? Or it is just a temporary world that you are using before connecting and go into the game?

Asking the distinction because I wondering why you need a ClientWorld in the latter. Of course you can use it, but it is not really designed for that use case completely.

If your intention is to “seamlessly” go from lobby to game or anything similar I can understand.

In all cases, I think the confusing bit it is that you are using a Client World that:

  • may be not connected to the Server
  • a connection that have the “NetworkStreamInGame” component does not exist.

If you don’t need a connection at all, the easiest thing that comes to my mind is to avoid moving the systems in the PredictionSystemGroup in first place.

So, an easy fix is:

    [UpdateInGroup(typeof(InitializationSystemGroup))]
    [CreateAffer(typeof(PredictedPhysicsConfigSystem))]
    public partial struct DisablePhysicsInitializationIfNotConnect : ISystem
    {
        public void OnCreate(ref SystemState state)
        {
           state.World.GetExistingSystemManaged< PredictedPhysicsConfigSystem >().Enabled = false;
        }
    }

This would avoid moving systems around and physics would work as usual.

UPDATE:@CMarastoni
I didn’t understand how to set up MultiPhysics, so I decided to try adding the NetCodePhysicsConfig component to the scene, as in this video with this timecode.

In case you may need something more advanced and you may have connection, but not in game (i.e connected to the lobby, but without ghost data exchanged)

You can:

  • Use the previous approach (disable the system) and eventually re-enable when and if you need to have now physics to be predicted.
  • Use the multiple world setup.

For the latter: it is not currently working for you because the ClientWorld will never run the PredictedSimulationSystemGroup until the NetworkTimeSystem will update the NetworkTime with valid data (at least).

I can help you fixing this the setup, no problem.

Steps:

  1. Add the NetcodePhysicsConfig to your scene (you already did)
  2. Set the Client Non Ghost World to either 1. Unfortunately you can’t use 0 without another little modification.
  3. Add to your physics objects (gameobject) a PhysicWorldIndex authoring component (part of physics) and set the world index to be the same as you set in the previous step (so 1).
  4. Create a secondary physics world group that simulate a specific PhysicsWorldIndex.
    [WorldSystemFilter(WorldSystemFilterFlags.ClientSimulation)]
    [UpdateInGroup(typeof(FixedStepSimulationSystemGroup))]
    public partial class LobbyPhysicsSystemGroup : CustomPhysicsSystemGroup
    {
        public const int WorldToSimulate = 1; //this should match the value you set into the `Client Non Ghost World` property.
        public LobbyPhysicsSystemGroup() : base(WorldToSimulate, true)
        {}

        protected override void OnCreate()
        {
            base.OnCreate();
            //any other things you need.
        }
    }

With this setup, theoretically, you now have your physic scenes and object updated as part of the FixedStepSimulationSystemGroup (client) and running physics as normal.

I said, “THEORETICALLY” because there is a nuance (that is the real problem): to make this work, it is required that the physics systems as run at least one time!!!

But that is never the case until the connection is in-game. Reason: the CustomPhysicsGroup add to itself all physics systems when it run the first time, but that require the PhysicsSimulation type to be set.

Solution for this (I don’t like it but work):

    [UpdateInGroup(typeof(InitializationSystemGroup))]
    public partial class ForcePhysicsInitializationIfNotConnect : SystemBase
    {
        protected override void OnUpdate()
        {
            if (SystemAPI.GetSingleton<SimulationSingleton>().Type == SimulationType.NoPhysics)
            {
                //Force a single update of physics just to ensure we have some stuff setup
                World.PushTime(new TimeData(0.0, 0f));
                World.GetExistingSystem<PhysicsSystemGroup>().Update(World.Unmanaged);
                World.PopTime();
                Enabled = false;
            }
        }
    }

With that, now as expected, you can have your own physics world running on the client, even if not connected with the server or in-game at all.

There are other possible approaches but more complex) to achieve the same that would allow the client to also run the prediction loop etc etc. But I will not cover this here.

Thank you very much for promptly answering literally every question I have!

Ideally, I need a client world where I can use RayCast both before and after connecting. My idea is that the game does not have a UI, but all interaction takes place with objects on the stage (in the example above, the player would “click” on the elements on the monitor screen on the table).

At the moment, I’m trying to create a custom physical world (it has nothing to do with the server or the connection, it’s just needed to process the RayCast and the player can select items on the “screen”).

I found an examples in the official repository:
EntityComponentSystemSamples\PhysicsSamples\Assets\Tests\MultipleWorlds\CustomPhysicsGroup
and:
EntityComponentSystemSamples\NetcodeSamples\Assets\Samples\MultyPhysicsWorld

but it’s like I’m missing some last detail to make it work.

I think that’s exactly what I’m missing:

Thanks!

I tried myself both logic (disable and multi with that extra) and I can confirm it work.

This was a sorta custom setup not document by us at all. So I will add something to the doc.
For now I can stick this convo.