Issues with PhysicsWorldIndex

I am creating a new entity and immediately remove its PhysicsWorldIndex component as i do not want it to take part in the simulation.

However i have a problem where the entity disappears from the screen and i noticed that the LocalToWorld matrix seems to stop getting updated.

When i add back the PhysicsWorldIndex things go back to normal.

Is this a bug or is it supposed to work like that??

Probably has the physics smoothing component still which overrides the local to world calc I think