Tests so far:
Using a mesh without a physics material: No change.
Decreasing phsyics timestep to 0.01. The objects actually stuck more fiercely.
Using a static box collider in the wall. No change.
Separating the mesh into 3 meshes. No change.
Increasing contact offset to 0.02. No change.
Increasing contact offset to 1. No change.
Increasing solver iterations to 10. No change.
Tested using continuous, continuous dynamic and discrete. No change.
The mesh they are colliding against is quite simple:
They receive a force just after spawned, but nothing too strong either.
This is actually an older picture since I had to join the bottom floor with the floor inside the corridor (players were clipping through the seam some times). @Edy do you think splitting up each section would solve or at least improve? I mean like this: