Like you mentioned, this happens because the objects are colliding too fast.
The collisions are calculated once per physics step. What's happening here, is that the objects are moving so fast, that they are not colliding in one physics step, and have already passed collision in the next physics step, never actually have collided.
What you can do, if write a script to predict a collision between objects one step early using their velocities and their positions. This would cause them to bounce off each other a bit early, but it won't matter if they're traveling so fast.
did you mean "high" speeds, or have you had this problem at low speeds as well?
– anon62758266