I haven’t tried to do anything exactly like this myself, so I’m just speculating here. But, here’s what I’d probably try first if I were to tackle something like this.
As you noted, waypoints probably aren’t really an ideal solution here. What might be better would be a spline.
The nature of the spline is secondary, more or less (you could use, for example, a connected series of Bezier curves). The important part is that using a spline would allow you to position the cart (and the track geometry itself) with more or less perfect accuracy.
For moving the cart along the spline, you’d probably want to either use arc-length reparameterization, or ensure that the spline control points are such that constant speed in parametric space equates to nearly constant linear speed in 3-d space.
For ‘tipping’, you should be able to determine the ‘tipping force’ based on the cart’s speed and the curvature of the track at that point. Up to the ‘tipping point’ you could simply animate the cart tipping up on two of its wheels; past that, you could decouple the cart from the spline and let normal rigid body physics take over.
For ‘jumping’, you can determine whether and how much the cart should leave the tracks based on the current speed and, again, the curvature of the track relative to the cart. For small ‘cosmetic’ jumps, you could just animate the effect. For larger jumps, you could again put the cart in ‘rigid body’ mode. For landing, you could use normal rigid body motion and collision detection or a closest-point-on-spline test to determine if the cart has landed on the track (and with the proper alignment), and can resume following it.
There may be other ways to do it as well (for example, having a ‘virtual anchor’ follow the track and have the cart be attached to the anchor by a spring). Overall though I think a spline-based approach would likely be the way to go, as it would probably be difficult to get the cart to follow the track in a natural and realistic way using only rigid body physics.
Again though, I haven’t tried anything like this myself, so I’m just speculating.