Prevent the seat on a two-wheeled object from rotating?

I’ve got a wheelchair model. On it, are two wheel models that each have a hinge joint on them, attached to the main seat. The legs are just a model that’s a child of the seat and don’t have a collider or anything of the sort, just a visual effect since the game I’m making is in first person.
The seat has a mass of 5, and each wheel has a mass of 3.
The seat has a frozen X-rotation.

When the wheels spin, the chair moves, and that’s all fine and dandy, that’s what I want. The problem is, is that over time, the chair tilts backwards or forwards depending on how many times the wheels have spun.

This tilting happens slowly, but surely. It obviously reverses when the wheels are spun the other way (the player moves forward).
I’ve tried freezing the Z and Y rotations, but then the wheelchair just doesn’t turn at all when only one wheel is spun. Is there some way to stop it from doing so?
In case it is useful, here’s the object hierarchy:

Wheelchair (Empty gameobject)

There’s not much special about it. They’re only linked by the hinge joint on each wheel.

Solved it. I made the seat invisible and then created another seat that has frozen rotation and frozen position and is kinematic, calculated the midpoint between the two small nubs at the center of the wheels, then added ~0.173 to the y of that vector to raise this uninteractable seat to the proper level. Then I just did Quaternion.LookRotation to rotate it to face the proper way based on a vector created from the difference between the two nubs rotated 90 degrees to point forwards.