How to lock local Y rotation?

I’ve made one of those child’s toys where it’s like a box with a maze in it, and a little ball, so you tilt the board around to make the ball move through the maze. It’s working great, but now I’m trying to introduce doors, and it doesn’t work at all, even though the concept is super simple, and I thought it would work immediately.

Here’s an example.
I have an L shape, with the pivot point at the corner. I have two box colliders on it, one for each wall of the L shape. I have a rigid body on it, so that the ball can interact with it, pushing it to make it rotate. I want the location to be completely locked, and the rotation of the X and Z to be locked. This way, if the ball bumps into it, it will spin, working as a door. The problem is, the door seems to be spinning on ALL axis for some reason. It ends up horribly disfigured, sticking into the floor a bit, and getting all messed up.

I imagine this is because the door is part of the whole object, which tilts forwards and backwards and side-to-side, so that you can make the ball move around the maze. So I’m guessing the locking constraints on the door’s rotation are applying to the global axis, and since the board is tilted, the tilt isn’t in relation to the board’s rotation.
I know you can specify rotation by script, but I want to use physics to make the door spin, because the whole game is using physics to direct the ball and interact with the environment.

How can I make the door only rotate on the local Y axis, so that it will act the way I want?
Thanks for any help you can provide!

EDIT:
As I’ve done more, it seems like the freeze position and rotation doesn’t work at all how it should. My one piece is locked in position, all three axis, and yet, when I hit it, it starts floating upwards into the air. Nothing else is moving, nothing has changed, but the object is moving, even though it’s location is locked. Why would that be happening?
Also, I tried forcing the rotation of the object’s on the X and Z axis to reset themselves in a lateupdate, according to the local rotation of the board, thinking that would force it to work. However, now they won’t rotate on the Y axis for some reason, even though it’s not locked, and I’m not resetting that. I understand the concepts here, but it’s like nothing is working how you would “think” it should.
EDIT 2:
Honestly the whole system seems broken to me, but I know it’s probably my own ignorance. If I make an object freeze all 3 axis of position, and none on rotation, but then in script I make the rotation reset, they don’t rotate at all, but they move all around the board, even though their position is locked. Then, oddly, if I stop the script portion fixing the rotation, all of a sudden they do NOT move around the board. How does that make any sense? If I fix the rotation it breaks the position, but if I don’t fix the rotation then the position will lock correctly? That makes zero sense to me.

So I’ve managed to get it roughly working, except it spazzes out the whole time, rather than working according to physics. So if I gently nudge it, it rotates like 180 degrees in a single frame, rather than how physics would interact properly. I’ve turned off all the freeze position and freeze rotation, since those seem super buggy, and don’t apply locally. I also turned off gravity, because it shouldn’t be doing anything. It also sometimes moves if I move the board, even though it shouldn’t move at all, since it’s a child of the board, and doesn’t have gravity applying to it, and yet when I tilt the board the object slightly rotates on the Y.
Here’s the code I got currently, BoardV is a V3. It’s a very simple concept, so I have no idea why it’s not working. I’ve also tried update, fixedupdate, and lateupdate, no difference. Doing the below code makes them not move, which is great, and the X and Z rotations don’t rotate, which is great! So almost there. It’s just that the Y rotation spazzes out at the slightest bump. How do I make it interact properly like a physics object?

void Start()
    {
        OriginalPos = this.transform.localPosition;
        OriginalRot = this.transform.localEulerAngles;
    }

    private void FixedUpdate()
    {
        this.transform.localPosition = OriginalPos;
        BoardV = OriginalRot;
        BoardV.y = this.transform.localEulerAngles.y;
        this.transform.localEulerAngles = BoardV;
    }

I’ve just tried putting a limiter on it, so it can only rotate so many degrees each update, in hopes of stopping it from spazzing out. That made it better, but still, the thing doesn’t follow physics at all. It will bump in the same direction as the thing applying force (which is weird), it’ll just keep spinning at times, the ball will warp through it (yes it’s on continuous collision detection), it’ll stop rotating all together as if something is blocking it, etc. Just lots of weird behavior. There has to be some sort of fundamental thing I’m missing here, because the physics here is stupidly messed up.

EDIT:
I’ve just tried adjusting the angular velocity, so that the thing isn’t allowed to rotate too fast. Sometimes this works somewhat well, and other times the object still spazzes out. I have something stopping it from rotating too fast, and something else stopping it from rotating too much too quickly, so it definitely shouldn’t be able to rotate fast, and yet, it spazzes out, rotating way more than it should be able too. It honestly just feels like the physics system in Unity is just super buggy, because by all accounts what I’m doing now should work.