Maze crawler does not generalize

I am making a game where an agent needs to traverse a maze-like dungeon but my problem is that every time I train it the agent overfits to that particular layout. One of the agents runs well to about when exiting the starting room and the freaks out. How can I design an agent that does well on every type of dungeon?