How to distinguish between overfitting, local maximum, and just needing to train longer

When training stalls out at a certain point in my levels I’m trying to diagnose what is happening so I can try and tweak hyperparamters accordingly

overfitting has good results in Training but when changing the env a little bit it wont work, for local maxima just let it train longer and see (maybe you will need better training like curriculum)

In many cases I had a bug causing issues