ArgumentException: NaN increment passed to AddReward.

I was training my agents and suddenly during the training this error popped out, I do not find a reason why can this happen, there has to be a number in that place.
That is why my training was stable until this happen:

2020-06-24 16:59:39 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 3900000. Time Elapsed: 10656.156 s Mean Reward: -6.788. Std of Reward: 9.317. Training.
2020-06-24 17:08:36 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4200000. Time Elapsed: 11193.373 s Mean Reward: -6.011. Std of Reward: 9.916. Training.
2020-06-24 17:19:44 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4500000. Time Elapsed: 11861.222 s Mean Reward: -290.725. Std of Reward: 695.414. Training

Maybe a division by zero somewhere? That’s where I usually get my NaNs from :wink:

The method where it happens is this:

public float GetEntropyCompleta2()
    {
        int[] histogram = getHistogramaCompleto();
        double[] lista = new double[histogram.Length];
        double entropia = 0;
        int deads= histogram[0];

        for (int i = 1; i < histogram.Length; i++)
        {
            lista[i] = histogram[i];
            lista[i] = lista[i] / (144 - deads);
            if (lista[i] != 0)
            {
                entropia += (lista[i] * Math.Log(lista[i], 3));
            }
        }
        return -(float)entropia;
    }

The int deads is never bigger than 144, and the values is lista can not be negative. Any ideas?

But can it be 144? That would give you a division by 0, resulting in a NaN in lista<em>.

It could be but is quite hard, it it happens I will let you know

Also note that Math.Log(0) will produce a NegativeInfinity, and 0 * NegativeInfinity results in a NaN.

There is an if before for that.

Sorry, I missed that part. I’m not sure how lista _* Math.Log(lista*, 3) will behave if the value is very small (but non-zero) though.*_

I think that might be it.