Is there some hidden time or memory limitations with recursive tasks ?

Hello everyone, thanks in advance for your time.

I’m trying to implement a Chess game that can be played automatically by AI using the MinMax algorithm.
I have made a Node class to construct the tree of possible moves along a Coroutine to test and generate children recursively to see if the tree builds correctly. I start the coroutine in another MonoBehaviour by the way.

Using Rider, I’ve setup the debugger and used the System.Diagnostics.Debug.WriteLine to gather info in realtime.
Up to a depth of 2, the coroutine completes and show the time taken to do so but with a higher depth, it seems the process dies or stops for unknow reasons. The fact that the logging of the debugger shows the same symptoms and don’t encounter any Exceptions on the way genuinely leaves me baffled I must say.
And for the sake of being clear, time isn’t the issue. I would be glad if the process could complete even if it takes few minutes instead of weirdly timeout.

I tried many things : with/without a Coroutine, same for the independant thread part, without any logging…
If someone have already experienced something similar with time/memory consuming operation like this or have some hints on what is happening or where I could investigate more, I will be very grateful.

Thanks again, in advance, for you time and your knowledge !

For all intents and purposes, here a stripped version of my Node class with the coroutine and the recursive method to generate all the children nodes.

public class Node
{
    public int Depth;
    public Side? MaxingSide;
    public Node Parent;
    public List<Node> Children;
    public Piece[,] Grid;
    public float HeuristicValue;

    public Coordinates? Origin;
    public Coordinates? Destination;

    public Node(Piece[,] grid, int depth, Coordinates? origin, Coordinates? destination, Node parent = null, Side? side = null)
    {
        Grid = Matrix.DuplicateGrid(grid);
        Depth = depth;
        MaxingSide = side;
        Origin = origin;
        Destination = destination;
        Parent = parent;
        Children = new List<Node>();

        if (Origin != null && Destination != null)
            PerformMove();
          
        // HeuristicValue = EvaluateHeuristics();
    }

    public void GenerateChildren()
    {
        Side sideToEvaluate = MaxingSide ?? Side.Light;
        List<Piece> sidePieces = Matrix.GetAllPieces(Grid, sideToEvaluate);
        Children = new List<Node>();

        foreach (Piece piece in sidePieces)
        {
            foreach (Coordinates move in piece.AvailableMoves())
            {
                Node child = new(Grid, Depth + 1, piece.Coordinates, move, this, sideToEvaluate);
                Children.Add(child);
            }
        }
    }
  
    public static IEnumerator RunNodeGenerationCoroutine(Piece[,] grid, int depth)
    {
        Stopwatch stopwatch = new Stopwatch();
        stopwatch.Start();
          
        // Start the node generation on a separate thread
        var thread = new Thread(() =>
        {
            Node rootNode = new Node(grid, 0, null, null);
            GenerateNodesRecursively(rootNode, depth, 0);
        });

        thread.Priority = ThreadPriority.Highest;
        thread.Start();

        while (thread.IsAlive)
            yield return null;
          
        stopwatch.Stop();
        UnityEngine.Debug.Log($"Process completed in: {stopwatch.Elapsed.Minutes}m {stopwatch.Elapsed.Seconds}s {stopwatch.Elapsed.Milliseconds}ms");
    }

    private static void GenerateNodesRecursively(Node node, int maxDepth, int currentDepth)
    {
        string indent = new string(' ', 4 * currentDepth);
        if (currentDepth <= maxDepth)
        {
            Debug.WriteLine(currentDepth == 0 ? "0 Root node" : $"{indent}Depth {currentDepth}: {node.Origin} -> {node.Destination}");
            node.GenerateChildren();

            foreach (Node child in node.Children)
                GenerateNodesRecursively(child, maxDepth, currentDepth + 1);
        }
    }
}

9814227–1409946–Node.cs (6.02 KB)

Recursion or not, Unity will lock up 100% of the time EVERY millisecond your scripting code is running.

Nothing will render, no input will be processed, no Debug.Log() will come out, no GameObjects or transforms will appear to update.

Absolutely NOTHING will happen… until your code either:

  • returns from whatever function it is running

  • yields from whatever coroutine it is running

As long as your code is looping, Unity isn’t going to do even a single frame of change. Nothing.

No exceptions.

“Yield early, yield often, yield like your game depends on it… it does!” - Kurt Dekker

Thanks a lot for your answer. I quite agree with you.
Actually, in my particular case, I would mind blocking everything.

What I made is surely not optimized but it allows me in Rider to see if the process is running or not. I’m also glad that, beyond the flagrant issue, I was able to not freeze Unity with that makeshift job.

So far, I believe that whenever the time it takes, the recursion will reach it’s end, trace back all way up (as it doesn’t return something) to the coroutine, which ends soon after, as it do nicefully with a depth of 2.

What bother me the most is that I have something running but I don’t know what could block the process as the coroutine yield until completion and the recursion should end with enough time.

To understand recursion, you must first understand recursion.

Since a chess game’s brute force search space expands at an insane rate with each step deeper, most likely you are just going compute bound for a really long time.

You can prove or disprove this by inserting instrumentation that yields more frequently within your coroutines, or alternately only considers ONE random possible move per depth, just to prove it exits.

Oh, I understand more what you’ve been suggesting in the first place.
I didn’t tought about checking the recursion itself as the base implementation works for the minimal depth of 2.

A simple thing I did is to generate only one children, one possible move per node. At least the recursion can process and exit without problems.

So I circle back to what I was afraid of… If it isn’t a issue with recursion or execution logic whatsoever, it only leads to problems with performances. I’m still studying so maybe it’s silly of me to believe I would get an overflow, out-of-bounds or any critical Exception in that regards. Even if catching an exception isn’t guaranteed when I’m toying with Coroutine or Threads, my application or pc could just crash at some point which it doesn’t even after trying to let the coroutine runs for a hour.

For instance, I’ve just tested, keeping one child per node, to recurse a thousand times and 10 thousands times.
I could reach a one-branch tree of a thousand nodes but the test with 10 thousands resulted in a StackOverflowException (considering the amount of copy I’m making data-side and haven’t implemented any kind of Alpha–beta pruning yet, it was expected).
So my goal here is not to find a magical solution but understand what could happen behind the scenes since I’m kind of blind even while debugging by myself.

EDIT: Also, maybe my first .gif is misleading by lasting twenty seconds… It does show the process stoping for a unknow reason I’m trying to uncover.

Considering how fast a chess games moves balloon outwards, it is very unlikely you would ever find a suitable solution via brute force searching:

It gets big FAST, really fast.

And if your fitness function has any serious meat to it, it’s gonna be out of reach all that much faster, especially if you’re digging into Unity engine-side objects for individual queries / computations. All that stuff stacks up quickly in the inner loop.

There’s a reason NASA strictly forbids the use of recursion on any non earth-based missions. It is very handy and can make even seemingly impossible tasks trivial in terms of logic but machines have limits on stack size and when you have these kinds of cases those limits can be reached in a matter of milliseconds. And it’s super difficult to get it right when trying to execute over time like you are doing here.

Anything you can do with recursion you can do with iteration and I think given the search size as well as the need to deliberately yield execution or even cancel execution at strict points you’d be better off using that instead. You very well might even find performance improves if you take caching into account.

3 Likes

You just can not MiniMax chess, at least not completely. You could go to a certain depth and then have to stop. Though chess AI usually has to be much more clever to overcome those restrictions. I recommend watching this computerphile video on AI and Minimax. They compare tic-tac-toe, chess and go and essentially explain why you can not minimax chess. The classical MiniMax algorithm goes through the end to all possible outcomes and then goes backwards to assign the best move to each node. You can not iterate through ALL possible chess games.

2 Likes

Thanks all of you for your answers.
After persevering a bit more I finally decided to found another way around and start a diffirent approach.
Thanks again for all your suggestions and advices.