Slow down time effect causing FPS with pathfinding. Any solutions without fixedDeltaTime?

Greetings, so I’m using a slow down time effect I saw in a tutorial to randomly slow down time when an enemy dies. It works fantastically unless it happens when there are a lot of enemies spawned in at once. I have narrowed it down to this definitely being the issue as if I deactivate it I get no FPS issues at all when killing enemies even with large amounts of enemies. Now I don’t know why this is happening, I believe due to all of my AI trying to do pathfinding and when the time is slowed it severely drops the FPS. The slowdown effect doesn’t cause issues unless there are a lot of enemies pathfinding at once (somewhere in the 30+ range) if only a few are out it works as intended. Below is code my for the slow down time effect, is there something I can change to avoid this from happening? I am also using the A* Pathfinding Project for my AI. The SlowDownTime method is called whenever an enemy dies.

Another person suggested that Time.fixedDeltaTime is causing the issue, which makes sense after explanation since its basically causing my AI to calculate there paths in extremely small intervals making a higher CPU work load. So I guess is there a way to achieve this slow down time effect without affecting Time.fixedDeltaTime?

    public float slowdownFactor = 0.05f;
    public float slowdownLength = 2f;

    void Update()
    {

        if (!gameManager.GetComponent<UserInterfaceHandler>().gamePaused)
        {
            Time.timeScale += (1f / slowdownLength) * Time.unscaledDeltaTime;
            Time.timeScale = Mathf.Clamp(Time.timeScale, 0f, 1f);
        }

    }

    public void SlowDownTime()
    {
        int randomChance = Mathf.RoundToInt(Random.Range(1.0f, 5.0f));

        if (randomChance == 4f)
        {
            Time.timeScale = slowdownFactor;
            Time.fixedDeltaTime = Time.timeScale * .02f;
        }
    }

You should just be changing your FixedUpdate code to scale with time instead.