As others have mentioned already, even with 100 characters you are already running into performance issues on both the CPU and GPU.
You can try this experimental Unity package for vertex animation textures (VAT). It’s not very well written, but it should get you started. The downside of VATs is that you can’t blend between animations.
Here is a great article, if you want to learn more about VATs.
You can also try Kinematica, which uses Motion Matching. AFAIK, it is more performant than Animators because it uses burst jobs. You usually need motion capture data, though. Also, it has been discontinued, unfortunately. There is also a plugin you can purchase called MxM.
Unity is aware of this problem. That’s why they are working on DOTs Animation . Unfortunately, it’s not yet compatible with the current ECS version.