I tried to use an object pool but my games fps will still drop significantly even after just 5 enemies even though I had 50 preloaded.
How can I possibly optimize this to have around 20-25 enemies on the screen at once time?
I am using Invector melee controller and enemyAI scripts on the enemies.
If you have many calls per enemy within the Update function, you should try using IEnumerators (Coroutines). There you can spare a lot of CPU usage. 
Check out this:
