Unity won’t spend 34ms because of “5 bullets”. With all due respect, this is a story you’re telling yourself because you don’t know what is going on! 
The number 5 though isn’t what’s relevant. I could add 5 GameObject but it might contain a million colliders. I’m not saying it does but numbers like this are irrelevant. The important part is knowing what those 5 are doing.
For example, here’s a real case reported as a bug but it happens all the time: I created a single trigger collider and my FPS drops to 20 fps. It turns out that this trigger covered a whole tilemap. It might have to figure out from the 5000 shapes, which if any it must trigger on. I could shout, “I’m only adding a single trigger!” There are many variations of this and the more common one are a bunch of overlapping objects for a few physics frames, often they have complex colliders so many, many shapes. You end up with the number of objects squared pairs of contacts to calculate.
There is a new 2D physics profiler coming that will help solve this as it provides much more detail for the broadphase here looking at broadphase updates/pairs. The manual will go into hot spots to look for and common causes:
For you, you’re saying you only create “5 bullets”. But there you mean a body and a polygoncollider2d. So how many shapes does this polygoncollider2D produce? Colliders don’t come into contacts in the physics engine, it knows nothing about Unity or colliders. Let’s say each created 100 shapes because it’s less than optimal. I might’ve forgot to stop those bullets from contacting each other using the LCM so the “5 bullets” equates to 5*100=500 * 500 potential contacts or 250,000 potential pairs because these 500 shapes are overlapping each other. 500 potential contacts, maybe 500 real contacts.
Hopefully you get my point in that to even stop these, the Layer Collision Matrix needs to be set correctly to stop these potentials before they even begin. Keep your shapes down to a minimum if you’re going to allow them to overlap and not use the LCM to stop them from interacting. In my example I’d ensure that my bullet colliders were not overly complex which is why they’re often simpler single physics shape colliders such as a circle, box or capsule. A polygon collider can create 1 to n shapes.
But they overlap when you create them and sounds like this is where you’re saying, “why the spike?” And you’re also saying they’re each a polygon collider with an outline of 16 points. These 16 points are an outline, not a physics physics. It gets broken down into multiple convex polygons. You can look in the inspector to see how many shapes they produce too. ShapeCount is shown in the Collider2D Info rollout.
Note that you didn’t mention you’re using the multi-threaded solver until now. A key piece of information. So not everything can go wide. It makes no difference if you have 12-Cores if the bottleneck isn’t something that’s going wide. Box2D is not multi-threaded, we had to add that into key places that we could. Some areas just cannot because it would take a rewrite but I don’t think that’s the issue here.
Triggers can be the silent enemy. They don’t produce a collision response so you don’t visually see them interacting. If you don’t add in a trigger callback or you don’t monitor that you’re getting bullet/bullet triggers then again, you’ll not know. You know when you suddenly add complex triggers overlapping complex triggers because you’ll see “FindNewContacts” and it working furiously. A simply check in the LCM stops it.
