Unity2D Polygon collision not working

So I have the following scene, with Polygon collision detection (2D green lines)

alt text

and I setup polygon collision detection for the player aswell, but whenever the player collides with the collision in trees and other places, the screen spins. Any ideas on how to prevent this?

With the way my game is setup, I just had to lock the z AXIS in the settings for collision.