Parenting an object causes it to change rotation

When I parent a child object to rotated parent object it makes my child object to change rotation. This happens in edit mode and play mode. Why this happens? I thought objects should stay visually same when parenting them. I use 2D mode. Unity version is 5.0.0f4.

A Child has its position and rotation relative to the parents position and rotation. no problem here.