It's fine to use an iPod touch for development. As long as you avoid iPhone-specific features (which I don't think any are directly supported by Unity anyway; you'd have to add code for that yourself), they are the same, keeping in mind that the iPhone tends to have more services running.
The biggest consideration going in is resolution. If you are making a 2D game, I would suggest starting with iPad and scaling down to iPhone. No one likes a 2D iPad game that just has all of the textures doubled in size, they want more detail in the art; might as well start developing at this higher level of detail rather than trying to add it in later.
As far as performance, the devices perform at a linear level from the first all the way up to the iPhone4/iPad. One of the restrictions on the earlier devices is physical RAM. You should decide what the lowest phone/touch you want to support is and build to that. Aim to get the performance running as you want on that, and you don't have to worry about the later ones.
There are few performance considerations related to the phone-functions specifically that cannot be solved by restarting the device.
If money is a consideration, get a beat-up old phone or touch on ebay, you can save a lot of money if you can deal with some cosmetic damage.
Also, for my first app; I actually developed entirely on iPod Touch, then borrowed my girlfriends iPhone for a couple of test runs before Apple submission.