How to protect your DEV sites from crawlers

Something sometimes ignored, is that you can restrict access based on IP on Apache:

http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#ord...

Easy access for everyone, and more importantly REST APIs work out of the box, etc..