User-agent: * 
Disallow: /search/ 

# htdig 
User-agent: htdig
Disallow: