# Allow all crawlers # Developed by Magazine Deals Now Nov. 30, 2007 User-agent: * Disallow: /cgi-bin/ Disallow: /customers/ Disallow: /database/ Disallow: /Demo/ Disallow: /premium/ Disallow: /includes/ Disallow: /professional/ Disallow: /Standard/ Disallow: /userfiles/ Disallow: /control/ # For google bot User-agent: Googlebot-Image Disallow: /*.doc$ Disallow: /*.pdf$