Example Of A robots.txt File
FAQ - Search Engine Optimization (SEO)
Wondering how your robots.txt file needs to look? It's pretty simple.
The following is an excerpt from a robots.txt file we used for www.NewSunGraphics.com in the past. Obviously your own robots.txt file should reflect the domain and content of your own site. For more information on what you'll want to "disallow" with your robots.txt read this page from our FAQ.
# robots.txt for http://www.newsungraphics.com/
Sitemap: http:/www.newsungraphics.com/sitemap.xml user-agent: *
disallow: /css_styles
disallow: /aspnet_client
disallow: /enewsletter
disallow: /images
disallow: /images_old
disallow: /javascript
disallow: /scripts
disallow: /software
disallow: /templates
disallow: /error.htm
disallow: /ftp_*
disallow: /favicon.ico
disallow: /111_*
disallow: /ecommerce_*
disallow: /Email_*
disallow: /emarketing_*
disallow: /google_*
disallow: /ecommerce_*
disallow: /thank_you.htm
disallow: /features/create-html-Email-campaigns.jsp
disallow: /for_Email/sell_sell_sell.htm
disallow: /googlecdfd6ce2afe5e1db.html
disallow: /web.htm
disallow: /web_analysis.htm
disallow: /web_metrics_sites_benefits.htm
disallow: /open_source.htm
disallow: /ftp_login_help.htm
disallow: /hosting_*
disallow: /photography.htm
