SEO Resources - Frequently Asked Questions
How Many of my Website Pages are Indexed?
Wondering how many of your pages are indexed by your favorite Search Engine? The easiest way is to perform a Search for your company's name.
Not finding anything? This is rare, but it probably means one of two things:
- Your site is new.
- Your site has been banned.
These two obviously carry much different implications. If your site is new you may not have even begun optimizing your pages for Search Engines, submitting your site to Search Engines, building links, etc. If your site has been banned, on the other hand, you have a major problem. The best thing you can do is ensure that nothing on your site will be flagged for spam and file a re-inclusion request.
Some Webmasters concern themselves with their current Inclusion Ratio. This is a fancy way of referring to how many of your pages out of all of your pages are indexed by Search Engines. To see how many of your pages Google has indexed go to Google and do a Search for site:www.yourdomain.com - replacing "yourdomain" with your domain name.
If you're finding that not many or any of your pages are indexed by Search Engines, you're probably wondering what you can do about that.
Well, Search Engine crawlers, the modified browsing programs that index pages and follow the links they find there, are pretty simple. They look purely at the HTML content of your pages. That means that they don't see images or any of those fancy Flash animations. They're interested in one thing and one thing alone: content.
The best way to ensure that Search Engine crawlers can navigate their way around your pages and find/index all of them is to ensure that there are plenty of links from each page pointing to as many of the other pages as possible. A navigation system that forms a kind of template for your site and is found on every page is the best way to ensure that crawlers can find all of your pages.
Additionally, submitting sitemaps to major Search Engines (Google and Yahoo! for example) through their recommended method is a great, free way to get your site indexed as quickly as possible. Since crawlers will first look at your sitemap and use it as a hub from which to access all of your site's pages it is a good idea to keep your sitemap updated.
Using JavaScript, frames and other technologies that modify the way users browse your pages can become road blocks for Search Engine crawlers. It is usually advisable to provide static HTML links to your pages wherever possible to avoid linking methods that crawlers cannot see or follow.
It is also important to note that Search Engine crawlers will usually only index the first 100,000 characters of your documents. This may sound like a lot, but it can add up very quickly if you're including your JavaScript functions and Cascading Style Sheets at the top of your documents.
Dynamic pages can also be troublesome for crawlers. Since scripting languages such as PHP and ASP can potentially lead to infinite loops, crawlers are programmed to ignore certain URLs that appear to be dynamically generated and complex.
In general, the best idea is to keep your site updated and managed on a daily basis, keep Google and other Search Engines up to date on your site's content via your sitemaps and avoid any practices that may result in rendering your site impassable by Search Engine crawlers.
