Making certain that search engine spiders can crawl your website is one of the goals of effective search engine optimization. Your site can be well-designed and contain great content and graphics, but if the links to and within your website don’t allow search engines to access and index all of the pages in your site, you’re paying for a lot of dead space. Pages that aren’t crawlable are pages that aren’t actively contributing to your Internet marketing program. The more material on your website that is accessible — crawlable — by search engine spiders, the higher your potential ranking when Internet users initiate a search. One of the primary goals of search engine optimization is to maximize the amount of material accessed and indexed by search engine spiders.
Spiders, also called bots or ants, are software programs initiated by Internet search engines to find content, including web pages, images, videos, pdf files, etc. Spiders revisit website on a regular basis searching for new material. When spiders can’t follow links to or within a website, the destination page is not included among the index of web pages available to satisfy search results. As far as the search engine is concerned, that page doesn’t exist. Which means that the information contained on that page will not be available to potential customers initiating a web search.
What are some of the linking errors that negatively impact crawlability?
> Embedding navigation links in Flash. Although Flash indexing is expected to be available in the future, most search engine spiders cannot currently crawl links in Flash files.
> Embedding navigation links in JavaScript or Ajax. Googlebots and other spiders just don’t seem to like the taste of JavaScript or Ajax and are not able to crawl pages that use them.
> Embedding site navigation links in forms. Most search engine spiders don’t have the capability to interact with or make the selections required by online forms so they avoid them.
> Failing to provide authoritative links. Links provide the path by which search engines discover new websites. Inbound and reciprocal links from one site to another positively impact SEO rankings. Internal links lead from a website’s home page into important interior pages. A site that fails to provide authoritative links, while not uncrawlable, diminishes the frequency with which it is crawled which, in turn, negatively impacts SEO rankings.
Search engine optimization corrects these linking errors to make your site more crawlable and improve your SEO ranking.
Next Tuesday: Tips for improving your website’s crawlability
