spider

A spider (also known as a crawler) is a program that browses web sites extracting information for search engine databases. Spiders can be summoned to a site through search engine registration or they will eventually find your site by following links from other sites (assuming you have links from other sites).

Spider Tips

• Spiders do not read pages as browsers do. They generally cannot execute JavaScript, including links performed by scripting, or frames links.