Sunday, August 21, 2011

Create Your Own Googlebot - CodeProject

Introduction to Article

Googlebot finds and indexes the web through hyperlinks. It still goes to the new pages through hyperlinks in the old sites. My searchbot (Xearchbot) can find sites and stores its URL, title, keywords metatag and description metatag in database. In the future, it will store the body of the document converted to plain text. I don't calculate PageRank, because it is very time-consuming.

Create Your Own Googlebot - CodeProject
Enhanced by Zemanta

No comments: