With your google Webmaster tools account, you can view the graphs
of number of pages fetched per day in the Tools->Set crawl rate
option. On the wiki I manage, with about 2,000 pages of content, it
is hit an average of 130 pages per day from google over the past
four months. Certainly not a load of much importance on the WEB server.
Beware of alias names that google may see your site as, such as:
website.com
www.website.com
... etc ...
--Hiram
Agon S. Buchholz wrote:
David A. Desrosiers wrote:
Log into Google Webmaster Tools, add your site to
their system, and
adjust the crawl rate to be slower using the tools provided there.
I tried this; it has no noticable effect, neither on the effective
server load nor on the threads/queries I see in mytop.
Thanks anyway! -asb