[Labs-l] Google bot

• Previous message: [Labs-l] restarting qa-morebots on tools
• Next message: [Labs-l] Google bot
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi,

I saw a high load (dozens of queries [1]) hitting one of my tools
(catscan2). The queries looked like they came from a template on French
Wikipedia (category name different, other parameters the same). Access log
shows (among other things) Google bot. When I added that to my bot
exclusion list in .lighttpd.conf and restated, the number of queries went,
and so far stayed, down.

I vaguely remember that indexing bots (like the Google one) were filtered
out by Labs already? If it does, was that query drop random then?

Cheers,
Magnus

[1] https://tools.wmflabs.org//catscan2/server-status
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20141019/3128f560/attachment.html>

• Previous message: [Labs-l] restarting qa-morebots on tools
• Next message: [Labs-l] Google bot
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]