[Mediawiki-l] googlebot

Tue Aug 16 15:18:38 UTC 2005

Am 16.08.2005 um 17:14 schrieb andres:

> Accidentally i watched my logfiles (tail -f) and noticed a weird  
> behaviour of the google spider.
> It spends literally days following all links in Spezial:Recentchanges.
>
> It may be a apache configuration mistake (how?), but it also may be  
> a mediawiki problems.
>
> How can i disallow search engines from indexing all the recent  
> changes?
> They are worthless to index anyway.

you can use http://en.wikipedia.org/robots.txt

ciao, tom

--
http://de.wikipedia.org/wiki/Benutzer:TomK32
http://www.tomk32.de