[Foundation-l] Re: wikistats (was Information flow)

Fri Aug 26 13:36:59 UTC 2005

Jakob Voss wrote:

> Cormac Lawler wrote:
>
>>> Collecting statistics from full database dumps is a slow and heavy 
>>> process.  We could do better.  But only if we know which stats to 
>>> collect.
>>
>
> Most statistics can be created out of the database dumps but first you 
> have to know how to get it, where to put it and how to treat it.
> I have updated http://meta.wikimedia.org/wiki/Help:Export but you 
> still need some hardware and programming skills.

There are many statistics, particularly user counts and attempts to 
determine authorship of a particular artile or revision histories of 
articles, that simply can't be obtained from the Special:Export feature, 
as it is currently implemented.  And for other statistics that would be 
useful, I fail to see how the Special:export feature is any different 
from simply scraping the HTML page itself.  In short, you need a full db 
dump to do most statistical analysis.  I wish that you could get user 
(contributor) information through the special:export pages, but I havn't 
been able to get it.  That is, who did what and what has been added by a 
given contributor.  What is there in the special:export function is 
terriffic, but it is only a good start.

-- 
Robert Scott Horning