[Foundation-l] Re: Hosting scans of the 1911 Britannica on Wikimedia

Wed Nov 9 22:06:58 UTC 2005

Lars Aronsson wrote:
> Brian wrote:
> 
>>I recently (today) acquired a DVD containing scans of every page 
>>of the 1911 Britannica, along with index files for it all, 
>>[...]
>>TimStarling specifically asked me to tell you all that he is 
>>"confident that the server requirements will be minimal." They 
>>would set up a domain name, generate some web pages 
>>automatically using the index files, and host the entire set of 
>>29,700 files totaling about 4 GB.
> 
> 
> What is suddenly wrong with using Wikimedia Commons and 
> Wikisource?  Why do you need a new domain and server just for this 
> book?  Didn't you see
> http://en.wikisource.org/wiki/The_New_Student%27s_Reference_Work

When Brian came on to IRC and asked us "What is the best way to upload
30,000 images requiring 6 GB to commons?" the reaction from Brion and I was
a groan. The hardware requirements for commons are rapidly increasing, and
uploading and storing such content in MediaWiki is inefficient and
non-portable. If we had them in a separate directory on a separate domain,
we could copy them from server to server, make tarballs, run batch
conversion jobs -- all with a minimal amount of programming and system
administration work. And it wouldn't require writing a bot to create 30,000
index pages, we could just write a hundred lines of PHP to index the whole
lot. The collection will be easier to use and more reliable, and it will be
easy to maintain and update the index pages.

All of the navigation text, the headers and footers, could be editable in
wiki fashion. You could let anyone change the header that will be displayed
on 30,000 pages, with no server strain whatsoever. This is in stark contrast
to the system requirements of templates which are used on large numbers of
wiki pages.

Wikisource has suffered so far due to a lack of specialised software. This
kind of initiative could see it become more usable generally.

-- Tim Starling