Tim Starling wrote:
My idea for faster, cheaper access to Wikipedia in
poorer countries:
http://meta.wikimedia.org/wiki/Reducing_transit_requirements
It requires patching squid and extending the HTCP protocol. The latter
should be no big deal, the protocol is only in the "experimental" phase
after all.
On reflection, inventing new protocols for taking a real-time feed is
probably uncessary for what will be a batch operation.
How about simply making plain-ASCII logfiles available, where each line
consists of
<timestamp> <url>\n
which is grown in real-time by appending to it?
People wanting to freshen their caches can then download, trim off all
entries previous to the last time they fetched, uniq the file to get
only a single copy of each url, and then run a script to wget all the
uls from inside their cache during off-peak hours, thus freshening it.
Very little software, a twenty-line perl script, no new protocols, and
(I hope) achieves the same effect.
-- Neil