Servers
I have wanted a simple easy to understand description of what
servers are and what they do. Here is what I have figured out.
After reading the discussions about servers I looked in
catalogs and read the descriptions of the servers that they had to offer.
A server is very much like an ordinary personal computer except it is
larger and may have several very large hard drives to provide a lot of
memory. There has to be some way that other servers or computers can
connect to it and communicate with it. An ordinary personal computer
could be used as a server but it couldn't store very much information.
One server model may have a full tower case, large power
supply, special motherboard with two to four microprocessors, and six
10,000 rpm SCSI hard drives four of which are hot-swappable (can be
changed while the server is operating). They cost about $6,000.
Several servers may be involved in routing a call from your
computer to its destination. One server may be able to handle the calls
from several computers at the same time. A group of several servers may
be located in one place and do different things and more than one can be
loaded with the same data so that if one server fails another can take
over or more than one server can share the traffic.
As the Encyclopedia gets larger and activity increases Wikipedia
will probably need more and more server capacity.
Merritt L. Perkins
Donations
I think that before a person donates he would like to know how
much has already been donated, how many donations have been made, and
perhaps the amount of the smallest and largest donation in each currency.
If one person donates twice it would be counted as two donations.
Most people probably would not want their name, date, or amount of their
donation published and it would be a lot of extra work to publish such a
list. If you want to keep such a record for your own information that is
up to you but I think it is not necessary.
Merritt L. Perkins
Jim Guide wrote:
>I was a developer a while back for the (purportedly)
>17th busiest website in the world and though the
>sysadmins were more directly involved in improving
>response speed, I ended up doing a lot of stuff myself.
Eh gads! We really need somebody with your experience. Please sign up for
Wikitech-l at http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>The way that Wikipedia is slow is very similar to problems
>we had. The delays are almost all in the initial request for
>new pages.
I am not a developer, but I've gotten the same sense by reading/skimming
Wikitech-l (checking to see if links are active or edit also was a big drain
until we started to cache pages for non-logged-in users).
>Once the connection is made, content usually comes
>across rapidly.
Nod. At least for non-logged-in users accessing cached pages...
>This usually points to some sort of full queue in software,
>or a full queue due to excessive connections on a single
>machine causing a hardware wait state.
We often get "too many connections" errors and Apache has to be restarted
often. I also remember the developers having to increase the number of
connections allowed setting a few different times.
>[lot of technical stuff and seemingly good suggestions]...
>I'm not sure what your current server setup is,
Two machines: ref
http://mail.wikipedia.org/pipermail/wikitech-l/2003-September/005976.html
Pliny (supposed to be just a database server)
Software: MySQL, Apache 1.3.27, Linux kernel 2.4.20
Tasks: Host all wiki databases, crunch database queries, webserving the
non-English Wikipedias, Wiktionary, meta, Wikiquote and Wikibooks.
Hardware:
Tyan S2462 motherboard
2 Gig RAM (4 512Meg modules of ECC Reg. 266 MHz PC2100 DDR RAM in 4
slots)
2 Processor AMD Athlon MP 1800+/266FSB
Adaptec AIC-7899P SCSI U160 controller
1 IBM 36 GB SCSI Ultra 160 / 10 K RPM
1 IBM 36 GB SCSI Ultra 320 / 10 K RPM (running as U160, of course)
Dual onboard 3Com 10/100 (3C982) adapters
Larousse (supposed to the webserver for all wikis)
Software: Apache 1.3.27, Linux kernel 2.4.20
Task: Webserving the English Wikipedia
Hardware:
TYAN S2518UGN motherboard
2Gig RAM (4 512 Meg modules of ECC Reg. 133 MHz PC133 SDRAM in 4 slots)
Pentium III 866MHz processor w/256K cache
Adaptec AIC-7899 SCSI U160 controller
1 Quantum Atlas 18 GB SCSI U160 Drive, currently unused and known to be
troublesome (awaiting some testing)
IBM 36 GB SCSI U160 Drive
Dual onboard Intel 10/100 (eepro) adapters
http://mail.wikipedia.org/pipermail/wikitech-l/2003-September/005976.html also
has info on our short term and intermediate term upgrade plans. Oh and you
may find this bit from Jimbo interesting;
"..[short term upgrades will leave] us with two nearly identical machines, and
thus well-poised for these two to be equal load-balancing frontend
webservers in the future, when we may buy a MEGA MONDO db machine."
So I think we are already onto your line of reasoning. IIRC we have the money
for the short term upgrades, and hopefully will have the US$6,000 needed for
the 'MEGA MONDO db machine' after a few more months of donations.
>[more great stuff for wikitech-l]...
Cheers!
-- Daniel Mayer (aka mav)
Hello folks. I posted the following on the 'Why Wikipedia Runs Slow' meta
page, but thought it might get more exposure here on the list. My apologies
if you've already discussed this in depth....
I was a developer a while back for the (purportedly) 17th busiest website in
the world and though the sysadmins were more directly involved in improving
response speed, I ended up doing a lot of stuff myself. The way that
Wikipedia is slow is very similar to problems we had. The delays are almost
all in the initial request for new pages. Once the connection is made,
content usually comes across rapidly. This usually points to some sort of
full queue in software, or a full queue due to excessive connections on a
single machine causing a hardware wait state. New url requests are made to
stand in line, sometimes because settings for maximum simultaneous
connections are too low, or the settings are high enough but all RAM is
consumed servicing current requests, etc,. This may seem obvious, but it
lets us de-emphasize other potential problems such as bloated overworked DB,
bogged disk fetches, etc. So, based on all this, I would say the greatest
single improvement would be to set up some sort of simple DNS round robin
(true load balancing could come later). I'm not sure what your current
server setup is, but if you could have at least two Apache servers running
on two machines with one of them running the Round Robin algorithm I think
the majority of your response problems would disappear. Don't listen to
those who say Round Robin is a naive approach. It's true that allocation of
new connections is done in a "dumb" way (in a two server setup it will just
throw every other connection to the secondary webserver)-- but that's all
you really need, I think. Suddenly each machine is servicing half the client
connections and everything is fast... Of course, maybe the reasons for your
slowness are more complex, but based on what I can see from the client side
my suspicion is that a simple Round Robin would clear it all up and that
simply adding new Apache processes on new servers as you grow would make you
at least 10 times faster during peak times than at present. -- JDG --
tarquin wrote on Wikipedia-l
>Please wait until we have our logo decided
The end of the final vote is on the 25th, I'm sure we
can have the new logo up by the time the first press
releases are distributed on the 29th. If not then we
can wait a few days (or even wait two months for the
more conservative 300,000 alternate article count.
See:
http://www.wikipedia.org/wikistats/EN/TablesArticlesTotalAlt.htm
).
>& a clearer page layout!
? Which page? The meta version of the press release is
just that; a nice and tidy English version will be
created from that and it will be posted on the English
Wikipedia. I suggest that every Wikipedia host their
version of the press release on their own wiki so as
not to confuse people into thinking that meta is a
Wikipedia.
>Everybody stop writing articles now!!!!!!!!!!! ;)
That's not going to happen unless we implement
Nupedia-style editorial controls or pull the plug on
the server. ;)
Things that, IMO, have to be done before we distribute
the press release:
1)
http://meta.wikipedia.org/upload/6/68/Fundraising.html
needs to be finalized by the community and approved by
Jimbo.
2) Fundraising.html needs to be displayed at
http://wikimediafoundation.org/fundraising/
3) http://wikiquote.org and http://wikiquote.com need
to be forwarded to http://quote.wikipedia.org
4) A minimal webpage needs to be created for the
Wikimedia Foundation and displayed at
http://wikimediafoundation.org (http://wikimedia.org
should continue to be a portal to the different
Wikimedia projects; the only change that is needed is
a link to the foundation's website).
Things that would be real nice but not absolutely
necessary, IMO:
1) Upgrade Pliny (our webserver) and Larousse (our
database server). I'm tempted to move this item up...
2) Translated/different language versions of
Fundraising.html created.
3) Move Wikibooks to http://wikibooks.org
4) Move Wikiquote to http://wikiquote.org
5) Either revive Nupedia.org (the website) or remove
all mention of it from the Wikimedia.org page. We will
still have to decide what to do with this comatose
project. But not today.
Lower priority (mainly symbolic but still important in
the long run, IMO):
1) Move meta to http://meta.wikimedia.org (temp
Wikimedia logo needed)
2) Move the mailing lists to http://mail.wikimedia.org
3) Move the database download page to
http://download.wikimedia.org
4) Move the test wiki to http://test.wikimedia.org
5) Move the English Wikipedia to
http://en.wikipedia.org and create a language
selection portal at http://wikipedia.org (I know, way
too much work for a week for little real immediate
benefit; that's why these items are marked "lower
priority").
-- Daniel Mayer (aka mav)
Related links:
http://meta.wikipedia.org/wiki/Wikimedia%27s_first_press_releasehttp://meta.wikipedia.org/wiki/Image_talk:Fundraising.html
__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
> >Everybody stop writing articles now!!!!!!!!!!! ;)
>
> That's not going to happen unless we implement
> Nupedia-style editorial controls or pull the plug on
> the server. ;)
That might if wikipedia goes on being stuck every
couple of hours ;-)
__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Hi Mav (or Jimbo),
Re http://meta.wikipedia.org/wiki/Image:Fundraising.html
In your function of Treasurer for the board, I think it would be really
helpful to add Information (on a subpage?) regarding how much is already
there / who gave on what date. A lot of comments (especially on the german
wikipedia) are always "Who is this Jimbo", "What is this
Wikimedia-Fondation", "Why should we trust this people", "Can we not have
our own German Server/Organisation"... Maybe with a little bit of
transparency we could show them that this is a really "Open" Organisation.
Maybe something like this:
List of Donations as of xx.xx.2003 in total 123.4 USD:
Date / Amount / Real-Name / (Country?)
Date / Amount / Wiki-Username
Date / Amount / (wants to stay) Anonymouns
Date / (wants to keep amount secret) / Name or Anonymous
...
If the users see that the amount send was officially stated/received
somewhere, maybe they would put more trust (and money) into Wikimedia/pedia.
Just an Idea,
Fantasy :-)
OK, I hacked together a Wikimedia fundraising page for
use at http://wikimediafoundation.org/fundraising/
based on the en.wikipedia donations page.
Please leave comments on the image talk page
(translations/different language versions needed too).
http://meta.wikipedia.org/upload/6/68/Fundraising.htmlhttp://meta.wikipedia.org/wiki/Image:Fundraising.html
We are also getting real close to the 300,000 article
milestone so I would like to suggest that the meta
version of the press release be frozen to everybody
except Jimbo (who could make changes up to Sunday).
Then all next week the non-English versions would have
time to finalize their versions of the press release.
The week after that I would like to see the
non-English versions distribute their press releases
and for us to wait a week before releasing the English
version (in order to smooth out the load and give the
non-English versions a head start before everything
gets bogged down).
What say you?
-- Daniel Mayer (aka mav)
__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
See Wikipedia:FAQ -> hungarian interlanguage link. it links to the correct
page, but reaches nirvana.
It may be related to the messup between hu.wikipedia.org's Wikipedia vs.
Wikipédia namespaces, but haven't seen such clash so far. Seems "normal"
works, URI-encoded doesn't.
cya,
grin
The process that actually delivers mail for the mailing lists didn't start
for some reason when the server was restarted, so while it was queueing up
every message sent to the list, it didn't send anything out or put
anything into the archives. :P
Back up now. Sigh...
-- brion vibber (brion @ pobox.com)