Before I start, let me say Thanks to Tim for what may have been a very
expensive phone call this morning when pliny fell over. His prompt
action allowed Jason to get it going again quickly.
Forgive me if this is a naive question, but why do we
want each server
to be specialised?
We have been told that it would be best to do it that way. Also, we
are moving in the direction of a 3-server layout and then most likely
to a 4-server layout after that.
The 3-server layout is going to be like this --
First, larousse and pliny are going to be upgraded to much faster dual
CPU athlons with lots of RAM. They will be nearly identical hardware
after the upgrade. Both will be significantly faster.
Next, we are going to get a big honking database server, probably an
Opteron with as much RAM as we can afford. Base price of this will be
$6,000, and I'm going to beg for everyone to contribute money to help
me with this.
At that point, we'll have 2 webservers fronting for the db machine.
(I'll also loan an iptables based loadbalancing machine for the front
end if we want to do it that way.)
It is my belief, based on experience, that such a design will have
4-10 times the capacity of what we are doing now. Possibly more.
Even so, of course we want to be pro-active about the next step, and
that will likely be either a third webserver, if that's where the
tight spot seems to be at that time, or a second db server for
'difficult' queries, if that's where the tight spot seems to be at
that time. It's an empirical question, I think, which will be best at
that time.
Larousse is constantly maxed out for CPU. It loads
PHP, Apache, the PHP
scripts and a large portion of the cached HTML into memory, and it still
has plenty left over. It barely touches the disk.
The biggest CPU users on pliny are the apache subprocesses for
other-language wikis. MySQL needs all the memory it can get its hands on
(much of it in the form of buffers). Even with 2 GB (?) of memory, pliny
spends most of its time waiting for the hard drive.
Soon pliny and larousse will have roughly the same capabilities. So why
not give them roughly the same tasks? If we move the other-language
webserving to larousse, pliny's CPUs will have nothing left to do!
Well, that's a very good point. But Experts Have Offered Different
Opinions. :-) I'm not saying you're wrong, of course.
Probably the best performance would be obtained by
splitting the largest
tables of the database between larousse and pliny, although that's
technically difficult and not very robust.
Right. And we do need to consider the upgrade path as well.
The Opteron (or IA64, I suppose) machine will be purchased as soon as
possible. I've already contributed $2500 to get the Wikimedia
Foundation bank account underway. Various persons have contributed
just under $1000 so far, but we haven't really made a pitch for more
money. I think getting the rest will be pretty easy.
The biggest obstacle right now to getting that machine is not money,
per se, but rather my desire to wait _just a bit_ for the technology
to be more mature and widely available. Right now, finding Opteron
machines that accept SCSI drives is not impossible, but difficult, and
probably more expensive by a fair amount than it will be in a couple
of months.
This machine is going to be a big investment, so we want it to have a
clear upgrade path, which means we will want to be sure we can drop in
a faster pair of CPUs in the future. And we want to be sure that
there's plenty of slots for insane amounts of RAM once the 2 gig chips
fall in price to be the sweet spot.
--Jimbo