On Thu, 17 Mar 2005 10:04:00 +0000, David Gerard <dgerard(a)gmail.com> wrote:
Indeed. keats appears to be starting from a personal
distaste and
then claiming this will be the destruction of Wikipedia. I note a
curious lack of substantiating, ahh, numbers. keats, do you have any?
First off, you can call me "Alex". That is my name. If you want to
argue on IRC call me keats or whatever you choose to call me. I am not
John Keats, nor am I a character in a book. A nick is chosen on IRC to
be distinct from others. It could be waerth, jwales, TimStarling, or
whatever. But, we are discussing this on a public mailing list, where
my name is quite apparent, and I have signed my emails as such. Let's
be adults about this.
Second, I do not have any numbers. I said that I felt that we had some
problems. I proposed some solutions, and gave my analysis of said
problems. The first step in solving a problem is to identify the
problem. Then you go and find what might be possible solutions (more
disk on the master db, squids on freebsd, postgres on the backend,
multiply redundant masters, colos on different continents, a BigIP or
two), and you /test them/. The fact that I haven't gone out and built
my own wikipedia cluster and tested every solution I offered is hardly
a fair criticism. I have at my disposal two powerbooks, two ibooks,
and a single 2x 866mhz Linux firewall. I don't have the resources to
do all this testing. Nobody just cut me a check for $86,000, either. I
am offering my time and my expertise, and even some money. But the
testing has to be done by the foundation.
Fundamentally, taking out the word
"fancruft": keats appears to be
claiming that the mere fact of having 500k articles is unsustainable
in MediaWiki. Is this the case? Do we stop all article creation now?
If not, what do we do? Ration them?
Do you not see that we are having weekly outages? So we hit 500k
articles and lo, it holds together. Where do we go from here? With our
one master database server. Somebody goes out and drops $100k on an
8-way 848 opteron. Somebody drops a further $100k on disk. Problem
solved until we hit 100M articles. Right, because every single power
ranger should have their own page, and every single villain, and every
single care bear, and every character from charmed, and every dicdef
that never makes it into the wiktionary, and so on and so forth.
You can't just say "let's just stick with what we're doing, it works
for now, and we'll just grow the architecture we've got by throwing
cubic dollars at it until it works properly."
You're wasting my and the foundation's money by doing so. FIx the
architecture and you reduce the cost of operation.
I mean, have you actually ever designed anything near as complicated
as the wikipedia? Have you actually ever been in a board room with the
program manager, project manager, VP, eight developers, and two
sysadmins when you all realize at the same time that the architecture
you've got just won't scale to the point you need it to?
I've been there. I can draw you two distinct pictures on the
whiteboard I mentioned in my previous email. One will be called
"horribly fucked" and the other will be called "ideal". We can get
from "horribly fucked" to "ideal" (that's what I do for a living),
but
we have to stop calling eachother names, and start figuring out what
we can do to fix the problem. Test. Benchmark. I mean, start doing
what it takes. Right now we're doing nothing. We're putting out fires.
Adding more servers is only going to help you until you reach some new
unsustainable point. The current architecture is (let me spell it out
real big for you)
U N S C A L A B L E
period.
--
Alex Avriette
avriette(a)gmail.com