Hello.
I've been following the discussions on the problems with the web
servers. It seems that there are too many concurrent processes for the
limited hardware we are currently using. I believe I read numbers
somewhere of a steady load of 20+ processes in the run queue on one of
the servers. Of course there will be a lot of context switching going
on, with some overhead. But worse, it will guarantee that all processes
get the worst possible total running time.
Example: n processes arrive at the same time. If the necessary CPU time
for every process is t seconds, all processes will finish after t*n
seconds in a concurrent system.
If, on the other hand, the processes run completely serialized, one
after another, the first process will finish after t seconds, the second
one after 2*t seconds and so on up to n*t seconds for the n:th process.
This gives an average running time of t*n/2 seconds, with a worst case
time of t*n.
In other words, I think we could gain a significant performance gain by
reducing the number of concurrent processes. I admit I have no idea how
this works today, if there is such a limit in place already. Nor do I
know whether this can be easily solved with some apache or PHP directive
(although I find it very likely).
With this in mind, you can have a look at the attached code, which
should illustrate the concept. It can surely be optimized, I don't need
pointed out what kind of atrocity it is to implement counting semaphores
by using MySQL named locks. Nevertheless, it was quickly implemented and
works quite well. Also, any overhead from implementation inefficiencies
is too small to be easily measured on my not-too-powerful PC. If it has
the predicted effect, any overhead would probably be neglible compared
to the gains.
Should anyone care to test this out on a live wiki (I haven't got a
clone running, so it's untested in that aspect), it's quite easy. Just
copy the two semaphore functions somewhere where they are included, and
then edit wiki.phtml so that it calls the functions before and after
main work, as well as starting and flushing the buffer. Like so:
ob_start();
wait_for_semaphore();
// Page generation - wikicode parsing
release_semaphore();
ob_flush();
Yours,
E23, swedish wikipedia.