Lee Daniel Crocker <lee(a)piclab.com> said:
I just did an ad-hoc benchmark on Piclab of the same
installation
with and without link-checking, and on the limited set of pages
used by the test suite, the speedup was only about 3%. Of course
all benchmarks on single-servers may be less applicable to the
multiple-server installation we're going to have soon.
Was this a representative setup (e.g., with the full database)?
If so, that sounds like removing link-checks won't help
much - at least by itself and for single servers.
Thanks for the info!!
As I said, measuring is the only real answer, and I'm willing to
accept "nope, wrong guess" as the answer.
So turning to other ideas and performance measurements...
The swapping overhead does suggest that an excessive use of memory
by MySQL is resulting in the performance hit.
Perhaps reducing the amount of data
stored in MySQL (by moving the text of cur and old text into
the filesystem and OUT of MySQL entirely).
As another poster noted, the "old" text is
an especially large database, but presumably only a few rare
articles are actually read from "old".
Obviously reading an article from the filesystem will read the
data into memory too, but the underlying OS doesn't try to preload
the entire filesystem into memory, which MySQL appears to be
trying to do. Doing this will mean that archiving the encyclopedia
would have to archive both the MySQL metadata and the
article filesystem, but archiving a filesystem is a rather
well-understood problem :-).