On Wed, 25 Feb 2004, Nick Reinking wrote:
Sorry to be a balloon burster. ;) Do we have any
recent profiling on
the code that shows where the bottleneck is?
E23 allegedly has been working on profiling code, but I've not played with
it in a long time so can't say what results look like.
Last time we did profiling, the Prime #1 Enemy was
OutputPage::replaceInternalLinks(). The rest of the page parsing was zippy
fast even with a hojillion passes.
This requires splitting up the page, identifying links, normalizing
titles, checking lookup tables for page existence, and generation of
output links. Make it fast and you win a prize!
I imagine most of the time
is spent poking about in the database. If that truly is the case,
developer time would best be spent trying to maybe optimize the database
layout or adding support for a faster database...
That's what the next revision will concentrate on: cleaning up the schema,
moving the large data blobs out of the most-queried tables, etc.
Note that page parsing time is largely separate from (though can be slowed
down by locking or disk churning due to) database slowness. But an
overloaded database does slow everything down and makes people
very unhappy.
-- brion vibber (brion @
pobox.com)