Well, the Sun patch didn't hold us. So we are going to do a gradual
move of directories, some data, and the actual thumbnail regeneration
over to another server, ms4. How much data we move depends on how fast
ms1 can push it over to ms4, currently very slow indeed. Even removing
files on ms1 is painfully slow. So...
In order to get ms1 out of its hole, we're turning off uploads once
again, so that it can free up space at a reasonable pace. Once we have
enough free space to make ZFS happy, we'll turn it them back on. We
expect that the rest of the migration of data and/or empty directories
should take place over the next several days without further outages.
Ariel
Στις 15-07-2009, ημέρα Τετ, και ώρα 14:28 -0700, ο/η Brion Vibber
έγραψε:
Brion Vibber wrote:
We're applying a Solaris kernel patch for the
ZFS performance problem on
ms1, our main media file server.
While in progress, uploads will be temporarily disabled. Cached images
should still display, but you may see some missing images in the meantime...
The kernel patch is applied, the machine's rebooted, and uploads & math
are re-enabled. So far all looks well; this'll keep things running
smoothly while we continue work on rearranging the files (moving thumbs
out to a separate server will simplify things and keep the main
filesystem backup snapshots from filling up with short-lived files, for
instance).
-- brion
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l