I have just posted a new draft RFC on changing the thumbnail storage
and caching pipeline for Wikimedia projects [0].
This RFC was started as part of the Multimedia tech debt project.
Aaron, Faidon and I would like to move discussion forward on the issue
of the thumbnail caching layer in general and specifically would like
to find consensus on a reasonable change to address some of the
shortfalls of the current system in place for
upload.wikimedia.org and
related wikis.
The core problem has been discussed on list previously [1], namely
that scaled media files (aka thumbnails) take up a lot of space in the
storage servers. This RFC proposes following a course of action
proposed in that prior discussion [2] to customize the Varnish hashing
configuration for thumbs such that a single HTCP purge would
invalidate all size variations for a given file. It attempts to
document some of the benefits and drawbacks of that proposal as well
as present a few related variations.
Any and all feedback from this group would be greatly appreciated in
vetting the core idea and determining reasonable next steps towards a
viable long term solution.
[0]:
https://www.mediawiki.org/wiki/Requests_for_comment/Simplify_Thumbnail_Cache
[1]:
http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not…
[2]:
http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not…
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID
irc: bd808 v:415.839.6885 x6855