[Labs-l] swift available from within labs

Thu Mar 29 23:03:52 UTC 2012

On Thu, Mar 29, 2012 at 2:49 PM, Platonides <platonides at gmail.com> wrote:

> On 28/03/12 19:28, Ben Hartshorne wrote:
> > Hi,
> >
> > I'm interested in testing the documentation I wrote on how to set up a
> > new swift clutser.  Would anybody be interested in being my guinea pig
> > and attempting to set up a second swift cluster in labs?
> >
> > If you find any problems with the docs, please feel free to update
> > them.  If you have questions, ping me (maplebed) in IRC
> (#wikimedia-labs).
> >
> > Thanks!
> >
> > -ben
> >
> > p.s.  Whether or not someone's interested in trying out the docs, I will
> > have to get 2-3 swift clusters running in labs for testing.  I will
> > probably do this late this week or early next week if someone doesn't
> > beat me to it.  The things that need to be tested:
> > * upgrade swift from 1.4.3 to 1.4.7
> > * set up inter-cluster replication
>
> What would that be? Create a new labs project, which also clones from
> upload.wikimedia.org?
> Is it supposed to reuse manifests/swift.pp ?
>

I think yes to both.  :)  I think the puppet changes necessary to set up a
new cluster are limited to creating the ring files (empty at first) and
adding a new class to manifests/role/swift.pp (copy/paste the pmtpa-labs
class and do string replacement appropriately).  This could, in theory,
work within the same project, but I think it'd be more interesting to make
it a separate project.  The backend should remain
upload.wikimedia.org(this is how the 404 handler fetches missing
content).  If you wanted to
test originals stored in swift (like Aaron's been doing), you might want to
set the thumbhost to something different to protect the full
medaiwiki/swift integration.

For inter-cluster replication, the thumbhost doesn't matter so much, since
it's more geared towards testing what happens to content once it's in swift
rather than how it gets into swift.  I'd leave it as
upload.wikimedia.orgjust to make it easy to get test data into swift.

For more context, check out some of the documentation on wikitech (starting
here: http://wikitech.wikimedia.org/view/Swift but of particular interest
for understanding how thumbhost works is
http://wikitech.wikimedia.org/view/Swift/Dev_Notes#All_Swift - that's the
state production is in currently).

-ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20120329/fb02ebe7/attachment.html>