[Labs-l] A (21) day in the Labs

Sun Apr 28 14:28:54 UTC 2013

"Marc A. Pelletier" <marc at uberbox.org> wrote:

>> I'm all for the "lazy sysadmin" paradigm, but I think that
>> shouldn't preclude usable databases.  River's trainwreck is
>> freely available
>> (https://svn.wikimedia.org/svnroot/mediawiki/trunk/tools/trainwreck)
>> and open source, and the effort to port it to Ubuntu and set
>> it up is a valuable investment (or the setup of any other
>> replication engine that provides the *needed* functionality;
>> many-to-one/many isn't something only Tool Labs requires).

> There's no reason that can't be looked into, and it seems like a good
> idea if we can swing it.

> Another possibility, given suitably designed queries, would be to add
> federated tables to the appropriate places.

I don't know how they are implemented in MySQL, but commonly
there is no real caching which means that not only likely
millions of rows have to be passed through the network per
query, but that this is repeated again and again for every
query.  It's not a question whether "you" can swing it - the
Toolserver (and other DBs) demonstrates that it can be done.

> That said, the priority is to first get sanitized replication /up/, in a
> form that is suitable for extant tools.  Having iterative improvements
> after that is definitely a Good Thing.

The problem there is the snail's speed at which this trek is
moving.  Replicated databases were originally scheduled for
February/March, and the allocated time for that project was
*very* generous (several months IIRC).  If we need to wait
another, say, six months till we have a working environment
for non-trivial stuff, it'll kill any excitement there might
be about Tool Labs.  Having the financial and organizational
force of WMF behind it should have accelerated the process.

Tim