[QA] run browser tests against production Wikipedia?

Thu Sep 19 20:04:53 UTC 2013

I see no reason why we shouldn't have some kind of early warning
system.  It would even be possible to ping some talk pages hidden off
in the ether from time to time.

We certainly did this at Lulu.  If this search ever failed we would
sound the claxons:
http://www.lulu.com/shop/search.ep?type=&keyWords=test+book+xyzzu&x=0&y=0&sitesearch=lulu.com&q=

We also bought a book every five minutes from off site with a fake but
valid credit card.  Our system ate the order right before sending it
to the credit card processor so we could be pretty sure that most of
the chain was still working.

That feels like the right level of paranoia.

Nik

On Thu, Sep 19, 2013 at 3:57 PM, Chris McMahon <cmcmahon at wikimedia.org> wrote:
>
> Open question:  is it worthwhile to run read-only browser tests against the
> production wiki, e.g. https://en.wikipedia.org?
>
> At one point we thought a couple of sanity checks would be valuable in
> production, but since then our deployment process that updates test2wiki
> weekly may be be adequate for the purpose.
>
> Tests currently running in prod:
>
> PDF
> Print export menu
> PageTriage
> Preferences
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>