[QA] Weekend testing americas on September 7th

Tue Aug 20 20:01:43 UTC 2013

On 08/16/2013 07:41 AM, Justin Rohrman wrote:
> This sounds like a great idea to me. The timing would certainly be tight
> but I think with the VMs and Vagrant usage we should be good. I'll ping
> Michael and see what he thinks.

It is a great idea and it is technically ambitious. Participants must 
understand that they need to do some homework before the weekend. It is 
all well documented (and I will go through the process & docs again 
before the end of this month) but it takes time and good bandwidth to 
get your environment downloaded and installed.

>
>
> Date: Thu, 15 Aug 2013 14:12:08 -0700
>
>     From: Chris McMahon <cmcmahon at wikimedia.org
>     <mailto:cmcmahon at wikimedia.org>>
>     To: "QA (software quality assurance) for Wikimedia projects."
>              <qa at lists.wikimedia.org <mailto:qa at lists.wikimedia.org>>
>     Subject: Re: [QA] Weekend testing americas on September 7th
>     Message-ID:
>
>     <CAJohBHTurqx7G5EhK1Q2Ant+gTFZP4gPvqJ3XjzH2_x2fskRZg at mail.gmail.com
>     <mailto:CAJohBHTurqx7G5EhK1Q2Ant%2BgTFZP4gPvqJ3XjzH2_x2fskRZg at mail.gmail.com>>
>     Content-Type: text/plain; charset="iso-8859-1"
>
>     On Thu, Aug 15, 2013 at 6:43 AM, Justin Rohrman <rohrmanj at gmail.com
>     <mailto:rohrmanj at gmail.com>> wrote:
>
>      > Let's hear it :)
>      >
>
>     Here goes...
>
>     There is a certain tension in the testing community between
>     advocates for
>     UI/browser test automation and advocates for human "sapient"
>     testing.  I'd
>     like to go there.
>
>     Specifically, I'd like to do this with Weekend Testing:
>
>     * Pick a complex feature for which browser test automation exists.
>       Right
>     now I'm thinking CirrusSearch or VisualEditor.  Because we're using
>     Cucumber, the test Scenarios should be understandable by people who
>     are not
>     programmers.  Because we're using the page_object Ruby gem, the guts
>     of the
>     tests should be readable by people with programming experience in just
>     about any language.   Regardless of level of expertise, we should be
>     able
>     to provide everyone a local test environment from which to run the
>     automated tests against WMF hosts either natively or with a
>     VirtualBox VM
>     configured via vagrant (http://www.mediawiki.org/wiki/Vagrant). We have
>     done this before:
>
>     online: https://www.mediawiki.org/wiki/Meetings/2013-07-18
>     live: https://www.mediawiki.org/wiki/Meetings/2013-06-27
>
>     Given a robust set of automated browser tests and the ability to run
>     them,
>     let us then:
>
>     * Identify deficiencies in the automated test coverage for the feature.
>       This may or may not include analysis of the test code itself, but
>     would
>     certainly at least include analysis of the ATDD-style stated feature
>     coverage in the Cucumber Scenarios.  Do we have any technical debt
>     in our
>     test code?
>
>     * Identify test charters for which automated testing is not possible and
>     which are only testable by actual human beings.  Are there tests that
>     cannot be automated, and is such testing worthwhile?
>
>     Outcomes:
>
>     * Participants will be able to analyze ATDD-style automated browser
>     tests
>     * Participants will be able to run Cucumber + page_object browser
>     tests and
>     analyze the results of those tests
>     * Participants will be able to demonstrate automated browser test
>     practices
>     with examples from the open WMF browser test code
>     * Participants will be able to begin to contribute to WMF testing
>     efforts
>     if they wish, whether automated or not
>
>     This might be too ambitious.  I'm pretty sure this would be the most
>     technically challenging session in the history of WTA.  OTOH, we've
>     already
>     built the infrastructure to do this kind of thing, let's spread the word
>     about what is possible.
>
>     -Chris
>
>
>
>      >
>      > Message: 2
>      >> Date: Wed, 14 Aug 2013 08:56:54 -0700
>      >> From: Chris McMahon <cmcmahon at wikimedia.org
>     <mailto:cmcmahon at wikimedia.org>>
>      >> To: "QA (software quality assurance) for Wikimedia projects."
>      >>
>      >>         <qa at lists.wikimedia.org <mailto:qa at lists.wikimedia.org>>
>      >> Subject: Re: [QA] Weekend testing americas on September 7th
>      >> Message-ID:
>      >>         <
>      >>
>     CAJohBHQzWK5H9weGS-SERHe0faZRSS08WjiGw-UNb7ABe-xC0w at mail.gmail.com
>     <mailto:CAJohBHQzWK5H9weGS-SERHe0faZRSS08WjiGw-UNb7ABe-xC0w at mail.gmail.com>>
>      >> Content-Type: text/plain; charset="iso-8859-1"
>      >>
>      >>
>      >> I actually have an idea that I think would be of interest to WTA
>     and of
>      >> benefit to WMF, but I'd like to encourage others to reply first..
>      >>
>      >> -C
>      >>
>      >>
>      >>
>      >>
>      > _______________________________________________
>      > QA mailing list
>      > QA at lists.wikimedia.org <mailto:QA at lists.wikimedia.org>
>      > https://lists.wikimedia.org/mailman/listinfo/qa
>      >
>      >
>     -------------- next part --------------
>     An HTML attachment was scrubbed...
>     URL:
>     <http://lists.wikimedia.org/pipermail/qa/attachments/20130815/e6688311/attachment-0001.html>
>
>     ------------------------------
>
>     _______________________________________________
>     QA mailing list
>     QA at lists.wikimedia.org <mailto:QA at lists.wikimedia.org>
>     https://lists.wikimedia.org/mailman/listinfo/qa
>
>
>     End of QA Digest, Vol 4, Issue 13
>     *********************************
>
>
>
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>

-- 
Quim Gil
Technical Contributor Coordinator @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil