[QA] bugzilla and needs-test

Wed Jan 15 23:07:53 UTC 2014

In a previous position where I also used Bugzilla, the solution that we 
came up with was simply to add a new custom field to Bugzilla, which I 
think might have had the label "Test Status".

That field was implemented as a drop-down, and the possible values were 
something like the following:

  * Blank (default)
  * Test needed
  * Test not needed
  * Test created

The idea was that whoever RESOLVED the bug was responsible for setting 
the "Test Status" flag, and then whoever was responsible for adding 
tests for the (Product + Component) in question could monitor the "Test 
needed" items via a saved search.

Once the test was in place, the status was changed to "Test created" and 
(if the tester was a Good Person), a comment was added to the bug with a 
link to any relevant test automation, manual test cases, etc.

Our workflow included the extra step of changing the bug status from 
RESOLVED to CLOSED when the test(s) were created, but I don't think we 
use the CLOSED status in the WMF's Bugzilla instance.

It's not an elegant solution, but it's very lightweight and it worked 
well enough.

- Jeff

On 1/15/14, 2:25 PM, Greg Grossmeier wrote:
> Observation 1:
> We have test-writing related keywords in Bugzilla:
> * need-integration-test
> * need-parsertest
> * need-unittest
>
> Obvservation 2:
> Without hard data to back up my claim, I feel like a large number of
> bugs reported are fixed but do not also include a test to the
> functionality. I might just be missing them. I don't think I am (at
> least always); I think they just aren't created. That's kind of the
> standard way of developing, right? ;)
>
> Suggestion:
> We use some kind of way of tracking this. Maybe not every bug fix needs
> an associated test, but I'd like to believe it is more than the
> ridiculously low percentage number that we are doing now. We might not
> use all of those current keywords (maybe just "needs-test" and let the
> dev team/QA person on point for that team figure it out). We can figure
> that part out later.
>
> Thoughts?
> Is this too cumbersome? Does anyone have experience with trying to
> accomplish something similar with a different group of engineers (both
> "dev" and "QA", where/if those are separate roles). We don't really have
> the bandwidth to start this in earnest right now, but should we start
> mulling it over?
>
> Greg
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140115/9a3d694b/attachment.html>