[QA] Forget developers in test, we need testers in development

Thu Jan 2 15:50:33 UTC 2014

Hi Nik,

apologies for the late reply, I was on vacation. Comments are inline.

On Sat, Dec 21, 2013 at 8:18 PM, Nikolas Everett <neverett at wikimedia.org>wrote:

> 1.  What do you do with developers that think it is beneath them to write
> integration tests?  Unit tests?
>

I would also ask why would a developer think that. All code is created
equal, right? Writing test code should be not different than writing
production code. Or am I wrong?

> 2.  Where should you store those tests?  (We've mostly settled on next to
> the production code which seems like the right answer to me for lots of
> reasons.)
>

+1

>  2.  What code/features/changes should have unit tests?  Integration
> tests?  What is the difference, any way?
>

Looks like you have two questions with number 2. :)

As a rule of thumb, code that is more likely to break, that is hard to
understand, that is written recently... should have more tests than other
code.

I would say that most (if not all code) should have unit tests, and things
that are hard or impossible to test with an unit test should be tested on a
higher level.

The difference?

"A test is not a unit test if:

- It talks to the database
- It communicates across the network
- It touches the file system
- It can't run at the same time as any of your other unit tests
- You have to do special things to your environment (such as editing config
files) to run it."[1]

> 3.  How do you make sure you stick to #3?
>

Well, this is #3, but I guess you were asking about previous question. I am
not sure what to do.

> 4.  Where does documentation live?  Do you build documentation from
> tests?  Do you build tests from documentation?  (Which one of these do we
> do or is it even listed?)
>

I have been struggling with this myself. Feature files are one form of
documentation.

>  5.  If integration testing is really just another speciality like
> databases what does this mean for people that have spent years doing one or
> the other?
>

I am not sure what you ask here.

> 6.  What do you do when your organization doesn't jib with this reality?
> (WMF has a QA team when it might make more sense to matrix integration test
> specialists into teams to teach them how to write integration tests, for
> example.)
>

We are moving towards integrating QA team into other teams. The problem is
that there is just a few of us and a lot of teams.

>  7.  Holy cow how does this all line up with volunteers who only want to
> do one thing or the other and we really should take any help we can get.
>

I do not think it will be trivial, but I am pretty sure we can handle
volunteers that just want to test or just want to code. :)

> 8.  Who pays attention to build failures and what do they do about them?
>

A lot of people actually. Mobile team, language team, QA team... When a
build fails, we fix it.

>  9.  What system, exactly, are those build failures testing?
>

A build can fail for several reasons. We had an entire workshop about it[2].

> Now that I think about it, who is coming to the architecture summit?
> These kinds of questions would be pretty interesting to talk about and
> might deserve a (late) RFC.
>

I am not sure if anybody from QA team is coming.

Željko
--
1: http://www.artima.com/weblogs/viewpost.jsp?thread=126923
2: https://www.youtube.com/watch?v=fs2VmlQ_h7Q
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140102/fb9fd985/attachment.html>