I have no argument with your points. However, figuring out the code coverage of
parserTests is low hanging fruit, i.e., relatively easy to determine and at least somewhat
valuable. By knowing which files are touched by parserTests and how much of their code is
covered is a first step in figuring out what needs to be done (e.g., identifying those
files that parserTests doesn't even visit). After all, parserTests is all we have at
the moment.
--- On Thu, 7/30/09, Happy-melon <happy-melon(a)live.com> wrote:
From: Happy-melon <happy-melon(a)live.com>
Subject: Re: [Wikitech-l] parserTests code coverage statistics
To: wikitech-l(a)lists.wikimedia.org
Date: Thursday, July 30, 2009, 4:26 AM
I suspect that much of this has to do
with the way Parser.php is eleven
thousand lines of programmatic sewage, and the way the
ParserTests
infrastructure requires a lot of the rest of MediaWiki to
be initialised in
order to run the tests. As long as the rest of the
system 'works' well
enough to allow the parser to parse, ParserTests is happy:
I wouldn't be
confident to say that it "tests" anything outside of
Parser.php; as you say,
it only marks lines 'visited'.
--HM
"dan nessett" <dnessett(a)yahoo.com>
wrote in message
news:922893.25678.qm@web32507.mail.mud.yahoo.com...
I failed to mention that xdebug ignores non-executable
lines of code. So,
the statistics are for executable lines of code
and do
not include lines
like comments (in either the covered or uncovered
counts).
--- On Wed, 7/29/09, dan nessett <dnessett(a)yahoo.com>
wrote:
> From: dan nessett <dnessett(a)yahoo.com>
> Subject: [Wikitech-l] parserTests code coverage
statistics
> To: wikitech-l(a)lists.wikimedia.org
> Date: Wednesday, July 29, 2009, 4:36 PM
>
> I decided to investigate how well parserTests
exercises the
> MW code. So, I threw together a couple of
MacGyver
tools
> that use xdebug's code coverage
capability and
analyzed the
> results. The results are very, very
preliminary,
but I
> thought I would get them out so others can
look
them over.
> In the next couple of days I hope to post
more
detailed
> results and the tools themselves on the
Mediawiki
wiki. (If
> someone could tell me the appropriate page to
use
that would
> be useful. Otherwise, I will just create a
page in
my own
> namespace).
>
> The statistics (again very preliminary) are:
>
> Number of files exercised: 141 Number of
lines in
> those files: 85606
> Lines covered: 59489 Lines not covered:
26117
> Percentage covered: 0.694916244188
>
> So, parserTests is getting (at best) about 70%
code
> coverage. This is better than I expected, but
still it means
> parserTests does not test 26117 lines of
code.
What I mean
> by "at best" is xdebug just notes
whether a line
of code is
> visited. It doesn't do any logic analysis
on which
branches
> are taken. Furthermore, parserTests may not
visit
some files
> that are critical to the operation of the MW
software.
> Obviously, xdebug can only gather statistics
on
visited
> files.
>
> I want to emphasize that there may be errors in
these
> results due to bad assumptions on my part or
bad
coding.
However,
it is a place to start.
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
=
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l