On Wed, Aug 12, 2009 at 4:48 PM, dan nessett<dnessett(a)yahoo.com> wrote:
--- On Wed, 8/12/09, Roan Kattouw
<roan.kattouw(a)gmail.com> wrote:
I read this paragraph first, then read the
paragraph above
and
couldn't help saying "WHAT?!?". Using a huge set of pages
is a poor
replacement for decent tests.
I am not proposing that the CPRT be a substitute for "decent tests." We still
need a a good set of tests for the whole MW product (not just the parser). Nor would I
recommend making a change to the parser and then immediately running the CPRT. Any
developer that isn't masochistic would first run the existing parserTests and ensure
it passes. Then, you probably want to run the modified DumpHTML against a small random
selection of pages in the WP DB. Only if it passes those tests would you then run the CPRT
for final assurance.
The CPRT I am proposing is about as good a test of the parser that I can think of. If a
change to the parser passes it using the Wikipedia database (currently 5 GB), then I would
say for all practical purposes the changes made to the parser do not regress it.
Also, how would you handle
intentional
changes to the parser output, especially when they're
non-trivial?
I don't understand this point. Would you elaborate?
Dan
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
To elaborate on the final point. Sometimes the parser is changed and
it breaks output on purpose. Case in point was when Tim rewrote the
preprocessor. Some parts of syntax were intentionally changed. You'd
have to establish a new baseline for this new behavior at that point.
This also comes down to the fact that we don't have a formal grammar
for wikisyntax (basically it's whatever the Parser says it is at any given
time). This makes testing the parser hard--we can only give it input and
expected output, there's no standard to check against.
Finally, I don't think we need to dump all of enwiki. It can't require that
much content to describe the various combinations of wiki syntax...
-Chad