Gregory Maxwell wrote:
On 9/1/06, Rob Church <robchur(a)gmail.com>
wrote:
[snip]
It's a casual rating system to give people a
bit of an idea as to how
good various people think a page is. Supplemental to stable
versioning, perhaps, but it still doesn't replace people checking
their own sources, etc.
Your point about feedback coming from anons is a good one, and quite
salient, since it's quite correct that most (and the best, from the
point of view of what our "audience" thinks) feedback will come from
viewers who might not have accounts.
Whatever is chosen in the implementation is going to be a toss-up,
because life is not perfect, people are not perfect, and shared IP
addressing is a sordid reality we have to put up with. :)
If it's just a casual rating system: don't make it a vote. Allow
everyone to provide input (believe it or not, we do get useful
complaints to OTRS from non-editors)... Collect everything, expose
what we've collected.... people can then write their own tools to do
things with the data.
The software should implement mechanism but not policy wherever possible.
Absolutely. There are all sorts of ways to address sockpuppetry, bogus
reviews, and ballot-stuffing.
For example, a Bayesian "rating the raters" approach might be useful,
given a small set of "ground truth" ratings. If this is designed right,
it should even be able to extract useful information from raters who are
actively trying to game the system, for example by performing honest
ratings of numerous other articles as well as plugging their favourites
or down-rating articles they disapprove of.
Yet another approach might be to offer a "rate a random article"
feature. This will both guarantee breadth of coverage from such ratings,
and also make ratings of random articles more difficult to ballot-stuff
by several orders of magnitude. (Note that calling up an article this
way, and then _not_ rating it, would also convey useful information).
-- Neil