[Wikipedia-l] Re: Trust metrics

Sat Feb 14 22:41:57 UTC 2004

Jimmy Wales wrote:

>I am not advocating anything in this post, I'm just sharing some of my
>thoughts over the past few days.
>
I'm glad you're not advocating anything, and I trust that constitutes 
permission to shoot these ideas down before they get off the ground 
(just kidding - I'm glad to see it articulated, although I disagree with 
the approach).

Anyway, I'm not sure the eBay model improves much on what we have now, 
or even on Slashdot karma. It's fairly simplified, and the reason it 
works reasonably well is because it addresses a simple question - "If I 
make a contract with this person I've never met, can he/she be relied on 
to send me money or merchandise?" That's a straightforward question, and 
eBay feedback provides just enough information to make that decision.

Our user reputation issues, much like Slashdot, are more complex, and I 
don't think they should be approached by implementing metrics. If 
Slashdot's system is broken, I would venture that it's partly because 
they oversimplified it. The simplicity of voting somebody's feedback up 
or down is appealing, but leaves behind little useful information for 
others.

I also don't think the eBay model translates to Wikipedia. There's less 
incentive to give positive feedback, and a bigger crowd willing to give 
negative feedback. Also, without transaction-based limitations, you have 
less restriction on the negativity. I would expect many users that you 
or I might consider very positive contributors to have *negative* 
reputations, unless we set the zero-point so low as to make the system 
meaningless. I consider this probable even if we devise a way to weight 
the scores. But basically, I have concerns about generating any kind of 
reputation score, because it provides no context for the feedback that 
went into it.

>Some possible downsides, and there are many...
>
>2.  People might be dissuaded from taking controversial and brave
>stands, if it's going to get them some negative feedback.
>  
>
I think this is a huge problem for any kind of formalized feedback 
system, especially given the purposes of Wikipedia. Wikipedia could turn 
into a place where the content is largely a tasteless, watery gruel 
because everyone is on pins and needles to avoid provoking negative 
responses. In my opinion, using oversimplified feedback is a terrible 
way to promote NPOV. Neutrality can be achieved by addressing 
controversial issues and finding a balanced way to present them; it 
can't be achieved by avoiding the controversy altogether.

I don't believe the system is broken. At most, maybe we need a little 
more informational material in the Wikipedia namespace. Specifically, 
something that presents ideas of how to find out information about any 
given user, so people can assess that user's reputation for themselves. 
We have user pages and user talk pages. People can check the links to a 
user page for posts the user has signed. People can check user 
contributions, and a few diff-checks of a user's more substantial 
contributions can give anyone a perspective on the quality of that 
user's edits. It's a little work, but if knowing more about another user 
is worth it to you, it should be worth putting forth the effort.

I also think even an experiment can be dangerous, because really, 
there's no such thing as an experiment here. The only way to set one up 
is by getting the community involved with it, and nobody will 
participate if they don't see the results. Once they see results, some 
people will insist on using those results, even if we *know* that the 
information is misleading. And the only way to stop the experiment would 
be for Jimbo to make a very divisive decision to stop it. As a 
prediction, I believe an experiment would end up as one of two things: 
largely unutilized, like WikiMoney accounts, or dysfunctional like Slashdot.

--Michael Snow