[WikiEN-l] About the reliability of the Wikipedia process and content

Wed Nov 17 17:34:22 UTC 2004

I have been preoccupied with other things and have not followed the 
list-serve closely, so I apologize if I am missing something (and would 
love it if someone would bring me up to speed).  But I recently signed onto 
a "Forum for Encyclopedic Standards" project, and now see people discussing 
peer-review.   I'd like to point out that "peer-review" can take many 
different forms, and in fact has had different effects.

The form I am most familiar with is an article is sent to two to five 
reviewers who comment on and evaluate it, and the editor of a journal makes 
recommendations.  In some cases, the reviewers put considerable time and 
effort into commenting -- the result is something far beyond the casual 
hashing out of ideas at a bar or cafe.  In many of these cases I know I 
have benefited considerably and my article is ultimately much, much better 
for it.  In other cases, reviewers put little time into the job and often 
grind their ax (as some have pointed out) or just insist that the article 
under review cite ten more people (surely, friends of the reviewer).  In 
such cases the result is that those articles that are published are often 
mediocre, conform to some current fad, and all sound alike.

But surely, peer review means nothing more than making our words 
accountable to the judgement of our peers.  Broadly put, isn't this what 
Wikipedia is all about, and haven't we been doing this all along?

I agree that there is an issue of concern here.  I am currently involved in 
an edit war with a couple of users and at the heart of it, I think they 
have done no research or very sloppy research.  I have read too many 
articles where it seems that the primary contributors relied on their 
memories of what they learned ten years ago in a college class, or what 
they have picked up on other web-sites (and frankly, I think the 
information available on the web is mostly mediocre).  How do we deal with 
this?

Given my own commitments to Wikipedia, I am absolutely against a panel of 
people who are charged with approving or vetoing articles or edits.

I have two suggestions.  First, I think that we already have established 
values that address some of the problems I and others are concerned 
with.  I am referring specifically to the prohibition on original research, 
and the call for verifiability.  Most of the edit wars I have been involved 
with have involved either NPOV or one of these two values.  Frankly, I 
think that the Wikipedia community as a whole has focused too much on NPOV 
at the exclusion of these two other values.  I do not mean this to be a 
criticism.  I have seen considerable (and necessary, useful) debate over 
the past few years on what, exactly, NPOV is -- debate on talk pages, 
meta-Wikipedia, and on this listserve.  I think the community as a whole 
has developed a pretty sophisticated idea of NPOV and how it can be 
achieved.  I think most veterans are very sensitive about this, and do a 
pretty good job of policing articles for violations of NPOV, and educating 
newbies.  I just think we haven't done as good a job with 
"verifiability."  I think the explanation we have on the verifiability page 
is very good -- but I am talking about the culture of the place. I think it 
is time for people actively to look for opportunities -- on article talk 
pages, personal pages, and here -- to talk about "verifiability;" how do 
you recognize that an article is verifiable; what are the warning signs 
that it may not be; how to go about making it more encyclopedic.  I think 
the culture of Wikipedia is strong and when directed in a thoughtful way, a 
powerful tool.  I think if the majority of us just started to pay more 
attention to this issue on pages, and through debate and dialogue started 
evolving a more practical way to explain it to newbies and show people how 
to make articles more verifiable, we will not need a panel empowered to 
veto articles.

My second suggestion is to try to find a way to appropriate the one good 
thing I have found in peer-review -- when a few scholars who really are 
experts in a topic give considerable attention to one person's work, and 
give them focused feedback.  NOTE, the purpose is not to give a stamp of 
approval or to veto the work, the purpose is to engage the work on a very 
high level, to call attention to the elements of the work that can be 
developed, and to suggest alternatives.  I am trying to imagine a process 
that is more intense than what usually happens on a talk page.  What I 
suggest is that there be a directory of editors grouped by expertise on 
major topics (biology, art, US history, etc).  When it is clear that one 
editor is writing a new article or substantially rewriting or adding to an 
existing article, members of one of these panels can review the changes as 
a whole with an eye towards making constructive suggestions and educating 
the new editor in Wikipedia values.  There need be no threat of veto or 
sanction power -- I think if three veterans reviewed an article of mine and 
told me "look, deal with our criticisms now, and address these issues, or 
sometime over the next year other wikipedians will spot these problems and 
delete or change your work at will," it will be strong enough incentive.

These two ideas are off the top of my head and I am sure that even if you 
like them they need much more refinement.  I hope there are enough other 
people out there who get what I am suggesting, and are willing to improve 
on the ideas!

Steve

Steven L. Rubenstein
Associate Professor
Department of Sociology and Anthropology
Bentley Annex
Ohio University
Athens, Ohio 45701