[Foundation-l] Ensuring veracity of articles based on print sources

Sat Oct 7 17:56:07 UTC 2006

Ray Saintonge wrote:
> Andrew Gray wrote:
> 
> 
> 
>>Fundamentally, use of an offline (or subscription, etc) source is a
>>good and sensible thing, but it requires a modicum of trust that we're
>>getting a reliable link between the page and the information quoted;
>>we can't get around this by preparing lists of reliable and unreliable
>>texts, we can only get around this by someone "trusted" saying yes,
>>I've looked at that, it's there.
>>
> 
> Everything should be checked and re-checked independently, but that's 
> only an ideal.  We're already having difficulties getting software that 
> gives us a stable version that has only been checked for common 
> vandalism.  In time we should go much further than that, and allow 
> statistically based algorithms that will give a measure of probably 
> accuracy based on the review of multiple readers.
> 

Yes, a blind-vote citation-checking system in which aggregate results 
are captured.  Once nice property of such a system is that the votes of 
good-faith, competent citation-checkers will correlate strongly with one 
another, while the votes of bad-faith and/or incompetent checkers will 
have a basically random distribution.  This, of course, assumes no 
widespread collusion among checkers, but in most cases such collusion 
will be more trouble than its worth.  In addition, it would be possible 
to seed the citations shown to checkers (even on an individual basis) 
with random false citations which they would be expected to flag as 
incorrect/fabricated.  Access to open bibliographic catalogs would allow 
for the creation of completely random but quite legitimate-seeming 
citations, as it is only a matter a randomly picking a work returned by 
querying on the article's main subjects.

I think it might also be useful to use the results of citation-checking 
as a feed into some sort of trust ecology.  Fact-checking is mostly 
tedious, unrewarding work, and so the users who have shown themselves to 
be competent and reliable at it are probably going to be trustworthy or 
at least good-faith in other areas as well.  This would of course not be 
the only input to a user's "trust rating", but probably one of the more 
significant ones.