October 18, 2007
Reliability of Open Source Editing
According to a new report from Dartmouth College, "the beauty of open-source applications is that they are continually improved and updated by those who use them and care about them." But what about the quality and reliability of individuals who occassionally contribute to open source programs like Wikipedia?
Dartmouth researchers looked at the online encyclopedia Wikipedia to determine if the anonymous, infrequent contributors, the "Good Samaritans," are as reliable as the people who update constantly and have a reputation to maintain. Surprisingly, the answer is yes. The researchers discovered that Good Samaritans contribute high-quality content, as do the active, registered users. They examined Wikipedia authors and the quality of Wikipedia content as measured by how long and how much of it persisted before being changed or corrected.
In most studies, it is difficult to study the so called "good samaritans" because carefully designed studies don't allow for participation outside of the study. In this case, a real-life situation such as the online Wikipedia, allowed for such an evaluation since through the very nature of Wiki software, a history of every change is recorded.
Study
The researchers from set out to examine the reputation and reliability of contributors to Wikipedia. Wikipedia has an archive of the history of changes and edits to its entries, which allowed the researchers access to analyze the perceived quality of content.
By subdividing their analysis by registered versus anonymous contributors, the researchers found that among those who contribute often, registered users are more reliable. And they discovered that among those who contribute only a little, the anonymous users are more reliable. The researchers were most surprised to find that the reliability of Good Samaritans' contributions were at least as high as that of the more reputable registered users' contributions.
According to the authors, "wikipedia is a great example of how open-source contributions work for the greater good, and because it welcomes input from anyone, not just programmers and geeks, it is a great research tool. We can mine information from Wikipedia that helps us understand human behavior."
This study has been presented at academic conferences, and it is available online.
Posted by rsk at October 18, 2007 08:41 AM