User:Cailil/Wikipedia studies literature review 1

This page comprises a literature review of peer reviewed analyses of English Wikipedia, its structures and how they impact on issues of editor conflict and editor retention.

This page will summarize and analyse these secondary sources about English Wikipedia. It will examine their findings, and will comment on how these findings relate to policy and to helping us identify ways to prevent, address or help resolve conflict, and to improve editor retention.

It aims to produce a report on the scholarly literature about wikipedia editor retention and suggest a series of aims to address the gender gap on English Wikipedia and help increase both new user retention and improve the over all quality of the user experience of English Wikipedia for all editors. It is hoped that these findings can be transferred to other language wikipedias as well as other wikimedia projects.

Studies of conflicts on wikipedia

Relationship of policy and editing practice

Community, Consensus, Coercion, Control

Community, Consensus, Coercion, Control: CS*W or How Policy Mediates Mass Participation
by Travis Kriplean, Ivan Beschastnikh, David W. McDonald, Scott A. Golder. 2007

This paper discusses how "when large groups cooperate, issues of conflict and control surface because of differences in perspective [Wikipedia..] has responded [to this] with an evolving body of policies that provide shared principles, processes, and strategies for collaboration. [This paper employs] a grounded approach to study a sample of active talk pages and examine how policies are employed as contributors work towards consensus.

This paper seeks to investigate how wikipedia policies are used in disputes and help work toward conflict resolution: "A body of policies is not a silver bullet for negotiation amongst diverse perspectives [... the authors find that] Contributors may interpret a situation differently and draw on different policies to substantiate their views." In this situation the paper examines how "complex power plays that contributors make to control content and coerce others during the consensus process."

Unfortunately other than identifying that "power plays" based upon policy, pre-existing consensus and what the paper describes as the "legitimacy of a user's contributions" (ie whether they are considered by others to be POV pushing or going against expertise in the topic area) the article does not come to any conclusions.

Creating, Destroying, and Restoring Value in Wikipedia

Creating, Destroying, and Restoring Value in Wikipedia
by Reid Priedhorsky, Jilin Chen, Shyong (Tony) K. Lam, Katherine Panciera, Loren Terveen, John Riedl. 2007

"Wikipedia is the great success story of collective action on the Web. It is also a source of wonder: its essential idea – that a useful encyclopedia can be created by allowing anyone to create and edit articles – seems absurd. Some people are ignorant, some are malicious, and some are just bad writers. Yet, Wikipedia seems to work."

This paper contends that: "frequent editors dominate what people see when they visit Wikipedia, and that this domination is increasing [... it shows] that the probability of a typical article view being damaged is small but increasing, and [the authors] present empirically grounded classes of damage. Finally, [they] make policy recommendations for Wikipedia and other wikis in light of these findings." The paper builds on other research estimates "the value of Wikipedia, based on how many people are affected by a change to an article." With this in mind they ask 3 questions:

1. Who contributes to Wikipedia’s value?: Other research has demonstrated (with sites like Usenet and MUDs that "a small minority of participants in an online group produce most of the content, while the vast majority of users produce little or no content". To this end this paper examined two things how long an edit persists for, and how many views it gets, in order to assess who contributes most (the uses the term "persistent word views" to indicate this). "This metric builds in an assumption of quality which is useful to helping improve the encyclopedia rather than simply measure flat statistics."
   The paper "analyzed 4.2 million editors and 58 million edits [and the] total number of persistent word views was 34 trillion." Using this methodology they found that "Editors who edit many times dominate what people see when they visit Wikipedia. The top 10% of editors by number of edits contributed 86% of the PWVs, and top 0.1% [of editors] contributed 44%."
2. What is the impact of damage such as nonsensical, offensive, or false content?: "In 2004, Víegas et al. published a seminal paper [... and identified] cases of conflict and vandalism. Focusing on one particular class of vandalism, mass delete, where all or nearly all of an article’s content is deleted, they found that this vandalism was repaired in a median time of 2.8 minutes." Priedhorsky et al found that while the overall impact of damage in Wikipedia "is low, it is rising. The appearance of vandalism-repair bots in early 2006 seems to have halted the exponential growth." However, and in line with the discussion of reverts in other studies listed here the author point out a serious problem with the misuse Wikipedia's revert/undo function: "it is common to find edit comments asserting vandalism repair on reverts of revisions which are apparently users practising editing on a regular article, something which is not welcome but is (by policy) explicitly not vandalism." In order to prevent this the author make a suggestion similar to flagged revisions - that edits are view by a certain number of human editors before being accepted.
3. What types of damage occur, and how often?: The authors identify that nonsense and page blanking are low level in terms of their actual damage to Wikipedia but that "offensive damage is troublesome because it is common (28% of incidents)" But notes that "misinformation may be the most pernicious form of damage" because 20% of incidents in their study and "difficult to detect" (requiring expert knowledge to see and correct.

This study is useful in that it shows that a low number of users are responsible for a huge amount of content on English Wikipedia and that there is a persistent misuse of "revert vandalism" on edits that are not vandalism.

The social dynamics of wikipedia

These two articles 'He Says, She Says' and 'Us vs Them', both by KIttur, Suh, Pendleton and Chi, examine ways in which to analyze and examine editor conflict on wikipedia and how that impacts on "indirect work", or work on this site that doesn't improve articles. Their findings could be used with more recent tools such as the WikiWar Monitor to help, identify and reduce conflict.

A key finding in both papers is that the more indirect work the more conflict. Furthermore, how editors perceive reverts (which in their papers is classified as another editor having a major issue with their edit) is a factor in conflict. This second finding is a key one for editor retention in that how new users perceive being reverted will have a direct impact on their experience of editing here, and the likelihood of them coming into conflict.

Both papers also analyze POV driven edit-wars using the Dokdo article as an example. This data could equally be used vis-a-vis any POV or nationalist or outside agenda driven conflict on this site. Their suggestion is that flooding these controversial areas with more neutral editors prevents intractable conflict. This finding is in line with wikipedia's polices on consensus and requests for outside comment.

He says she says

He Says, She Says: Conflict and Coordination in Wikipedia
by Aniket Kittur, Bongwon Suh, Bryan A. Pendleton, Ed H. Chi
This article examines the growth of "non-direct work and describes the development of tools to characterize conflict and coordination costs in Wikipedia"

This paper contends that "conflict in online communities is a complex phenomenon. Though often viewed in a negative context, it can also lead to positive benefits such as resolving disagreements, establishing consensus, clarifying issues, and strengthening common values". They assess the cost to wikipedia of conflict by examining what they describe as indirect work: "excess work in the system that does not directly lead to new article content" The paper maintains that "the growth of conflict and coordination costs in Wikipedia and the effectiveness of ways of combating it are important to the Wikipedia community as a whole to maintain the continued forward progress of the system." They find that "despite the overall growth of Wikipedia, the percentage of edits made to article pages has decreased over the years [...] from over 90% of all edits in 2001 to roughly 70% in July of 2006. Furthermore, the percentage of edits resulting in the creation of new pages has decreased to less than 10%" They offer one explanation: "that the maturation of the topic vocabulary in Wikipedia is making it more difficult to find new topics to write about and easier to add or change an existing topic"

Much of the rest of this article deals with how to train a computer model to find a controversial topic. One such method was to consider the number of article topic edits and another is to see how many times an editor has added the controversial tag. They also identified reverting as a major point of contention between editors and a source of conflict. This paper also uses the same case study (Dokdo detailed below in the next section) to illustrate the use of the revert function in user conflicts.

Besides finding that visual anaysis of conflict is of benefit to resolving it and fiving oustide users an objective analytical tool they find that

A future application of the conflict model developed here is to identify controversial articles before they have reached a critical conflict point. Our data suggest that an effective way to resolve conflict is to increase the number of users involved in editing the article, rather than have the same few people arguing back and forth. Even if a small percentage of users involved themselves in these pages they could prove vital to defusing conflict before it gets out of hand.

Simply put the more users involved and the less a controversial article is left to same set of editors the less likely it is to fall into intractable conflict. One point that we can extrapolate from this vis-a-vis editor retention is that if a user has a narrow contact group on wikipedia (ie is focussed on one topic with one group of editors where consensus is limited they are more likely to have negative experiences and be engaged in "indirect work" or in common internet speak internet dramahz.

Us vs Them

Us vs. Them: Understanding Social Dynamics in Wikipedia with Revert Graph Visualizations
by Bongwon Suh, Ed H. Chi, Bryan A. Pendleton, Aniket Kittur

"As in any large collaborative system, as Wikipedia has grown, conflicts and coordination costs have increased dramatically." This paper's authors developed a tool called Revert Graph "that visualizes the revert relationships between opinion groups" and that they believe that this stuy could help form the basis of future conflict resolution tools. This would be of great benefit to wikipedia as it could help mediators get an over-view of the dynamics of a negative relationship and help them assess how best to intervene.

The previous study by these authors ('He says, she says' above) "strongly hints that the degree of conflict and disagreement between users has a close relationship with the amount of [edits to do with communication and policy making]. This is a very interesting finding that indicates disagreement has less to do with Mainspace edits than edits elsewhere.

In this and their previous study Suh et al deal with how wikipedians feel about being reverted. They note that (at least up to 2007 when this study was published) "approximately 6.7% of all changes in Wikipedia goes to restoring articles to previous versions." The paper undertook three case studies:

1. Nationalist or POV driven reverters (my label not the authors'): Dokdo is a disputed islet in the Sea of Japan controlled by South Korea, but claimed by Japan as “Takeshima”. The study grouped users "based on their points of view as exhibited by their editing history". They fond that:
   Group A claimed Korean heritage on their user pages; supporting the Korean claims in discussions on the users’ talk pages, and preferred the term “East Sea” over “Sea of Japan”, as well as preferred “Dokdo” over “Takeshima” or “Liancourt Rocks”.
   Group B showed mixed opinion on the issue. Unfortunately for us this is total informtion they give on this group.
   Group C disputed the points of view of Group A; dispute the official U.S. position (which supports the Korean claim); openly refute the Korean point-of-view on their talk pages; as well as directly claim Japanese heritage or affiliation on their user pages.
   While this analysis can be dismissed as merely showing that users who espouse nationalist POVs on their pages are likely to engage in nationalist editing conflicts. It does at least show us hard evidence of that long understanding.
2. "Mediator editors": The paper argues that this type of user reverts on the basis of their belief that "those edits did not satisfy Wikipedia policies" and has a wider number of editors whom they have reverted, as opposed to the above who revert POVs contrary to their own.
3. "Controversial editors": the author describe this group of reverters as "usually self-appointed experts, or [users that] have strong points of view" these editors engage in very large numbers of reverts on a specific topic.

The paper's overall point is that visual analytic tools can help give wikipedians an over-view on a conflict. The weaknesses are that while true it does not assess the quality of the edits being reverted. However it does, as does it the authors' earlier article, show that reverting is a major part of this site's social dynamic and is a serious point of conflict between editors. As a note, this study examined edits from wikipedia 2006 - new figures on what percentage of wikipedia's overall history is made-up of reverts would be very interesting in 2012.

Gender gap

There is some great work being done here on Meta about the gender gap. What I'm summarizing here is the literature from outside wikimedia.

Lam et al find that female wikipedians are more likely to be reverted early in their wikipedia-career and more likely to leave due to that. They also debunk a number of assumptions about women and wikipedia show that women and men are equally blocked, that female topics are more contentious than male ones, and that considerable research into internet adoption by women shows that the gender gap on wikipedia should be shrinking, but is not. This part of their research that the gender gap remains about the same as the UNU-MERIT survey contradicts Antin et al whose paper did not take into account whether wikipedians in their survey left, a crucial element in Lam et al's paper and to English Wikipedia's efforts at editor retention.
They also found that once women got past the 500 edit mark they were just as likely to be reverted as men and that the difference in treatment (with regard to reverting) is limited to female wikipedians early edits. They also caution that simplistic attempts at solutions that lack a detailed understanding of the actual problem are likely to fail, as many assumptions about gender differences on wikipedia do not hold up to scrutiny.

Antin et al debunk a number of stereotypes about female wikipedians doing different work on the site but illustrate that they are under represented in the top 25% of editors by quantity in their sample of new users in 2009. Their paper points to this being a serious contributing factor to a systemic bias against women on English Wikipedia as that top 25% are the users who in effect govern the site and form consensus on its policies.

WP:Clubhouse?

WP:Clubhouse? An Exploration of Wikipedia’s Gender Imbalance
by Shyong (Tony) K. Lam, Anuradha Uduwage1, Zhenhua Dong, Shilad Sen, David R. Musicant, Loren Terveen, John Riedl.

This paper presents "a scientific exploration of the gender imbalance in the English Wikipedia’s population of editors. We look at the nature of the imbalance itself, its effects on the quality of the encyclopedia, and several conflict-related factors that may be contributing to the gender gap."

The paper begins by illustrating systemic bias on wikipedia ("Wikipedia’s coverage of topics like friendship bracelets or Sex and the City pales in comparison to that of toy soldiers or The Sopranos and how it has been treated in news media.") and notes that the "Wikimedia Foundation has established a goal of increasing the female share in editors to 25% by 2015." Lam et al argue that while Noam Cohen's New York Times Article "article presents a compelling argument" that "there is need for more rigorous analysis that expands on the reported survey results and anecdotal evidence." They believe this paper "represents a crucial next step in understanding the nature of the gender gap and deciding what should be done to address it."

The authors' note that research concludes that women are more likely to engage in volunteer activities than men. Indeed other research by Taniguchi and by Wilson suggests there should not be a gender gap. Wilson argues that women generally volunteer more because it is seen as part of women's social life Lam et al base their argument on the thesis that "Overall, these findings suggest that females may be more likely to volunteer their time to edit Wikipedia, though they may edit less if they lack a social connection to the Wikipedia community."

While there is mixed opinion and data on women's technological adoption the figures suggest to Lam et al that "Wikipedia, a community of volunteers collaborating to build an online encyclopedia, ought to have a reasonable gender balance." And they start out from the premise based on wider studies of the gender gap on the internet) that this gap should be shrinking. The paper sets out to examine three things:

1. "What is the extent of Wikipedia’s gender gap, and how has it changed over time?" The paper found that a) "females comprised 16.1% of the 38,497 editors who started editing Wikipedia during 2009"; b) "despite females being 16.1% of the new editors in 2009, they only accounted for 9.0% of edits made by this cohort of editors"; and c) "the gender gap widens when looking at editors with many edits, and does not appear to stabilize until the percentage of female editors drops to around 6% for editors making more than about 500 edits". They point out that: "Females stop editing Wikipedia sooner than males, and the ratio of males remaining to females remaining for this cohort increases steadily as time passes. Therefore, one of the factors contributing to Wikipedia’s gender gap is a lower retention rate for female editors compared to male editors. And although evidence of the gender gap shrinking on the rest of the internet exists this paper's statistical analysis shows that the gender gap has not been closing over time.
2. "How is Wikipedia affected by the gender gap?" Lam et al contend that "females will tend to be more active than males in social- or community-oriented areas of Wikipedia that offer increased interaction with other editors and opportunity to build interpersonal relationships. If this hypothesis is supported, addressing the gender gap might lead to a healthier community in which there are more resources available for community-oriented tasks like helping new editors and organizing editor efforts." This hypotheses is not without its own issues as it relies on stereotypes of female behaviour and gender roles. However the rest of their analysis proves the long held assumption that that there is systemic bias on English Wikipedia in favour of male-oriented topics, or as Lam et al state: "the gender gap appears to have a detrimental effect on content coverage of topics with particular interest to females."
3. "What gender differences exist in conflicts in Wikipedia, and how do those differences relate to the gender gap?" This paper contends that, a) "Females topics are statistically more contentious than male topics" (with 5% being protected vis-a-vis 2% of "male" topics); and b) "Females who do decide to edit English Wikipedia may find it difficult to make contributions that are accepted by the community" with their first 500 edits. They base this second contention on the work of CS Herring who argues that "females who participate in mixed-gender online environments tend to be marginalized."
   The authors contend that

   Furthermore, Herring notes that even if participant gender is not made salient, features of a participant’s discourse style can often reveal gender information. Recent research has shown that males and females have measurably different editing behaviors on English Wikipedia. Thus, we believe there may be a systemic bias against females that cause their edits to be more likely to be reverted (undone) by another editor, particularly early on in their Wikipedia tenure.(emphasis mine)

   
   Lam et al's data shows that "Female editors are more likely to have their early edits reverted (emphasis mine)" and are more likely to "leave Wikipedia when being reverted as newcomers". Hwever the paper is careful to qualify that this relate sto their first 500 edits. "Interestingly, beyond this initial handful of edits, we see little statistical difference between females and males in how often they are reverted. This suggests that females and males who manage to reach a modest level of Wikipedia experience are on par with each other with respect to community-perceived contribution quality."

The paper again like the social dynamics articles, makes some interesting notes on reversion: "making more edits during one’s first 24 hours as a Wikipedia editor is associated with a longer activity lifespan, while having one’s early edits reverted for any reason, vandalism-related or otherwise, is associated with a shorter lifespan." Again the revert function of English Wikipedia is seen as a crucial bone of contention and has a direct impact on editor retention.

Furthermore the paper makes an extremely salient point: "The problem [of gender imbalance on English Wikipedia] is subtle, and simple attempts at solution[s] without detailed understanding are likely to fail( emphasis mine)." They close by saying that their findings "hint at a culture that may be resistant to female participation" and call for more research and focus groups.

Gender Differences in Wikipedia Editing

Gender Differences in Wikipedia Editing
by Judd Antin, Raymond Yee, Coye Cheshire, Oded Nov.

The "2010 UNU-MERIT survey found evidence of a significant gender skew: fewer than 13% of Wikipedia contributors are women. However, the number of contributors is just one way to examine gender differences in contribution. In this paper we take a more fine-grained perspective by examining how much and what types of Wiki-work men and women tend to do."

This paper is part of the Wikipedia Progression of Participation project and focusses on on a sam- ple of users who have recently created Wikipedia accounts. Examining the activities that individuals undertake as they begin their time as Wikipedians is crucial for understanding initial gendered work differences." It analyses different roles that wikipedians fill, concentrating on how gendered they are. The authors suggest that generally speaking individual "editors may gravitate towards certain types of work — for example those types which a Wikipedian feels best suit her skills or those which she feels are most important. There is already evidence suggesting that at least some Wikipedians specialize around specific social roles." They also note that a more in-depth study of "gender in Wiki-work could also help to reveal the influences of 'occupational stereotypes.'”

In its analysis the paper found no evidence "that men and women are attracted to different types of editing work." "However, the analysis of revision size is another indication of gender differences in editing behavior. Notably, two areas of work in which women made significantly larger revisions involved creative production, synthesis, and reorganization of text." The paper however places a caveat on its findings in that it "sampled only new Wikipedians who began work relatively recently" and therefore they "cannot make claims about patterns over time [...] Despite these limitations, however, this analysis suggests that the story of Wikipedia’s 'Gender Gap' is perhaps not as straightforward as initial reports have suggested."

Similarly to previous studies, this paper finds that "male Wikipedia editors drastically outnumber female editors overall [...] While the number of female editors (18%) was slightly larger than the number found in the 2010 UNU-MERIT study (13%), women remain under-represented". However they complicate this data by showing that in the bottom 75% of wikipdians by quantity) in their study "men and women made similar numbers of revisions in nearly every category of Wiki-work." However, examining the top 25% of Wikipedians (by edit quantity) in their sample, they find "that the overall number of revisions is far more skewed: just 27% of revisions were completed by women."

This report gives no further suggestions on solutions and interestingly in light of the others makes no reference to how reverting effects user experience.

[Moreover,] women were better represented than the UNU-MERIT study would have led us to believe. It is impossible to tell why this is the case. Wikipedia’s efforts to attract more women may be bearing fruit, or there may be inaccuracies in prior studies. Several factors are likely at work. On the other hand, our results confirm that, compared to men, there are far fewer women editing Wikipedia. Furthermore, there was a particular gender skew in revision quantity among the Wikipedians who do most of the work."

This is the group that the authors point out contribute most to policy, administration and arbitration, and they suggest women need more representation in "but also [the] one which may be more difficult to break into".

Editor retention

Effects of reverts on editor retention

These papers by Halfaker and Riedl, with Kittur, and with Geiger and Morgan, examine how the revert function on Wikipedia impacts on new editor retention. These papers throw-up a series of interesting points about reversion: a) that it encourages editor loss, b) that it improves the quality of contribution of new editors who don't leave, c) that reversion by twinkle, huggle and rollback contributes to a community that has become unfriendly to newcomers, and d) that as a quality control mechanism reversion works.

Overall the papers point to both the positives and negatives of the revert process and how this is part of a wider more "calcified" site structure that has fundamentally altered how Wikipedia works for new editors: moving from a site that anybody can edit to one that requires editors familiarize themselves with the existing communities norms, rules and processes in order to edit.

Don't Bite the Newbies

'Don't Bite the Newbies: How reverts affect the quantity and quality of Wikipedia work'
By Aaron Halfaker, Aniket Kittur, and John Riedl.

This paper argues that although reverts "are important to maintaining the quality of Wiki- pedia" they are damaging to new editor retention. Like other papers above and below they identify reverting as a proximate cause of editor loss. However the piece identifies that being reverted improves the quality of new editors' contributions and argues that established editors need to"encourage the learning" effect of reverting.

The paper notes that reverted new comers are more likely to stop contributing. This finding correlates with much of the work done on the gender gap (see above). However the key finding of this paper in relation to all other work on editor loss due to reversion is that:

reverted editors were more likely to leave Wikipedia and that the ones who stay will become less active and less bold in their work, this result shows that they will increase their quality, and therefore their productivity enough to more than make up for the difference. However, highly productive editors do not benefit in the same way that less productive editors do from being reverted.

This suggests that the "learning effect" of reversion is not universally experienced and generally not experienced at all by newcomers who edit prolifically to begin with.

The authors also note that editors "change their communication patterns after being reverted." They note that Wikipedia’s BRD cycle "encourages article discussion as a reaction to being reverted, and discussion should help reverted editors learn how to improve their work." On the other hand, these editors decreased "their communications to other editors after being reverted." Teh authors note that this "may be a sign of withdrawal risk".

The paper recommends: a) supporting reverted users, b) encouraging the learning effect of reverts, c) reaching out to newcomers.

The Rise and Decline of an Open Collaboration System

The Rise and Decline of an Open Collaboration System: How Wikipedia’s reaction to popularity is causing its decline
By Aaron Halfaker, R. Stuart Geiger, Jonathan Morgan, and John Riedl.

This paper argues that "recent research has shown that the number of active contributors in Wikipedia has been declining steadily" since 2007. It presents data on how "changes the Wikipedia community made to manage quality and consistency in the face of a massive growth in participation have ironically crippled the very growth they were designed to manage." It focusses on how "the restrictiveness of the encyclopedia’s primary quality control mechanism" as well as the "algorithmic tools used to reject contributions [are ...] key causes of decreased newcomer retention". However it goes on to suggest that the site's "formal mechanisms for norm articulation are shown to have calcified against changes – especially changes proposed by newer editors".

Hafaker et al. summarise their paper here, but for the puroses of this literature review I'll examine its pertinent findings. They start by identifying four types of editors at wikipedia: 'vandals', 'bad faith' (not on wikipedia to help it), 'good faith' (try to edit productively but failing), & 'golden' (editing productively). The paper's study shows that the number of 'golden' editors coming to wikipedia has fallen dramatically since 2005[1]. It also shows an escalation of teh rejection of edits by 'good faith' editors[2]. The crux of this issue is that, as per Halfaker's previous study, although the informal peer review system for wikipedia's content is good at producing value it is bad at retaining editors.

This study again highlights the issue of reversion as a significant issue in causing editor loss., however it goes on to deepen this analysis by examining the de-personalization of editor welcoming systems. The authors point to the use of bots and algorithmic tools like rollback, twinkle and huggle. The paper notes that while these tools do a good job of improving article content and decreasing the amount of effort put into it by editors they also add to the depersonalized atmosphere of wikipedia by welcoming new editors with warnings.

Another paradox identified by this paper is that policies which began as informal in 2005 have become less negotiable due to a) the formalization of policy development in 2005, b) policies being entrenched in technology (i.e bots), and c) the declining participation of newcomers in policy areas leaving "senior Wikipedians [with more responsibility] for curating and interpreting community policy than ever before." This has driven newer editors into essay writing (a less formal and unenforceable area of the site).

The paper concludes that wikipedia has changed fundamentally from an encyclopedia that anyone can edit to any encyclopedia anyone who understands "the norms, socializes him or herself, dodges the impersonal wall of semi-automated rejection and still wants to voluntarily contribute his or her time and energy can edit.". However at the same time the paper acknowledges that the current peer reviewed system of quality control works. The authors note that editor loss has grown since polices have become established and/or "calcified". It notes that the control of content is an issue for all internet sites using crowd sourced content and very briefly points to studies on the methods of Youtube and Slashdot for discovering copyright infringement. Other than pointing out that designers and developers must become aware of how "the tools that social systems use for enforcement" both enable and restrict certain "kinds of social activities" the paper makes no recommendations.

A short coming of this piece is that it does not tease out the apparent paradox it sets up between the retention of quality content and numbers of editors. Nor does it examine why good faith editors' work is/was being reverted.