Introduction

Oh, well, I’ll probably look at government sites first. ‘Cause I know I can trust those.

Anne, high school junior

Collaborative production of online content has contributed to a popular epistemic crisis: If anyone can contribute to a news site, an encyclopedia, or a journal, how do people judge the quality of information? Credibility is a cultural construct that helps explain why people trust, propagate and use information (Metzger 2007; Rieh and Danielson 2007; Rieh et al. 2010). In this paper, I argue that assessing the credibility of information sources is central to the processes of scientific and knowledge-building discourse that underlie many CSCL approaches and theories. Making credibility judgments and coming to a shared understanding of what constitutes credibility is an important aspect of knowledge construction in small groups, classrooms, and in online public venues.

Clay Shirky has observed that when people are looking for credible information sources, many “trust new classes of aggregators and filters, whether Google or Twitter or Wikipedia” (Shirky 2009). He called this new form of authority “algorithmic,” that is, trusting a process instead of a person. In fact, assessing information sources has always implicitly involved trusting the processes by which they are produced. An article in the New England Journal of Medicine by a Harvard graduate may be considered trustworthy because the institutions involved represent strong processes of publication and education. The name of the journal and the university are widely understood credibility cues that influence perception of the information they accompany. Much research on “web credibility” has focused on identifying the kind of cues that people look for on the web to judge the reliability of sources (Agosto 2002; Flanagin and Metzger 2007; Fogg et al. 2001, 2003; Kafai and Bates 1997; Kuiper et al. 2005; Metzger and Flanagin 2008; Sundar and Nass 2001) More recently, researchers have examined how people judge the credibility of information in the context of open collaboration projects like Wikipedia. These are information sources that are not as broadly understood as traditional publications and may require different kinds of cues since readers can contribute to them as well as read them (Forte et al. 2014; Menchen-Trevino and Hargittai 2011; St. Jean et al. 2011; Yang et al. 2013).

Novel information production processes are a ubiquitous part of life for learners who use the Internet; moreover, many information sources that learners encounter online are a product of open collaboration systems in which students themselves may choose to participate. If these new processes are not well understood, how do students and teachers assess their products and negotiate their use in schools? If students can contribute to information sources they use, what role might producing information play in classrooms? In a 2-year series of studies, I introduced new wiki publishing tools in two American high schools and investigated the intersection of information consumption and production inside high school science classrooms.

Open collaboration and learning

Open collaboration systems underlie many distributed information production projects such as Wikipedia, OpenStreetMap (Budhathoki and Haythornthwaite 2012), Cyclopath (Panciera et al. 2011), or FindaGrave (Willever-Farr and Forte 2014). The proliferation of collaboratively produced, openly accessible media has altered the context of information production and access dramatically within the past decade. Young people in the United States (Jenkins 2006), Japan (Ito 2008), across Europe (Livingstone 2002), and in other industrialized nations are growing up in a media-rich culture of participation. Youth are poised to contribute to the production of public information sources by tweeting, blogging, reviewing products, rating news articles, and collaborating on encyclopedia articles. Yet they seldom are encouraged to practice these activities with guidance from teachers.

Forte and Lampe define open collaboration as happening in

…an online environment that (a) supports the collective production of an artifact (b) through a technologically mediated collaboration platform (c) that presents a low barrier to entry and exit and (d) supports the emergence of persistent but malleable social structures. Combined, these characteristics yield complex socio-technical systems that offer new opportunities for people to form ties with others and create things together (Forte and Lampe 2013).

This definition of open collaboration harmonizes with conceptions of online learning that gave rise to flagship CSCL projects like Knowledge Forum (Scardamalia and Bereiter 2006) or WISE (Slotta and Linn 2009) and with project-based and constructionist approaches to learning that involve the construction of complex, personally meaningful public artifacts. Kimmerle, Cress and colleagues have studied collaboration on wikis as a process of knowledge construction and have examined the interplay between individual and collective learning through carefully controlled experimentation (Kimmerle et al. 2011). Peters and Slotta observed that negotiating content on wikis involves the kind of peer exchanges that learning scientists encourage in collaborative classroom work (Peters and Slotta 2010). Similarly, using the term “mass collaboration,” Halatchliyski et al. have argued that the discursive processes that accompany article composition on Wikipedia are tantamount to knowledge building discourse (Halatchliyski et al. 2014). Yet, despite attention to the learning potential of discourse underlying information production, there has been little attention paid to how information is selected, vetted, and verified by learners.

Collective cognitive responsibility and second-order learning environments

An important feature of successful open collaboration projects is shared responsibility among participants for building and improving artifacts online. This responsibility is akin to the concept of collective cognitive responsibility in the knowledge building literature, which means that responsibility for the “success of a group effort is distributed across all the members rather than being concentrated in the leader” (Scardamalia 2002). Moreover, Scardamalia and Bereiter have used the nomenclature of first- and second-order learning environments to describe the difference between traditional schools and knowledge building communities (Scardamalia and Bereiter 1996). They explain that first-order learning environments involve learning a set of pre-defined skills and becoming integrated into a set of stable practices and routines. In second-order environments, one learner’s participation changes the environment in which other learners operate; thus, learning becomes a process of continual adaptation. Knowledge building literature and literature on open collaboration echo each other in their emphasis on the need for constant adaptation and the distributed nature of control and responsibility as ad hoc collaborative groups form, set goals, and manage themselves to create things together (Benkler 2006; Zhu et al. 2012). For example, research on the emergence of collective cognitive responsibility in knowledge building classrooms emphasizes learners’ responsibility not only for their own contributions but for advancing the knowledge of the group while research on Wikipedia has demonstrated editors of the site develop a sense of responsibility that goes beyond their own interests to considering the quality of the site as a whole and the collective needs of the encyclopedia-writing community (Bryant et al. 2005).

Open collaboration and knowledge building both require participants to A. know how to make valued contributions and B. assume responsibility for actively building shared knowledge in the world. I contend that information literacy skills such as finding, assessing, and using information to support claims are critical skills for making valued contributions in knowledge building communities (A) and that participating in open collaboration projects to create public resources can provide a context in which students take responsibility for practicing these skills together (B).

In other analyses, I have presented findings about student learning, collaboration, writing, and adaptation of genres as high school and college students produce online information resources as part of their classes (see (Forte and Bruckman 2006; Forte and Bruckman 2007; Forte and Bruckman 2009)). In this paper, I build on these analyses to specifically examine information literacy skills associated with assessing and producing information in open collaboration projects. Peppler and Kafai have noted that, although producing media has historically been marginalized in favor of critical analysis (consuming media), in recent decades the blurring of producer and consumer roles has been widely recognized and production has become a more visible feature of media education (Peppler and Kafai 2007). I am interested in the intersection of these two concerns:

  1. R1

    How do high school students understand and assess information sources, particularly when they are products of open collaboration?

  2. R2

    How does participating in the production of information sources interact with this understanding?

Studying the new information literate: Methods, sites and participants

To understand the interplay of production and consumption in high school students’ academic information practices, I conducted field studies in two suburban American high schools over the course of 2 years.1 Students at these schools participated in an online project called Science Online (See Fig. 1), where they contributed science articles on a public wiki, which was specially designed to support classroom writing projects. The first study lasted approximately 8 months and included 19 student participants and their teacher. The second lasted approximately 3 months and included 14 students and a teacher.

Fig. 1
figure 1

Editing the science online wiki: inserting a citation

Full size image

Field sites and participants

Because of the flexibility and commitment that is necessary for teachers to use an experimental research prototype with their students, recruitment of teachers who were willing to deploy experimental software in their classes was a challenge. To support the design of wiki software, I conducted an exploratory study of science teachers’ use of communication technologies in the classroom. Participants in the study who reported agile use of new technologies were approached about revising their classes to integrate prototype wiki tools into their assignments. One teacher, a former scientist, agreed to redesign his advanced placement environmental science class and experimented over the course of the school year with seven wiki writing assignments that varied in terms of length, collaboration, specific editing and sourcing requirements. The wiki assignments were interleaved with in-class activities like presentations and discussions and included writing about ecosystems and habitats, endangered species, collaborating on pages about organic processes, and publishing an individual research project. The student participants were juniors and seniors; 8 male and 11 female. All members of the class participated in the assignments; 75 % of the class participated in the study. The first study took place in a suburban public school ranked among the top in its district with a graduation rate of 95 %. Ninety five percent of those graduates pursued further education.

Through close collaboration with the first teacher, I gained access to science teachers at other schools who were also willing to adapt their classes to an experimental technology. The second study took place in a honors organic and biochemistry class at a private suburban high school with 100 % graduation and college enrollment rates. The teacher was also a former scientist. Students in her class developed a section of the wiki on human diseases; each student worked on a page for a different disease as a final project that lasted for several weeks at the end of the school year. Again, all the students were juniors and seniors; two male and 12 female. 100 % of the class participated in the second study.

Data collection

In both field studies, I used a mixed methods approach to collect data about students’ information use and strategies for completing eight different assignments, which are described in more detail in (Citations removed for blind review). These assignments were diverse and included both collaborative and non-collaborative work, but all involved students contributing to a public information resource. The empirical anchors for this paper are 57 interviews with students that were conducted over the course of the two study iterations and approximately 200 hours of classroom observations. In-class observations as well as quantitative data about students’ editing activities on the site enriched my understanding of students’ and teachers’ experiences and helped to reliably interpret interviewees’ accounts. Because students typically work on school assignments in a variety of places and at different times, researchers were often not present. To capture salient features of their process, the interviewer asked students to reconstruct activities several times during the studies, resulting in 1 to 3 interviews per student. Examining wiki editing histories together with the students afforded a form of stimulated recall (Ericsson and Simon 1993) and ensured their recollections and reflections in interviews were based on their actual editing activity as recorded in the system.

Data analysis

Interview data were analyzed using a grounded theory approach, in which researchers use iterative rounds of data collection and coding in order to work from empirical data to construct theoretical explanations of the phenomenon under investigation (Glaser and Strauss 1967). I chose this method because “the result of grounded theory is an explanation of a set of practices” (Grinter 1998, p. 395) and I wanted to understand from the students’ perspective how they make sense of participatory media and where their experiences constructing information for others overlap with their efforts to assess information they find.

Grounded theory building is not a step-by-step procedure that is applied to a dataset; rather it is a way of thinking rigorously about data that yields new explanations for the phenomenon under investigation (Muller and Kogan 2010). The analysis and coding process gives rise to insights that can then be tested though comparison with more data, often collected after analysis begins during a process known as “theoretical sampling.” In this project, coding began in the first study iteration and continued throughout the data collection process and analysis in the second iteration, focusing on features of student practice that appeared promising for theoretical development.

Findings: First and second order information literacy practices

The goal of understanding the interplay between consumption and production sensitized me to themes in the data that pertain to these two aspects of student work. Researchers cannot approach data with an empty mind, we enter into inductive analysis with a conceptual vocabulary; thus, it is important to articulate sensitizing concepts (Blumer 1954). In this case: critical consumption and information production. Close reading of the initial dataset yielded broad categories that described aspects of students’ creative process (for example, “reflection on audience” or “use of wiki tools”) and aspects of their information seeking and assessment habits (for example, “criteria for clicking on search result” and “criteria for citing a source”). In subsequent rounds of coding for the second dataset, the categories that described information assessment strategies were refined and arranged in a shallow taxonomy, during a process known as “axial coding.” The taxonomy was developed by comparing specific categories to each other in order to identify salient characteristics and relationships between them in a process known as axial coding (for example, “expert author” and “cited other sources” are two specific examples of “criteria for citing a source”). Aspects of students’ processes for constructing their wiki articles were coded together with assessment strategies. Examining consumption and creation in tandem gave rise to insights about their interrelations and potential dependencies.

Finally, in a stage known as “selective coding,” I selected two concepts as strong organizing principles around which other emergent concepts could be arranged and explained. In grounded theory, these are the one or two core ideas around which a theoretical explanation of activity is forged. Initially, I loosely thought of these concepts as information assessment strategies that were either “by the book” or “creative”; however, as I refined the emergent conceptual vocabulary, I began to connect these strategies to relevant concepts from the education literature and called them first-order and second-order strategies. Remember that first-order learning environments involve learning a set of pre-defined skills and becoming integrated into a set of relatively stable practices and routines while in second-order environments, one learner’s participation changes the environment in which other learners operate; thus, learning becomes a process of continuous adaptation (Scardamalia and Bereiter 1996). By the same token, when engaging with participatory information sources and new publishing paradigms, students are confronted with a perpetually changing information landscape that they themselves transform through their participation in online spaces. Throughout the next sections, I demonstrate how understanding and reflecting on the process of information production (second-order skills) become important features of information literacy as learners cope with new forms of publication. Practicing and adopting a set of accepted standards or heuristics for identifying “good” sources (first-order) is insufficient when there are no well-established expert practices. Moreover, I demonstrate how participating in the process of information production—at times in a true collaborative fashion, but often by individually contributing to a collective resource—began to provide students with a model from which to reason about novel information sources that they encounter online.

Findings: Students’ assessment practices

Let us now turn our attention to the questions with which I began:

  1. R1

    How do high school students understand and assess information sources, particularly when they are products of open collaboration?

  2. R2

    How does participating in the production of information sources interact with this understanding?

Students primarily related sources to their work in terms of relevance, access, and credibility. In this section, I examine their use of both first- and second-order strategies to assess resources along each of these three dimensions.

Relevance

Whether or not a source contains information relevant to the task at hand is a basic criterion for establishing its value. Students’ strategies for assessing the information they encountered included dimensions of utility/relevance—did the sources have information they needed presented in a way they could understand and use?

When demonstrating their searches, participants exhibited a great deal of confidence in their search strategies. Some reported educating themselves about appropriate terms using sources like Wikipedia. In general, if they did not find what they needed, they often assumed that there must be a better place to look for information instead of assuming their queries were unsuitable. For example, if they were unable to find something they wanted using Google, some students reported that they would likely have better results in a specialized database.

When they needed to choose from among many possible sources based on relevance, first-order literacy skills dominated. For example, when confronted with a list of search results, students frequently used Google page rank as a proxy for relevance:

When I went online I googled [my topic] and it was the first website that came up… and since that was the first thing that came up it’s obviously relevant. – Paige

[viewing Google search results] … I always look towards the top because I feel these are probably the most relevant ones. I don’t know, I’ve been told that’s how they organize it. – Ella

These students suggested that Google ranks pages by relevance and applied page rank as a heuristic, although it is unclear what their model of Google’s page ranking algorithm is and none were able to explain it. Adopting the page-rank heuristic uncritically is a first-order strategy. Proprietary or complex algorithms can exacerbate this “black box” effect. The process by which information sources come to exhibit a characteristic (in this case, Google page rank) is obscure, which inhibits students (and other users) from productively reflecting on the meaning of page rank and engaging in second-order information literacy strategies. Second-order strategies are not ruled out as especially conscientious information seekers may attempt to deduce the meaning of “relevance” from their experiences with Google or by researching the algorithm; however, I saw no such attempts to do so by participants.

Access

Access played an important role in determining which information sources students found useful: if they can’t access it, they can’t use it. Most students were able to find a large number of sources and convenience played a role in determining which ones students used in their work. Ease of access was a recurrent theme as students described how they searched for information.

Like with the articles we’re finding, it’s so easy we can find those online cause otherwise we’d have to go to some special access thing. You know what I mean? I mean, it’s just so much more convenient. – Erin

Both teachers obtained passwords from local universities to allow their students to use scientific resources they normally would not be able to access. The overhead involved in accessing “good” sources included not only learning the different databases, but also obtaining privileged information from authority figures that allowed them to gain access on the sly.

As they accessed potential media and information sources, intellectual property also became a salient feature of source utility. Some students described looking for media that was licensed for reuse. This underscores the importance of media production as a component of information literacy education. Although many students were not meticulous about ensuring the licensing of imagery they used, some explained that they tended to use government sites and Wikipedia as a resource for imagery in particular, because these sources offered artwork and photographs they were allowed to reuse. Media production provided a reason for students to reflect on intellectual property. I observed classroom discussions about IP that resulted from students’ need to mix media. Although students often adopted a first-order approach to assessing the legality of using sources (legal codes are not as easily reinterpreted as fluid social constructs like credibility), discussions sometimes surfaced second-order reflections by students as they made note of why different kinds of sites adopt different licenses and noticed the service they were performing by making information behind a pay wall more widely accessible to others by referencing it in a freely licensed online resource.

Credibility

How students assess credibility is bound to their models of how and why media is produced and published. When students received in-class instruction about assessing sources, they were told to look for traditional markers of credibility like respected institutions and authors; naturally, these turned up frequently as they explained their search and assessment activities. Twelve of the 33 participants mentioned authorial credential as an important credibility cue and ten mentioned the publication type (peer-reviewed journal). Although many students commented that they were taught not to trust Wikipedia because “anyone can edit,” five out of 33 participants remarked on the number of editors as a regulatory feature of Wikipedia’s information production process. Large numbers of contributors is perhaps the most salient characteristic of new publication processes that have been made possible by participatory forms of media. Despite the fact that this was not part of their instruction in the classes I observed, some students have begun to reflect and learn about these processes.

These assessment strategies reveal glimpses of students’ models of publication. Different understandings of how information production is regulated give rise to different standards for evaluating credibility. For example, as I have noted, in the traditional models of publication that students learned about in the classroom, authors wield institutional accreditation like degrees and positions as markers of expertise that lend authority to their publications. In traditional scientific publication, an elite group of experts act to achieve editorial consensus about what ideas will be propagated by their community. In this “gate keeper” model of publication, it makes sense to use authors’ accreditation and/or employer as markers of a documents’ reliability. Most students reported using cues consistent with traditional publishing models, such as authors’ credentials and publisher, to identify expert-produced content.

Like government sites mostly. .gov .edu Something that seems like it’s a knowledgeable person publishing. Not just like, a normal mind. – Susan

I don’t know if the person that’s writing [Wikipedia] is, you know like I said, a third grader or if it’s an actual scientist with a doctorate, so it’s—to me Wikipedia still doesn’t have as much credibility as other websites could. – Carrie

You want to get the people who are scientists and really know what they’re talking about. You know, there’s no-one, there’s no extreme genius checking Wikipedia and making sure everything’s right about it. You know, it’s people doing it. – Jaime [emphasis her own]

These comments ascribe an almost chimerical authority to experts, who are contrasted with the “normal” people one might encounter online.

Part of the difficulty in assessing the reliability of documents returned by a Google search is the potentially dramatic heterogeneity of publication models that gave rise to the individual sources. In particular, students tended to dichotomize Wikipedia and scientific journals; however, few students had clear ideas about how Wikipedia is produced. When probed, most students struggled to respond. A handful suggested that information in Wikipedia was subject to oversight and some applied traditional heuristics like expert authorship to justify their sense that it could be a reliable source:

For some reason with Wikipedia I always imagine that it’s like really educated people writing these articles and I know it’s probably not. – Amanda

Like in Wikipedia and scientific journals the wonderful thing is, you get all this peer review from others who are at the top of their fields so if something’s wrong, they’re going to know that because they have that independent knowledge. – Reagan

In the cases above, the students describe some possible heuristics for deciding whether to trust Wikipedia content based, again, on authorial credential; however, these heuristics are applied globally, to the site as a whole. In the case of Wikipedia, where content quality varies across and even within articles, conclusions about content reliability need to be made locally, based on local indicators that the content has been groomed and is under surveillance (Suh et al. 2008). Many students misconstrued the mechanisms by which Wikipedia operates by attributing oversight to a central editorial authority; however, a few students explained that they believed Wikipedia has a process for ensuring quality that is distinct from traditional publishing mechanisms, represented by this reflection:

Wikipedia, it’s like, known that it possibly could be wrong but so many people read it that they can correct it that it kind of like checks, it’s like checks and balances. – Alli

These students noted the distribution of responsibility among not only authority figures, but anyone who reads the site. By using their understanding of its architecture, a few students were able to draw conclusions about how the site’s content is maintained. In this dataset, second-order reasoning about the site as a whole never translated to more sophisticated strategies for assessing specific Wikipedia content.

Findings: The influence of context and task on assessment strategies

The rules governing text production in academic environments set school writing apart from other kinds of writing. Students write to meet a set of explicit grading criteria that will be applied by a teacher who is thought to already know about what the students have written. Teachers gave students in our studies instructions about where they could find information sources. Beyond these directions, students often used past experiences with classroom rules and academic norms as guidelines for finding and using information. The academic environment as a venue for enculturation into scholarly norms can be said to encourage first-order skills development as students become familiar with expert practice in a wide array of disciplines. Some students reported that they invoked their teacher or turned to an authority figure to validate their assessments:

If it didn’t look like I could open the web page and have Dr. Baker look at it, I wouldn’t use it just because I felt like if it didn’t look legit then it probably shouldn’t be used.

INT: So, when you say open up the website and have Dr. Baker look at it?

Like if I had been in front of her and she was reading it, would I think that she would think it was okay? – Hans

I could not find anything on PubMed, and I was really struggling, and the librarian and I - she is, like, one of my good friends in the library, so she always helps me out - and she goes, ‘Well, can you use this web site?’ I said, ‘I have no idea… and then I actually caught Dr. Baker on a Tuesday going to pick up her kid, and she said that was fine, so I just kept going, going, and going. – Jerry

Interviews provided a glimpse of a larger set of rules to which these students have been exposed. Data from several students indicate that they have been influenced by an abstinence approach to new media sources like Wikipedia. For these students, following the rules meant avoiding Wikipedia altogether. Other students reported that their teachers encouraged them to read Wikipedia but not cite it and still others were instructed that they could only use it if they cite it. Overall, students received unclear signals in school about what constitutes “expert practice” when it comes to assessing participatory information sources, or sources created through the participation of many individuals using a collaborative platform. It could be argued that there isn’t any well-established, first-order expert practice to which their teachers can expose them.

I saw some evidence that students who had deeper understandings of how Wikipedia functions were able to use that knowledge to guide their assessment of the site as an information resource. Participants also modified their use of information sources as they imagined and met the needs of a broader readership on the Internet. Because they were providing a public resource, they were concerned about the responsibility they had as authors. For these students, finding “good” sources was not only a requirement; it also played a role in supporting the legitimacy of their writing as a public information resource. In some cases, when students reflected aloud on their information seeking activities, writing on a public wiki invited comparison with Wikipedia:

I just would not go Wikipedia first, because it is an edited source. It is just like I am writing my own, so people are writing their own thing on that. – Diane 2

Well, I think that Wikipedia is more established [than our site] since it’s got more, there are more people on Wikipedia, I think I would probably trust it more than SciOnline.– Paige

I think anyone can edit Wikipedia just like anyone—well anyone can edit our wiki thing. – Susan

These students leveraged their own experiences to reflect on sources they encounter online and vice versa. Participation became a foothold for reflecting on the system. These impromptu comparisons made during interviews suggest that with guidance in the classroom, students might use experiences contributing to online environments as a window to more profound insight about the nature of information production in online environments.

Findings: Taking responsibility for information on the internet

In the above sections, I have examined information assessment strategies that students use as they help create a collaborative online information resource and how participation in information production can influence these strategies. I began with the propositions not only that learning how to assess and produce information are important skills; but that learning these skills by participating in open collaboration projects can support the development of collective cognitive responsibility in groups. As they discussed these contributions, many students remarked on the sense of responsibility the experience evoked.

Producing an information resource provided a context for writing that made students feel as though their work mattered. Most of the students reflected in interviews that they felt a sense of responsibility toward their readership and that they felt differently about a homework assignment that could be read by others than about other writing assignments.

Makes me feel like really important and, like, I don’t know. It would be like kind of cool like if someone, like, cited me. – April

It’s like ‘oh my gosh, I have a huge responsibility now’ even if nobody actually uses this. It’s still there, somebody could use it so everything has to be exactly right and I want to put as much information on here as possible. – Reagan

These students suggest not only that writing for a non-teacher audience is motivating, but also that creating a public information resource is associated with a sense of responsibility that promotes critical engagement with information:

I mean like anybody can access it. So it puts more pressure on you to make sure that what you’re putting on the web is true and accurate information because other people could be reading it and could access that for their website or something like that. – Sara

I think if it’s going to be so open for other people to use, your work should be credible. You just don’t want to lead people wrong. – Jerry

This sense of responsibility provides an ideal context for practicing information literacy skills like identifying information needs, searching for, and assessing information sources. Providing a resource for others, in other words, addresses the challenge raised by Metzger (2007) of motivating information assessment in the first place. One particularly reflective student remarked that publishing her own resource involved even further reaching responsibility because:

It could be like an upward spiral if everyone has true information, then everybody will keep getting true information, but if one person has false information and the next person uses that, uses that-then you just have a downward spiral and no-one really knows what’s true – Sara

This student suggests not only that she has a responsibility for her own work but that users bear a collective responsibility for the quality of shared information and views her participation online as part of a larger system of information production. Although most students’ reflections indicated they experienced responsibility for creating good information, interviews generally did not provide evidence of collective cognitive responsibility.

Discussion: The New information literate, informed and informing others

In the preceding sections, I presented findings that students used both first- and second-order strategies to assess information and demonstrated how the classroom context and information production task influenced these strategies. First-order strategies were exemplified by practices like drawing on academic norms from other classes and invoking the teacher as an expert authority figure. Second-order strategies involved the creative construction of standards based on students’ understandings of how the information they found was produced and disseminated, exemplified by reflections on how peer review happens in open collaboration systems and the rationale behind intellectual property and accessibility of scientific materials. The classroom and the wiki supplied students with contexts that supported the use of both types of assessment skills. Our findings are congruent with those of (St. Jean et al. 2011) in their study of content creators’ assessment of credibility in which they developed a similar conceptual vocabulary of “heuristic” and “strategic” evaluation.

The limitations of these studies should be understood. My approach was interpretive and involved spending hundreds of hours with students and teachers to understand how they make sense of their own experiences and practices. My explanations of information literacy skills and strategies, then, are grounded in students’ own meaning making. The products of interpretive research are best framed as empirically driven hypotheses, the goal of these methods is to contribute ecologically valid explanations but this should not be confused with experimental approaches that seek to prove causal relationships. By providing examples of data and connecting findings to others’ where possible, I aim to strengthen confidence in these interpretations. Additionally, one implication of using students’ reconstructed activities is that our results focused on students’ interpretations of their tasks rather than on their actual activities. Without intending to, students may misrepresent their processes because they remember the parts that were most difficult, interesting, or aligned with their self-image. To avoid relying solely on memories, I used students’ editing activity on the wiki as an object of reflection to ground interviews in what they actually did on the site.

Looking forward, the findings discussed in this section have implications for educators and researchers of information literacy: First, access and open licensing are critical features of media for the new information literate. Second, production should be viewed as a core aspect of information literacy. Third, participation in open collaboration projects can engender new kinds of responsibility in students; however, guidance is necessary to realize its full potential.

Open access

In teaching how to assess information sources, traditional economic models of publishing are often taken for granted. New economic models create new targets for education. For example, commons-based peer production as described by Benkler is a form of production that relies on a workforce able to recognize opportunities for creation and, as discussed, adapt quickly to new working groups (Benkler 2006). Wikipedia exemplifies peer production of encyclopedic content; other examples include open source software and collaborative systems for aggregating and evaluating news, product reviews, etc. Without radical shifts in the ways that teachers, librarians and other mentors understand the ways that information is produced in these new contexts, students will remain underprepared to understand and contribute to (and within) such new models of production. Furthermore, there is a strong connection between peer production as a new economic model, the free culture movement/intellectual property law reform, and education. New forms of production signal not only a need for new competencies, but also a need for access to media and permission to reuse it as would-be student producers look for raw materials. Intellectual property, licensing, and open access become salient issues for students as they search for information, imagery and sounds to appropriate in their own work. In our studies, lack of access was a barrier for secondary school educators who hoped to groom scientifically literate students in their classrooms. Students were forced to gain access to needed materials and databases on the sly by using local university resources.

Larry Lessig noted a decade ago that “[w]e are well on our way to perfecting the “Read-Only” internet – that network in which every bit of culture can be bought in a single click, but bought with the rights to consume only” (Lessig 2005). For the new information literate, participation implies creation and consumption; this underscores a need for information resources that students can reappropriate and remix to build new media creations as they practice these skills in schools.

Production

In 2010, Pew Internet and American Life reported that about 40 % of teens were creating novel content online (Lenhart et al. 2010) and others remix and share content in other ways. Online content creation and collaboration are becoming regular features of information practices. This trend is inspirational from the perspective of educators for whom media creation represents a powerful context for learning, but it also signals a gap in the literature. Information literacy has been primarily defined as a set of skills related to finding, evaluating, and using information. Less attention has been paid to the role of information literacy skills in collaboratively creating information resources for others or how to guide young people in becoming adept contributors to knowledge building that happens in the wilds of the participatory web.

Scholars and educators have suggested that young people in particular require better skills for assessing information as resources become less mediated by authority figures and the “parental controls come off” (Grudin 2007). The studies presented in this paper suggest that producing information for others in online environments can give young people a starting point for reflecting on where information comes from; such experiences support second-order information literacy skills, which require students to reflect on the nature of information production. To help engender this kind of reflection, information literacy standards should explicitly include producing information for others.

Beyond production: Taking responsibility for content online

As noted early in this paper, participating in open collaboration projects entails more than simply adding content, contributors must learn what kinds of contributions are valued and begin to take responsibility for advancing the work of the collective. Collaborating on a Wikipedia article, for example, involves coming to a shared understanding of some phenomenon with co-authors and respecting editorial convention. We have seen that although students in these studies took responsibility for the potential impact of their work on others, they did not generally adopt a collective sense of responsibility for advancing the understanding of the group or improving the resource as a whole. However, the data suggest that the experience did influence participants’ understanding of their school work and, in some cases, brought about reflection on the ways information is produced and their own role in improving the quality of information on the Internet. Many of them described a sense of heightened responsibility because they understood that their work could serve as a resource for others.

As hoped, this sense of responsibility provided a context that motivated students to care about the quality of information sources they used and about citing them accurately; however, as in many other studies, a deeply-entrenched individualistic classroom culture often prevailed over collectivism. Based on these findings, I suggest that the experience of responsibility for contributing to knowledge building can be unpacked into different dimensions:

  • Shared/Individual

  • Local/Global

First, there is the sense of shared responsibility for advancing the state of knowledge in a group that is demonstrated in knowedge-building classrooms and some Wikipedia articles as well as the sense of individual responsibility for making good contributions to a collective. Second, there is the local context in which one can view collective efforts as a contribution to a community of peers as well as a global context in which one can view collective efforts to create knowledge as a contribution to a social good that may or may not be used by known peers in a perceivable way. Whereas the classroom wiki activities did not engender a shared sense of responsibility in the local context of the classroom, it did engender a sense of individual responsibility at a more global level.

I suggest that exposing students to the demands of publication for a real audience provided an important move toward local, shared collective cognitive responsibility by decentering students and placing them in the role of supporting others’ information needs. If we want to develop a more local, shared sense of responsibility, continuing efforts to incorporate public information production in classrooms should include opportunities for students to support and challenge one another in justifying and critiquing claims, as is done by co-authors on Wikipedia. Since some students seemed at first overwhelmed by the idea of participating in a public forum as a class assignment, it seems that the interventions in this study began at the right level of complexity; however, an additional phase asking students to identify weaknesses in the resource and find collaborators to help address them may have helped them approach collective cognitive responsibility. This approach integrates the coached ad-hoc collaboration introduced by (Zhang et al. 2009) with the responsibility-inducing real world audience of open collaboration content development projects.

Looking ahead

Information literates use their understanding of how information is produced to help them identify and assess the information they need as they help construct new knowledge. With more accurate understandings of how information is produced and regulated in open, collaborative environments, teachers and students can develop increasingly sophisticated strategies for using the information they encounter online.

Integrating participation in information production into formal education is one way to provide young people with experiences on which to build their interpretations of new information environments. Creating information for others effectively introduces an authentic need for accuracy and students have been shown in this study and others to adopt new, creative strategies for sourcing and critiquing claims. It is difficult to imagine that Wikipedians engage in the kind of work that they do without developing more sophisticated skills and knowledge about issues like intellectual property and heuristics for identifying credible sources; or that product reviewers on Amazon write reviews without becoming more adept at interpreting other consumers’ experiences and motivations for contributing; or that contributors to openStreetMap don’t become more savvy about what kinds of geographic features are important to people. These kinds of abilities to interpret new information environments and adapt to changing standards for contribution constitute second-order information literacy skills that go beyond noticing traditional credibility cues like publisher or author. However, it is also clear that without guidance from teachers, parents or other mentors, opportunities for such reflection come too seldom to be considered part and parcel of participatory web experiences.