Public Policy Initiative project details

OVERVIEW

Phase 1

Outreach to key institutions in the field of public policy (completed)
Development of educational materials (ongoing)
Evaluation of the results of the outreach phase (completed)

Phase 2

Establishment of a steering committee of public policy experts
Assessment of the current status of public policy articles on Wikipedia
Hands-on Wikipedia workshops for different target groups
Improvement of articles
Evaluation and documentation

Subject-matter experts have always been valued Wikipedia contributors, and a key goal of this initiative is to facilitate their collaboration with the Wikipedia editing community. We will experiment with different methods of using Wikipedia as a teaching and learning tool in universities, and ways to provide incentives and support participation by students, teachers, and volunteers. The overarching goal of this project is to effectively increase the quality of public policy articles on Wikipedia, and to support Wikimedia Chapters with a model for working with universities to enhance other topic areas.

We have chosen the particular subject area public policy because this topic area is interdisciplinary, and requires collaboration among many fields (including history, economics, law, and various social and hard sciences). We also believe this subject area is underdeveloped on Wikipedia and therefore offers a big opportunity for improvement. Furthermore we recognize that public policy articles may pose special problems – they may center on issues and debates that are more controversial and less settled than other articles in the sciences or in the humanities. We feel that if we can succeed with public policy articles, other topic areas can be improved based on this model.

During the 17-month time frame of the project timeline, the Initiative will be led by a project team at the Wikimedia Foundation working with two keys groups of volunteer Wikipedia editors: "Campus Ambassadors" doing in-classroom training and face-to-face evangelizing, and "Online Ambassadors" providing online support, coaching and mentoring. The Wikipedia volunteers will support university classes, students and professors as they engage in quality improvement of public policy articles on Wikipedia.

The execution of the Wikipedia Public Policy Initiative will take a phased approach. This will include the recruitment of an Advisory Board of public policy experts, establishment of quality measures, baseline assessment of the current quality of public policy articles, and development of educational and training materials specific to this project. We will then pilot quality improvement activities with 3–5 schools during the fall and winter of 2010, learn from the experiences of the pilot schools, and scale up to run work with an additional 7–12 schools during the spring of 2011. The project will culminate in a conference at which best practices will be shared and prizes awarded.

PROJECT OVERVIEW

Scope of proposed initiative

The objective of the Wikipedia Public Policy Initiative is to sustainably increase the quality of public policy content on Wikipedia. It is important that the project be designed so that the quality of public policy articles continues to improve after the project concludes.

This will be accomplished both by directly enabling and encouraging people who are knowledgeable about public policy to contribute to Wikipedia, and by establishing a collaborative workspace that will help existing Wikipedians to assist newcomers, thereby fostering longer term working relationships.

The execution of this initiative will include cultivating and generating new relationships with schools of (or relating to) public policy; promoting the Initiative within public policy academic communities; staging hands-on workshops for training aspiring Wikipedians; developing criteria for quality and categorization of public policy articles on Wikipedia; establishing an effective training/mentoring/support infrastructure, and creating a collaborative workspace to facilitate subject matter experts' work in the area and interaction with other Wikipedians.

The end results will be the improvement of public policy articles on Wikipedia in quality, quantity and structure; and the establishment of a community of practice that will continue to improve content after the conclusion of the funding period.

It is worth noting that (1) this is the first time that the Wikimedia Foundation has attempted a large- scale quality improvement initiative, and (2) there are inherent complexities in attempting to create quality improvements through volunteers. Due to those two factors, our approach will need to be iterative, and success is not guaranteed. We have set out to create a strong mix of encouragements and rewards for participation, that we hope will lead to quality improvements. Even so, we will need to review and finetune our approach as we go, an approach that the project is constructed to enable.

Guiding Principles

This is the first time WMF is attempting to systematically increase the quality of articles in a subject area. Guiding Principles were developed in this context, building on and complementing the norms of the Wikipedia editing community, in order to support an innovative and experimental approach.

Experimental approach. The project will follow an experimental approach. Routinized evaluation and documentation of process and outcomes is an essential element of the experimental approach.
Sustainability. The project will create connections and relationships among students, teachers, and Wikipedians, with the goal of enticing some to remain involved with public policy quality improvement following the conclusion of the project. Teachers will be encouraged to continue using Wikipedia as a teaching tool following the end of the project, and also to embark on longitudinal projects that span several years. Additionally, the project will aim to coax into existence lasting, extracurricular student-driven activities that survive past the project's conclusion, and serve to perpetually recruit new editors.
Subject Matter Experts and Students are a driving force behind content improvement. Wikipedia flourishes at the intersection of two groups: students and subject matter experts. Fundamentally, post-secondary students are the gasoline that fuels Wikipedia: they have the intelligence and the energy to devote many hours to writing encyclopedia content, and they are deeply engaged in the research, assessment, synthesis and writing that is the core work of encyclopedia building. Subject-matter experts are equally invaluable: Wikipedia's success depends on large numbers of people sharing their accrued individual expertise, and exercising their judgment to improve Wikipedia's quality. This Initiative looks to students and professors as the core resource for improving quality in Wikipedia, because our experience tells us those two groups are best positioned to be able to do the work, and also to be passionate about it.
Consultation of the existing community. While our goal is to recruit new editors, we will do so in consultation with the existing community of volunteers, in order to gain acceptance and ongoing support for the project. Buy-in from Wikipedians is vital to the long-term sustainability of this project's goals.
Developing a model. In addition to improving the quality of public policy content on Wikipedia, the project aims to develop a scalable and customizable model that may be adopted and adapted by chapters, volunteers, and other parties, in projects improving quality in a wide variety of topic areas.

Project stages

Stage A: Laying the foundation

To date, a substantial amount of planning work and relationship development have been undertaken to support the Public Policy Initiative and as groundwork for Phase II. (See report on Phase I of initiative for detail.) We have prepared a preliminary project plan and a budget based on that work. During Stage A, we will refine the project plan, secure the engagement of various stakeholders, and establish the educational and training framework that will support the content improvement that is the centerpiece of the project.

Engage interested participants

In this stage, the project staff will be hired and the project plan will be refined and finalized. The WMF project team will reach out to the existing group of highly engaged professors, identified in the initial outreach phase of the Initiative (November / December 2009). In the dialogue with these prospective participants, the project team will provide a detailed overview about the goals, the scope of the Initiative and project schedule in order to get 3–5 commitments for Stage B1 (content improvement pilot).

Establish Steering Committee

A Steering Committee of public policy experts will be established. It will be composed of 6–10 experts in academic fields relating to public policy and will:

endorse the project's goal and general approach;
provide feedback on proposed organization of content on U.S. public policy on Wikipedia;
approve quality criteria for an "ideal public policy article" on Wikipedia (to be used by the students and teachers in Stages B1 and B2);
approve priorities for the improvement of public policy articles on Wikipedia;
supervise evaluation of educational materials used in the Initiative;
identify opportunities for academic research built on the data generated by the Initiative.

Create collaboration workspace on Wikipedia

Concurrently, a collaboration workspace for public policy subject-matter experts will be set up on the English Wikipedia. This collaboration workspace will be designed to:

give students and faculty members an overview of which articles and public policy sub-topic areas need improvement;
provide access to interpersonal help options (e.g. Wikipedia mentors);
serve as a communication platform for the project participants;
facilitate relationship building among students, teachers, and Wikipedians.

Wikipedia Campus Ambassador Training: Train The Trainer

A key deliverable for this stage will be the development of a Wikipedia Campus Ambassador training and certification program. Wikipedia Campus Ambassadors will serve as trainers, working directly with classroom instructors to teach the basics of Wikipedia editing at the beginning of each semester (Stages B1 and B2). The initiative will leverage the training materials currently being developed as part of the Bookshelf project.

Potential first generation Wikipedia Campus Ambassadors will go through a three-step application process, and will be recruited from various sources: the Wikipedia community, university students, and faculty. In a 3-4 day hands-on workshop, Wikipedia Ambassadors will learn the basics of in-classroom training, presentation skills, effective feedback, and classroom facilitation skills. Additionally, Wikipedia Campus Ambassadors will learn about the technical and cultural aspects of Wikipedia editing and content creation. Potential trainers will receive follow-up evaluation and feedback.

To sustain the Initiative's productivity after the grant period, Wikipedia Campus Ambassadors will also work to build resources and a sense of shared purpose among the Wikipedia community that will develop on their respective campuses. Campus Ambassadors will act as mentors to students and faculty, help to start Wikipedia student groups, facilitate the exchange of ideas regarding Wikipedia as a learning tool, and plan social events.

Stage B1: Pilot with 3–5 public policy schools

Stage B1 (fall semester 2010) is the first of the two content improvement stages. In Phase I of the Public Policy Initiative, we interviewed about 40 professors from many schools and research centers. Stage B1 will incorporate 3–5 hand-picked professors who are the closest fit with the Initiative, and the most motivated. We will conduct ongoing evaluation during Stage B1, which will inform the improved execution of Stage B2 (spring semester 2011). Stage B2 will be a broader program in which a wide variety of schools are invited to apply.

With a solid background in Wikipedia editing, and a high-level understanding of public policy, adult learning theory, and teaching techniques, Wikipedia Campus Ambassadors will be assigned to support specific classrooms. During the fall semester (Stage B1), Wikipedia Ambassadors will:

provide learners with a general background on Wikipedia; explore culture, norms, and existing approaches to article quality improvement; and train in Wikipedia editing techniques, user account setup, etc.;
introduce the public policy collaboration workspace on Wikipedia, including help and mentoring;
distribute Wikipedia "welcome packages", i.e. learning materials developed in the Bookshelf Project.

Early in the semester, students will be trained in Wikipedia skills, and start to improve public policy content on Wikipedia. Among the expected activities are:

outlining and structuring sub-topic areas of public policy on Wikipedia;
rating the quality of existing public policy articles based on the recommendations provided by the Steering Committee of public policy experts;
improving the text of existing articles;
writing new articles;
creating charts and other visual aids designed to amplify and augment article text;
performing research and adding sources to existing articles;
reviewing articles that reached a higher quality level.

During the semester, the participants (students and professors) will get ongoing support from their Wikipedia Campus Ambassadors, as well as individual support from mentors and help on Wikipedia.

Beginning in September 2010, WMF project team members will advertise the Public Policy Initiative to a larger group of schools and secure the commitment of 7-12 schools for Stage B2 (Spring 2011). These efforts will include stipends for part-time student positions, and promoting Wikipedia and the Public Policy Initiative on campuses. Additionally, the second Wikipedia Campus Ambassador training will take place near the end of Stage B1. This training will include new potential Wikipedia Campus Ambassadors for Stage B2 schools and current Wikipedia Campus Ambassadors from Stage B1.

Throughout Stage B1, the project team will be conducting on-going evaluations of specific metrics (see "Stage C: Evaluation and Documentation" below for more details). Success at the end of stage B1 is defined broadly: a minimum of three schools have participated in B1; and there is evidence suggesting that the quality of public policy articles will improve.

If we have met the success criteria for Stage B1 and have captured enough data and analyzed it, we may seek to present our initial findings on the Public Policy Initiative at teaching conferences and encourage scholars to use the Initiative as a topic for their own research and publications.

Stage B2: Refined pilot with 7–12 public policy schools

Stage B2 (Spring 2011) will be a continuation and expansion of the foundational work completed in Stage B1. In addition to continuing to work with, support, and enhance the learning experience for Stage B1 schools, we will expand the project to include 7–12 additional schools for Stage B2.

Based on the ongoing evaluation from Stage B1, Stage B2 will begin prior to Spring 2011 with the selection of schools, updated Wikipedia Campus Ambassador trainer workshop, training materials, and a new group of Wikipedia trainees.

The general approach to Stage B2 will be similar to that of B1, with the following significant differences:

Students at 12 schools will be paid a stipend to actively promote Wikipedia and the Public Policy Initiative during the fall semester;
Schools will be selected using an open application process;
The Wikipedia Campus Ambassador trainer workshop for Stage B2 will include Stage B1 certified trainers as instructors and project staff, and will take place near the end Stage B1;
Early evaluation from Stage B1 will inform adjustments to the plan and execution of Stage B2.

Stage C: Evaluation, documentation, and forward-looking final event

At the conclusion of the quality improvement stages (B1 and B2), our focus will shift to evaluating the project on both its content improvement and educational merits. During Stage C, we will document the strengths and weaknesses of our experimental approach, and produce formal documents that will be useful to those seeking to carry our work forward. The results will be important for both informal, ongoing advocacy for incorporating Wikipedia into public policy curricula, and also as data for the sort of academic study that we hope will emerge from the project.

Evaluation during Stage C will build on and complement the ongoing process evaluation conducted during Stages B1 and B2 (Evaluation outcomes and learning will be used to modify or influence training materials, classroom experience, the Wikipedia Campus Ambassador Training program, and other support activities).

We will also celebrate the accomplishments of the students and professors who did the content improvement, and build a sense of community and shared purpose that will feed a continuation of the Initiative's work in classrooms around the country.

Evaluation

Evaluations will be undertaken for each of the following key areas:

1. Improvement of public policy topic area.

The project will endeavor to improve the public policy areas as a whole, including a potential reorganization of the sub-topic areas with input from the Steering Committee. It will measure the number of new public policy articles, and the number of articles that have been improved. Additionally, we will look at the number of active contributors in a specific topic area before and after the execution of Stages B1 and B2.

2. Success of educational effort.

The classroom experience, training materials, teacher behavior, student learning, and Wikipedia Campus Ambassador classroom trainers will be evaluated throughout each project phase. The educational aspects are vital to the buy-in of the academic community. The establishment of a clear and convincing case for the educational merits of the Initiative will be essential to a self-sustaining program that continues to generate content improvement over time.

Documentation

Documentation for this project will happen at two key points: 1) prior to Stage B1 (A Baseline of current Public Policy measurements); 2) at the end of Stage B2 (final documentation of quantitative and qualitative information and evaluation).

Celebratory and forward-looking final event

In the interest of sustaining our efforts, there will be a final event after Stage B2, inviting Wikipedia Campus Ambassadors, classroom participants, project team members, and Steering Committee members. The purpose of this meeting will be to share our initial findings with the group as a whole, gather feedback on what did and did not work. The feedback and evaluation will be used to develop strategies and support for ongoing classroom activities in universities; determine next steps as the project aims for continued quality improvements after the project's lifespan; operationalize and enhance the Wikipedia Campus Ambassador training; and determine how to use the Public Policy Initiative as a model for public outreach to educational institutions, and for Wikipedia chapter projects. Another key aspect of the final event will be to acknowledge and reward the success and good work of participants, so that they may feel a sense of accomplishment and shared purpose coming out of their work. This event will also serve to transition the ownership over the ongoing success of this endeavor to the professors, students, schools, and Wikipedia volunteers (including Campus Ambassadors and online mentors).

HIGH-LEVEL TIMELINE

The timeline below is a visual representation of the project plan proposed above.

LOOKING AHEAD

The primary objective of the Wikipedia Public Policy Initiative is to increase the quality of public policy content on Wikipedia, within the 17 month term detailed in the project timeline.

In addition, the Initiative will produce information and infrastructure that could inform the design and development of a longterm sustainable model. In such a model, we imagine professors, students, and the Wikipedia volunteer community collaborating together to ultimately take over stewardship and development of the support structures initiated during the first year.

We believe a selfsustaining model would be broadly desirable: efforts to improve public policy content would mature and become more productive over time, with a declining need for financial support.