Prioritizing equity within Wikipedia’s new desktop

The internet was an interesting place in 2010. Many of the largest internet websites today were just getting started. Smartphone usage was beginning to grow significantly. Also, Wikipedia had a brand new look.

The result of a two-year-long Usability Initiative funded by the Stanton Foundation, the Vector skin was launched in May 2010 and became the definitive look of Wikipedia.

And it remains there today.

There is something sentimental about Wikipedia’s old-school look and feel. It is reminiscent of the web’s early days, a time of idealism and optimism for the internet’s potential to change the world. It is easy to hold on to vestiges of the past, hoping their presence will inspire the mindset that created them in the first place. If Wikipedia’s 2010 design has kept it within the top 10 sites in the world fairly consistently, maybe that is a sign that it does not need significant change? However, twelve years after Wikipedia’s last redesign, it is critical to ask: who was that site built for, and does it reflect the needs of our audiences today?

Back in 2010, about 70% of the population in the United States (US) and the European Union (EU) had access to the internet. For Sub-Saharan Africa and South Asia, that number was 6.7% and 7.2%, respectively. Clearly, the internet was not a very equitable, inclusive place. The Stanton Foundation-funded Usability Initiative and its exhaustive user tests were mainly focused on the largest of our wikis: English Wikipedia. While distinct voices from emerging markets were welcomed within the conversation, not much was done to seek them out and, based on the low internet penetration of the time, only the uniquely privileged had the means to be a part of the conversation.

Although there is still much room for improvement, things have improved over the past decade. The most recent numbers on internet penetration show a world changed. In the US and the EU, about 88% of the population has access to the internet, a modest but steady growth pattern. Sub-Saharan Africa (currently at 39%), South Asia (currently at 35%), and a number of other emerging markets however, have grown and continue to grow exponentially.

If our Wikimedia vision is to imagine a world in which every single person on the planet is given free access to the sum of all human knowledge, we must begin to think about setting knowledge free and revolutionizing the long-standing models of power, privilege, and access. In 2019, we started the process to update Wikipedia’s desktop interface, with the goal of making it easier for everyone to use. It is a large and sometimes intimidating project, yet from the very beginning, three things became strikingly clear:

1. We are not changing our interface because we think we should modernize, or because there is new sophisticated technology we could use (although this technology does allow us to pursue necessary tools that were previously impossible to build). We are changing our interface because it was not originally built for every single person on the planet. It was built for the ones whose voices we could hear and whose needs we could identify back in 2010.

2. We need to do things differently this time. Focusing specifically on any one Wikipedia, such as English Wikipedia for example, limits us to a single perspective. Only having conversations in English limits the number of voices we can hear from. This time, we sought out the voices we had not heard from in the past, allowing their needs to stand along with, and in front of, the needs of those who have been with us from the beginning.

3. This was going to be a large effort, requiring more planning and consideration than anything our team had built in the past.

With these guiding premises at the forefront, we took the following steps to incorporate equity into the ongoing process to update Wikipedia’s desktop.

Equitable Growth

In the Product department at the Wikimedia Foundation, we talk a lot about “equitable growth”. It’s a term that succinctly captures a lot of the thoughts above. Yet, without precise definition it can lead to a scattered strategy. Thus, we’ve spent a lot of time defining – creating processes that we can follow, metrics we can measure against and commitments that will ensure we do not lose our focus. We were also beta-testers for the inclusive product development playbook – a department-wide framework that focuses on creating best practices for  diverse, equitable, and inclusive product development. Central to this process is the idea that we will build with emerging market communities rather than for them.

What that meant was that we would need to do things a bit differently with this project. Traditionally, when building out new features, we like to offer them to a small set of users, usually as a beta feature where some could opt-in and give us feedback. While there are some obvious issues with bias in this approach (people that are interested in technology are more likely to turn on beta features) it has been a solid way to get quick feedback in the past.

This time around, however, we wanted to show these features to a wider audience as well, not only to people that have accounts on the site. This gave us an idea – what if we were able to deploy features to everyone on a given wiki? It would allow us more reliable data from everybody on that wiki – readers and editors alike. We’d be able to get feedback from a wider audience and iterate quicker. We ran it by a few wikis whose communities we thought might be interested and – they said yes! We created a list of these partner communities and over time, built a close relationship and feedback loops to ensure we are listening and adjusting to their needs.

In this process, deciding which wikis to approach was crucial. We wanted to be able to get a sample of communities that represented different perspectives and geographies. We focused on a few different factors:

• Size: We wanted to be able to work with large wikis with languages that are spoken on multiple continents, as well as smaller wikis who have previously struggled to get attention to their needs. Imperialistic legacies have left a lasting effect on many countries and languages such as English, French, and Spanish are still the default for many countries. For example, many editors from Africa contribute either to the English or French Wikipedia, with a smaller percentage working within a local language. We knew we must speak to representatives of both communities to get a full picture of needs and requirements.
• We wanted representation in right to left languages, as well as left to right languages. We’ve learned the importance of this from past mistakes. It is absolutely crucial to test our changes on both prior to any deployment to ensure that the patterns introduced make sense in both scenarios.
• We wanted representation from different scripts. The more the better! Similar to the point above – we wanted to make sure we had wide coverage to ensure that we considered the experience for varying scripts from the very beginning of the project
• Different projects. We know Wikipedia is large, and important, but that should not relegate our other projects to afterthoughts. By ensuring that we represent multiple projects from the beginning, we ensured that specialized use cases were built for from the initial design, rather than retrofitting the design to fit once Wikipedias were ready.

Currently, our pilot wikis include: Basque Wikipedia, Bengali Wikipedia, French Wikipedia, French Wiktionary, German Wikivoyage, Hebrew Wikipedia, Korean Wikipedia, Persian Wikipedia, Portuguese Wikipedia, Portuguese Wikiversity, Serbian Wikipedia, Turkish Wikipedia, Venetian Wikipedia, Wikimedia Incubator.

Testing across different communities

While our pilot wikis ensured that our features would be shown in front of a wide audience, we knew that deeper feedback from both readers and editors was needed. The project itself is quite reader-focused, so we decided to start there. We wanted to look into what made the current interface confusing, what was difficult to understand and could be improved. This led to our first user research efforts.

Here, due to budget constraints, we had to be a bit more selective of our target audience. The considerations were similar – we wanted to test within emerging markets that had a variety of different languages, scripts, and with users that worked across different wikis, but for the first round at least, we had to select one market to start.

We began in India and hired a contractor, Hureo, to aid us in recruiting users. Our main focus was two-fold: 1)- testing readers that switched between languages frequently and 2) testing readers that only read in their local language. We also wanted to make sure that we included newer readers into the group rather than only focusing on readers that already knew of and used Wikipedia frequently. The test was a success – it confirmed many of our suspicions on the deficiencies of our interface – people didn’t understand our navigation, had a hard time switching languages, and found the interface to be old-fashioned and thus, perhaps less reliable. However, the tests also aided us in revealing issues as well as benefits from our current interface that we hadn’t thought of previously. For example that talk pages were viewed as places that could potentially be used as a general discussion, or, that when people were stumped by our navigation they didn’t necessarily turn to a different site for answers, but would rather search for our content within search engines such as Google as a way to circumvent the parts of our site that they found confusing. These findings were crucial to our development and, without this type of testing, would have been impossible to reveal.

With time, and as our project grew, we were able to continue this type of testing and expand it to include different countries within our focus. So far, we have been able to test in Ghana, Argentina, India, and Indonesia. The results of these tests continue to be instrumental in our development and oftentimes very surprising. More than once we have seen results that look strikingly similar. For example, the reactions to our new table of contents prototypes were nearly identical in Ghana, Argentina, and Indonesia.

A new approach to sharing project information

A lot of the documentation around the project is available on MediaWiki. In the past, this is also where the majority of the conversations would have taken place, with the addition of individual conversations directly on each project’s main forum. Yet sometimes this approach limited the audience that was involved – both linguistically as well as in terms of the years of experience on the wikis. We were more likely to hear from veteran community members than new active editors, and often the perspectives we heard were those of an individual, rather than of the many groups and Affiliates which give strength to our movement.

While this approach is still a crucial part of our engagement strategy, we also knew that we need to create more entry points for people to reach out, ask questions, and give feedback. We began a series of office hours, initially in English, but over time, growing into meetings that had support for live interpretation in multiple languages. As we get closer to releasing the new experience for everyone, we are able to host office hours specifically designed for a number of languages – Spanish, Arabic, Russian, and more. We’ve also begun looking outside of the wikis to find interested groups of community members – discussing changes with them across messenger apps and social media.

Building with accessibility in mind

Accessibility has also been a crucial tenet of our approach. Our approach here was to strive for an even higher level of planning than in 2010 when building out accessibility features. In the past, a lot of our changes specifically targeting accessibility were retrofitted solutions, this time we integrated them into the planning of each feature. This ensured that accessibility standards were followed and that fewer surprises were found once we began testing.

We also focused on changes necessary to improve the accessibility of the site as a whole, such as improving the contrast of our link colors or increasing our font size to aid people with visual impairments.

As the project is coming to a close, we are also working with the American Foundation for the Blind on testing the accessibility of the entirety of the new skin.

Conclusion

We believe we have made great strides towards inclusive development in both our process and decisions made on what to build. However, we cannot say that we have reached a perfect state. This is an ever-evolving process and we hope the additions that we have made to advance equity are valuable and will continue to be used in our product development plans in the future. We hope that these strategies will also increase in number as we learn more about our audiences and their complexities, and strive to make Wikipedia truly welcoming to all.

Learn more about the project to update Wikipedia’s desktop.