Research:Section Translation Entry Points Design Research
Improving discoverability of translation tools on Wikipedia
In 2020, a study of current and potential editors in seven small wikis revealed that most participants required direct support to find Content Translation - a tool used to translate Wikipedia articles (read more about that study here). At the same time, Content Translation has been used to create more than 800k articles, demonstrating the power of translation to help grow content and improve access to knowledge. Thus, helping to connect multilingual editors with translation tools is a promising strategy for increasing content available on small wikis.
To help solve the demonstrated problem of Content Translation discoverability, this project provided iterative evaluation of current concepts and prototypes for Section Translation entry points. Section Translation - a recently available (on Bengali Wikipedia) in 2021 mobile-friendly version of the Content Translation tool - currently lacks natural entry points. The entry points evaluated in this project included those designed for both current and new translators, but focused primarily on Section Translation and the mobile experience. They included concepts for both persistent and opportunistic entry points. The initial hypotheses upon which the prototypes were designed originate in part from findings from the Multilingual Editor Experiences in Small Wikis Project.
Highlights
[edit]- This project provided contextualized evaluation of eight Section Translation (SX) entry point concepts through moderated research sessions with potential and current editors of the Bengali Wikipedia (the first wiki where SX is available).
- Using an iterative approach, one concept was eliminated after the first round, and further improvements were made and tested for the other concepts. This report provides follow-up recommendations for these seven other prototypes (more details in general discussion), and proposed priorities for implementation (more details in results section).
- SX currently lacks natural entry points, including a way of easily accessing it. A persistent entry point in the mobile menu was identified as promising, but improvements were needed to the visual treatment of embedded items.
- To increase the existing SX translator base, the language switcher entry point should be prioritized due to the translation editing opportunities (content gaps) it can present to both potential and current editors.
- Some entry point concepts should be exposed to specific segments of translators. For example, the machine translation (MT) sections for readers entry point can be targeted to registered editors, and newcomers may benefit from the recently translated notice being refined to display for
- Provided it’s displayed to editors who actively edit two or more language versions of Wikipedia, the entry point after a regular (non-translation) edit is perceived as of greater ease given it’s a natural task extension.
- This report focuses on designs and improvements made during two rounds of research sessions, as well as some final recommendations. The latest designs and development progress can be tracked at this Phabricator ticket.
- Although it doesn’t eliminate the need for additional research, the general discussion provides some concepts and ideas for what may be more generalizable findings relevant for other teams developing entry points for other projects and contribution tasks. (more details at ‘entry points - beyond Section Translation’)
Thanks to Pau Giner and other members of the Language Team for discussion and review of project plans, details, and ongoing reports, including ideas included in this report. Thanks also to Anagram Research, who provided logistical support for participant recruitment, consecutive interpretation, and research session facilitation.
Research goals and approach
[edit]The goal of this project was to provide iterative qualitative assessment of entry point concepts, available as high fidelity clickable prototypes. We wanted to provide an initial assessment of which concepts were most promising and identify opportunities to improve current designs. The three main areas of focus were:
- Discoverability - Is the entry point easy to find and access? Do editors see and understand the opportunity?
- Contextual Alignment - Does the entry point align in context and timing with editor motivations and workflow. Do these opportunistic entry points provide a logical next step based on what the editor is working on, and do they introduce a relevant value proposition to leverage translation while at the same time not introducing an annoyance.
- Clarity and Ease of Use - Does the entry point clearly communicate what will happen next without causing confusion? What general usability problems exist?
Hypotheses and questions
[edit]For the persistent access point, we aimed to test one primary hypothesis (see complete description of entry point concepts and links to prototypes here).
Hypothesis 1 : By providing an easy-to-find persistent access point, returning translators know where they can always easily access Section Translation. (persistent entry point)
- Are experienced translators able to easily locate the persistent Section Translation entry point?
- Are new translators able to locate Section Translation after a minute or two of exploring once they know the tool exists?
- Do the first few actions taken by editors to find Section Translation align with the general location of the persistent entry point?
For each of the opportunistic entry points, we posit the following two hypotheses to test.
Hypothesis 2 : New and returning translators encounter translation opportunities at times and places at which they are likely to use Section Translation to expand article content. (opportunistic entry points)
- Does the entry point present a logical next step based on the editor’s/reader’s current activities?
- Does the opportunity presented by the entry point align with the editor’s/reader’s expectations for the next step and activity?
Hypothesis 3 : Section Translation opportunistic entry points present compelling value propositions without introducing annoyance or confusion.
- What confusion does the entry point introduce, either around the opportunity and/or next steps?
- What aspects of the entry point does the reader/editor find most/least compelling? Why?
For each of the entry point concepts, we defined a specific set of research questions. These were integrated into the research protocol through a series of follow-up questions and key observation points.
Research approach
[edit]Methodology and materials
[edit]The approach for this project was motivated foremost by the goal of supporting design exploration and prototype iteration. Leveraging the Rapid Iterative Testing and Evaluation Method, moderated remote research sessions were used to gather feedback on the entry point concept prototypes. Sessions were split into two rounds. After the initial eight sessions, design integrated initial feedback to update designs, and two of the eight initial concepts were removed before the final eight sessions. One was dropped because feedback was overwhelmingly positive and minimal issues were noted. The other was removed because it lacked discoverability and contextual alignment with editor tasks. All sessions (approximately 60-75min in length) were guided by a detailed protocol with guiding questions and observation metrics for each of the entry point concepts. The protocol helped set up a context for each of the concepts, and provided a natural and realistic flow for the sessions. All sessions were recorded (minimally audio and screenshare). From these videos we confirmed tracking of key metrics of interest, and extracted additional data points. Following each round of sessions, an interim summary was prepared for the Language Team. Research and design also gathered to discuss results and next steps, such as the design changes ahead of round two sessions.
Participants
[edit]The sixteen participants for the study represented both current and potential Wikipedia editors. Both segments were equally represented in both rounds of sessions. As Section Translation is currently only available in the Bengali Wikipedia, current editors were recruited from the existing community of Bengali editors through the use of community pump announcements, direct recruitment messages to contributor talk pages, and referrals. Interested participants responded by completing a screener questionnaire, which was followed by an invite to select a session date/time. Potential editors (multilingual Bengali speakers) were recruited with the assistance of Anagram Research partners and represented readers who had a clear idea of how content is produced on Wikipedia (i.e., collaboratively via volunteer editors) and had previously considered (but hadn’t yet) edited Wikipedia. All sessions were conducted in Bengali using consecutive interpretation.
Entry point concepts and prototypes
[edit]A total of eight entry point concepts and accompanying prototypes were used in this project. They included persistent, opportunistic, and proactive entry point concepts. A persistent entry point is one always available, an opportunistic entry point presents opportunities to translate and discover the tool, and a proactive entry point assists in identifying translation opportunities for existing editors. The eight initial entry point concepts are briefly described below, along with a link to the prototype. All links below are to the original version of the prototype (the most recent versions of the prototypes can be accessed by removing ‘/v1’ from the URL).
- Machine translation sections for readers (opportunistic)
- When a reader arrives at a short article with no sections, a panel at the bottom has machine translation versions of additional sections for users to read, along with an invite to improve and add them to the target language article.
- Language switcher (opportunistic)
- When a reader opens the language selector on an article, they can view missing languages and the option to translate for languages not available.
- After switching languages (opportunistic)
- After the user navigates to another language version of an article, they receive an invitation to expand the article by translating missing sections.
- Persistent access point in menu (persistent)
- When opening the menu and selecting ‘new contribution’, editors can select ‘translations’ as one of three options to begin contributing.
- Suggestions filtering - translation dashboard (persistent)
- Editors can narrow translation dashboard suggestions to a specific area of interest by selecting ‘based on previous edits’ or viewing ‘popular pages’.
- Notifications (proactive)
- Editors are notified when an article they contributed to with translation gets (1) a new section in the source language they can translate, or (2) the article gets a significant number of visits in the previous month.
- After a contributor makes a regular edit to an article, they receive an invite to translate the edited section to other languages they know where it’s missing.
- Recently translated article notice (proactive)
- When a reader encounters a recently published translation, they see a notice encouraging them to review and expand it if sections are missing.
Results
[edit]Results are presented for each of the entry point concepts below, including feedback from both the first and second round of sessions, as well as highlights of design changes made between rounds. A more general discussion follows the reporting on individual prototypes. Additional details and discussion notes were available to the Language Team via two interim reports delivered ahead of this final report.
Machine translation (MT) sections for readers
[edit]When a reader arrives at a short article with no sections, a lower panel has the machine translation versions of the sections for users to read, along with an invite to improve and add them to the target language article.
During the first round of sessions two main patterns emerged. First, the imperfect machine translation (MT) contents presented alongside human-curated article content had the effect of motivating many participants to want to improve the MT outputs (assuming they felt knowledgeable about the topic and the quality of machine translation output wasn’t too poor). Secondly, it wasn’t always clear to participants (especially potential editors) what content was part of the existing article and what content reflected MT outputs that could be added to the main article contents. The division between existing article content and potential article content was blurred.
The following design changes were made before round two sessions. First, call-to-actions (CTAs) around the MT sections were simplified as most participants were motivated by the ‘correct and add to article’ CTA, perceiving it to involve a smaller amount of work. Moreover, the difference between ‘correct and add’ and ‘improve a translation’ was not evident to all participants.
Secondly, it was recommended to more clearly distinguish the MT sections from the current article content in a way that made it clearer that the MT sections were merely potential article content. Visual treatment was held constant, but additional text was leveraged to call out MT vs existing article sections.
During the second round of research sessions, a main finding was that all participants continued to readily notice the MT sections, and all but one participant in the second round expanded the MT section available to read. Also consistent with the first round of sessions was that in the second round none of the potential editor participants readily perceived the MT sections as distinct from the existing article contents, as illustrated in the quote below. We also learned that when presented with lower quality language in a section, potential editors and readers may default to another language version of Wikipedia, or another source, for information. Quality of MT outputs, especially if they’re perceived to be a core part of the existing article content, may also affect perception of information quality, as noted by participant comments such as, “There are a lot of mistakes [in the language] so I’m not confident reading this article”.
As for current editors, if the perceived effort to improve the translation is not too high, the MT sections presented an easily completed task. The most common question among current editors was who the MT sections were being displayed to, as illustrated in the first quote below.
- "Can only current editors see these sections?” (e.g., CE9, CE10, CE15)[1]
- "They’re [MT sections] the same [as the main article sections], they just discuss different aspects of the topic” (PE3)
- "The translation and use of language [in the MT section] is surprising to see in an existing article” (CE15)
- "Normally when I see a short article, I add and correct the article so it’s more helpful to the reader” (CE1)
Overall, a striking difference between potential editors and current editors was in their perception of something as simple as the word “translate” in a CTA and interface. It was clear that participants’ relationship with Wikipedia tended to influence their interpretation of CTAs such as ‘start translation’. Potential editors (readers) commonly interpreted this to mean ‘read a translation’, whereas current editors interpreted the same CTA to mean ‘produce/edit a translation’. This is notable because it illustrates a mismatch of workflow for potential editors who are surprised to find they’re being prompted to edit.
Recommendations
- Selectively display MT sections and provide a toggle off option
- Plan a limited exposure to registered users only, providing an easy toggle off option.
- Increase the visual contrast between MT sections and existing article content
- More clearly distinguish MT sections from main article content, emphasizing that they are potential (not current) article contents.
Language switcher
[edit]When a reader opens the language selector on an article, they can view missing languages and the option to translate for languages not available.
During the first phase of sessions, we discovered that experienced editors have very fossilized patterns for accessing the language switcher that have to be weighed alongside newcomers’ ease of locating the language switching option. For example, when switching languages, experienced mobile editors may immediately load the desktop site on their phone and look for the switching option in the left sidebar. After the first round, discovery of the language switcher was noted as a watchlist item.
Upon opening the language switcher, participants see the search option near the top and ‘translate to more languages’ near the bottom. Participants were observed using both options.
As for barriers to initiating a translation via the language switcher entry point, the blockers observed fall into two main categories. First, the editor may not feel qualified for the task depending on the topic. Secondly, editors may not initiate the translation if they feel the MT output quality is too low.
During the second round of sessions, most importantly, we moved to more conclusive validation of the discovery of the language switching option. All participants in this second round, including current and potential editors, quickly and easily found the language switcher. After opening the switcher, we continued to see traffic to both the search and ‘translate to more languages’ option.
As part of this entry point concept, after selecting the target language, participants are presented with a new translation screen, in which they can adjust languages and tap ‘start translation’. Upon tapping ‘start translation’, there were some notable juxtapositions of participant expectations and the next screens, both for current and potential editors.
For potential editors, contrasting expectations stemmed from the focus on wanting to read the translation. PE16 remarked, “I thought the whole article would appear in Bengali”, expecting to read a full translation. Indeed, ‘start translation’ is ambiguous between a reading and editing activity. As for current editors, the sentence-by-sentence flow of Section Translation presented some confusion. At first some participants were disappointed when only the title appeared (per design). They didn’t understand the nature of the workflow, instead interpreting this first step as a signal that the translation process was more ‘from scratch’ as opposed to MT-supported.
Recommendations
- Maintain the language switcher icon and position as is
- Discovery and navigation presented minimal challenges for current and potential editors.
- Refine logic for displaying ‘start translation’ and quick tutorial
- For returning translators, especially if the target language can be predicted, revise the workflow to move them directly to editing.
After switching languages
[edit]After the user navigates to another language version of an article, they receive an invitation to expand the article by translating missing sections.
This entry point concept overlaps with ideas from #1 Machine translation sections for readers. The main difference is that in this concept, there's a translate option presented at the top of the page, which redirects to MT sections at the bottom of the page (below the fold for any articles with more existing contents).
Overall, in the first round of sessions there was a very clear pattern for participants to immediately scan and read the Bengali contents, reviewing what sections and subtopics were available. Paired with this behavior was a notable lack of effectiveness in the ‘translate more sections’ banner to grab attention. Whether due to general web banner blindness, or specifics of attention to content on Wikipedia, it was clear that this entry point concept was not promising. However, we may note that the MT sections at the bottom of the page were very captivating, lending support to this concept as presented in #1. Given a lack of promise, this concept was removed from the protocol moving into the second round of sessions.
Recommendations
- Invest more into concept #1; do not proceed with this concept
- This concept was removed prior to the second round of sessions.
Persistent translation access point
[edit]When opening the menu and selecting ‘new contribution’, editors can select ‘translations’ as one of three options to begin contributing.
During the first round of sessions most participants were able to locate the persistent access point, but some more quickly than others and others with assistance only. Overall, figuring out the main hamburger menu was the first step was less challenging than figuring out translation was embedded under ‘new contribution’. One participant noted they found it through the process of elimination, and others reflected on the task, commenting it made sense after the fact, but initial discovery was difficult.
Ahead of the second round of sessions, the main page on the prototype was made scrollable to increase the realism of the scenario, and a language button was tested at the bottom of the page - a host to a ‘find contents to translate’ CTA.
During the second round of sessions, we observed a general solidification of patterns observed in the first round. Foremost, most participants immediately opened the menu when searching for the translation option (e.g., CE9, CE15, CE10, PE11, PE12, PE13). However, only two found the translation option embedded under ‘new contribution’ in the list of menu options without guidance (CE10, PE11). Notably, many participants did not expect the translation option under ‘new contribution’, and one participant mentioned the lack of visual cue that there were embedded options. Generally, participants were observed closing the menu and scanning the rest of the main page. Some even opened the menu more than once without finding the translation option (e.g., PE12, PE13).
Recommendations
- Improve discoverability of embedded menu options
- While the main menu was intuitive for participants, it was very difficult to naturally unearth the translation option when embedded under ‘new contribution’. A visual cue that there are embedded options for menu items is needed.
- Add a ‘new contribution via translation’ to the mobile web contributions page (additional entry point parallel with desktop)
- The contributions page is a high traffic area for editors. Currently there is a new translation option on the web version, but not mobile web. Mobile web editors should equally receive exposure to this translation entry point
Suggestions filtering - translation dashboard
[edit]Editors can narrow translation dashboard suggestions to a specific area of interest by selecting ‘based on previous edits’ or viewing ‘popular pages’.
As with all other prototypes, this one was contextualized and the suggestions filter wasn’t expanded by default, in part to test discoverability of the interaction. Before being prompted to interact with suggestion filtering (unless participants spontaneously navigated to it), most participants scanned the first screen for an article that matched their specific topical interests. This first screen presented an initial list of articles with visuals. When participants didn’t find a match, around half wanted to refresh the list and the other half opted to search. During the first round of sessions, the majority of participants did not find the suggestions filtering option without prompting. Once they were directed to the interaction, general reactions to suggestion filtering were positive. Many participants didn’t understand ‘active campaigns’, and some wanted the ability to target more specific sub-topic areas.
The most interesting filtering option for most participants was the ‘popular topics’ option. This was because participants understood these articles to be more exposed to readers. Moreover, it also matches how some editors currently identify topics to translate - they use social media to get a sense of popular/trending topics. Ahead of the second round of sessions, a number of design changes were made. Some examples of these changes include the following. Visual changes were made to increase the discoverability of the filtering option, and ‘popular topics’ was made the default filtering criteria. As for topic selection, when ‘art’ was selected, sub-categories were shown as a way to narrow down a broad topic. Finally, as for the initial list of suggestions, the number of suggestions was doubled to ten, and contrast was increased to try to draw out the distinction between ‘new pages’ and ‘expand with new sections’.
During the second round of sessions, we saw a repeat of participants’ initial actions. All participants spent the first few minutes scrolling through the list of suggestions, and a few tried to refresh the list. The visually presented articles remained of most interest through the various screens of this prototype. As for results of participants finding the suggestions filtering interaction, results remained mixed. Although results navigating to the filtering options interaction were mixed, some general dashboard patterns were very consistent.
- "Popular topics is most useful, otherwise I wouldn’t know what’s currently popular” (CE9)
- "People love to expand articles that are read by many people” (PE8)
- "I can easily view how much I’ve contributed through this section [visual presentation of stats]” (CE15)
- "Stats feels very motivating” (PE11)
- "The motivation behind popular topics is a lot of views. Others may want to look it up, read it, or access that information” (CE1)
As for the ‘most useful’ dashboard features, participants consistently noted ‘popular topics’ (e.g., CE9, CE14, CE10, CE10, PE11, PE13), ‘translation stats’ (e.g., CE14, CE15, PE11), and the general list of topics (e.g., CE10, CE15, PE12, PE13). A few current editors expressed concerns about pointing editors to popular topics due to vandalism and questions as to whether the articles would be protected/editable. Another expressed interest in seeing a ‘in recent news’ filter as a variant of popular topics.
Recommendations
- Improve discoverability of the suggestion filtering interaction
- One or two of the filtering categories may be previewed visually to help signal and communicate the purpose of this key interaction.
- Increase the prominence of search
- Along with refreshing the initial list of suggestions, many participants wanted to quickly and easily search for topics and articles.
- Further differentiate ‘popular topics’
- Depending on technical feasibility, the ‘popular topics’ category could be further split into ‘frequently read’, ‘in the news’, etc.
- Support saved tasks and to-do lists
- When primarily engaged in reading, editors want an option to bookmark (save the entry point) for later work. There’s also the potential to integrate a ‘save for later’ feature with the language switcher entry point.
Notifications
[edit]Editors are notified when an article they contributed to with translation gets (1) a new section in the source language they can translate, or (2) the article gets a significant number of visits in the previous month.
Translation opportunities presented via notifications received immediate interactions, especially from current editors. A few potential editors had to be prompted to interact with them. Especially for current editors, notifications appear to be a valuable and promising place to present translation opportunities due to high visibility. Following round one of sessions, this prototype was removed from the protocol because confidence was immediately established as high.
- "I always check notifications to see if there are any updates on an article I edited or if someone responded to me. Then I look up articles and see if they exist on Bengali Wikipedia yet.” (CE10)
Of all notifications presented, the ones that received the most interest were those regarding articles that had been expanded with a new section to translation and those presenting translation views and contributions from others. Participants immediately wanted to know more about (1) who viewed their translations, and (2) who also contributed to translations they worked on. In part, the second question was driven by wanting to monitor for vandalism.
Overall, no notifications presented in the prototype were noted as distracting, except for the ‘thanks’ notification, which some participants noted as misplaced when presented alongside the test notifications. Per the recommendations below, notifications should be categorized at time of presentation; this is particularly relevant as the number of potential notifications increases due to editor activity.
Recommendations
- Organize the notifications list
- ‘Views’ and ‘contributions from others’ notifications should be given prominence via ordering based on participant feedback. All notifications should be grouped according to type; for example, ‘thanks’ notifications should be grouped together.
After a regular, non-translation edit
[edit]After a contributor makes a regular edit to an article, they receive an invite to translate the edited section to other languages they know where it’s missing.
Both current and potential editors easily found the ‘contribute to other languages’ panel. Those unfamiliar with translation tools on Wikipedia had questions about how the tool would work. Other questions included whether or not a MT output would be automatically generated or whether this option was presenting a manual translation opportunity.
In general, short additions to articles represented more promising options for a ‘contribute to another language’ extension of the editing task. “If I can do it quickly,” in one participant’s words. Importantly, contents of the most recent edit are still ‘fresh’ for participants, making this a natural task extension.
In the second round of sessions, again we observed all participants easily and quickly noticing this entry point. Compared to other entry points presented, this one was commented on as being of greater ease given editing of the same section just prior. One participant noted, “I’d want to make sure the Bengali article is just as complete.” Any hesitancy observed was around questions of language availability, confusion if the translation would be done from scratch or with MT help, and a general desire to ‘save for later’.
- "Since I already edited in English, it’s going to be easy to translate” (CE9)
Recommendations
- Presented to the appropriate subset of editors, this entry point is very promising
- For editors who actively edit in two or more language versions of Wikipedia, the general ease of this translation task compared to others makes it promising.
- Surface and preview MT outputs earlier
- More editors may be likely to complete these tasks if they know early on that they do not have to write the section again from scratch.
- Support as a saved task and display more frequently for shorter sections
- Target this entry point for shorter edits that may be more easily accomplished in the same editing session, and provide a ‘save for later’ option.
Recently translated notice
[edit]When a reader encounters a recently published translation, they see a notice encouraging them to review and expand it if sections are missing.
During the first round of sessions, we quickly observed that participants were easily and quickly discovering the ‘recently translated’ notice. It was interpreted as a signal that some action was needed for the article. For example, some saw it as a ‘warning’ that the article may have some errors.
As for the two action items ‘review’ and ‘expand’, the former received far greater interest. This was largely due to the fact that participants felt that fixing an existing section is more important than adding new sections, or should at least come first. When selecting to review or edit, editors wanted the ability to toggle between the source and target language sections to cross reference contents.
Ahead of the second round of sessions, the ‘review’ label was updated to ‘improve’ to reflect participants’ interpretation of the task behind the entry point. Icons were also added to the labels to reinforce their meaning.
During the second round of sessions, unanimously, current and potential editors’ first reactions were to the poor quality of the language. Next, they all discussed ways of improving the section; again, expanding the article was perceived as secondary to the act of improving the displayed section translation.
- "It has to be fixed first, everything else comes later” (CE9)
One current editor elegantly captured how they perceived three possible next steps when encountering a recently translated notice. The next steps are a direct function of the quality of the MT outputs (or more generally quality of the translation):
- If the language is incomprehensible, then a quick deletion proposal is best.
- If the section needs a reasonable, but not insignificant, amount of work, then flagging it for others to work on is appropriate.
- If the section merely needs a small fix, then it should be immediately edited and resolved.
The sentiment that the nature of the translation quality impacted next steps was echoed by other current editors. Overall, current editors agreed that such notices are helpful, but with the exception of one editor, all noted hesitancy about them being shown to general readers.
As for the effect on readers/potential editors, those who participated in this study noted that if the language is poor enough (as in the prototype example), then it degrades the reading experience, but could also drive traffic to larger wikis as larger wikis are assumed not to have such notices. Overall, both feedback from current and potential editors supports an approach to recently translated notices that attempts to factor in a more nuanced level of translation quality and/or translation quality predictors.
Recommendations
- Factor translation quality and section length into display logic
- While translations needing a small number of improvements might be appropriate for a more general audience, those requiring more work are best suited to more experienced editors, whose perceptions of wiki quality are less prone to being negatively affected.
- Suggest article expansion as a follow-up to those selecting to ‘improve’
- Given almost no interest in expanding before improving, the multiple CTAs could be simplified to a single ‘improve’. For any editors selecting to improve, they should be presented options to expand the article upon publishing the improvement edits.
- Refine logic for the ‘quick tutorial’
- Experienced editors may be annoyed by any intermediary steps between tapping to improve and edit. Focus on displaying guidance to newcomers.
General discussion
[edit]From indidividual concepts to cohesive plan and strategic priorities
[edit]The previous sections of this report focused on reporting results for the eight initial concepts and prototypes. We reviewed results through two rounds of research sessions, noting some of the adjustments and pivots made after the first round. We now turn from individual concepts to a more global view of entry point strategy.
As previously noted, Section Translation is currently only available on the Bengali Wikipedia, and lacks natural entry points (persistent or opportunistic). As such, adoption is limited by two main factors: availability across wikis and ease of discovery by new users. While release to other wikis is outside the scope of this report, it’s clear that prioritization of at least one persistent entry point and a few opportunistic entry points would be most advantageous to increasing the size of the current translator base for the tool. In other words, entry points that provide ways for new editors to discover and use translation. Proactive entry points - those identifying opportunities for repeat users of the tool - may be lower in priority from the perspective of a strict stack-ranked priority list.
As for a persistent access point, there was positive feedback that the proposed placement in the main mobile menu is promising as long as discoverability of the embedded ‘translate’ option is improved (see recommendations in results, section 4). For opportunistic entry points, we learned that the ‘after switching languages’ concept (#3) was not promising, but both ‘MT sections for readers’ and the ‘language switcher’ entry points worked well. The language switcher entry point is least controversial as it does not introduce the concept of proactively displaying MT sections. Although not grounds for exclusion, this may be perceived as controversial by some, and may not be suitable for non-registered editors. As such, it’s suggested to prioritize development of the language switcher entry point before implementing the MT sections for readers because it has wider reach and incurs less risk.
Finally, zooming out a bit further, as we discuss a cohesive plan for Translation tool entry points, we should not forget that the ultimate end user experience for readers and editors is not one defined solely by translation on Wikipedia. Multiple product teams are all simultaneously considering entry points for various forms of contributions. For this reason, we have to consider how our strategies for entry points and discovery overlap with those for other modes of contribution. For example, without logic to selectively only display an entry point banner or notification if we’re certain no others are being displayed, there’s the risk of overwhelming a reader or editor with information that is secondary to why they’ve come to the page - to access and/or contribute knowledge. Inadvertently moving main article contents below the fold on mobile is one example of how the reader experience could be negatively impacted.
Moving towards a more global strategy of contribution entry points comes not only with risks, however. It also brings potential advantages. For example, we saw in this project how translating an edited section to another language can be a natural extension of a non-translation task. There could be other examples of such symbiosis of tasks that from an organizational structure standpoint are arbitrarily split across product teams for reasons of organizational efficiency and efficacy. For example, there may be newcomer tasks that, if completed successfully, could prove useful signals for identifying editors who may be particularly well-suited to contribute through translation.
Latest updates and progress tracking
[edit]In parallel with research reporting and discussions, design has continued to iterate on the prototypes and concepts, per recommendations detailed in earlier sections. The latest designs and development progress can be tracked at Phabricator ticket T286641, which provides an overview and links to individual items.
Entry points - beyond Section Translation
[edit]Although this project has focused on entry point concepts for Section Translation, the challenge of providing intuitive entry points to contribution tasks is one spanning multiple product teams and projects. As such, in this final section, we ask what we can abstract away from our learnings about translation entry points to entry points more generally. This is intended as guidance to others exploring entry points, and we hope it provides some general strategic guidance and may lead to additional research questions other teams may wish to address in the context of their own projects.
To begin, let’s briefly revisit three general framing questions that any such project can use to begin testing new ideas.
- Discoverability - Is the entry point easy to find and access? Do editors see and understand the opportunity?
- Contextual Alignment - Does the entry point align in context and timing with editor motivations and workflow. Do these opportunistic entry points provide a logical next step based on what the editor is working on, and do they introduce a relevant value proposition to leverage translation while at the same time not introducing an annoyance.
- Clarity and Ease of Use - Does the entry point clearly communicate what will happen next without causing confusion? What general usability problems exist?
Next, let’s explore what may be some more widely generalizable findings from this particular project. As a caveat, some of these border on more general heuristics, in which case the real question is ‘what heuristics are most pertinent when it comes to entry points? And, as a disclaimer, while these may offer some general guidance, they should not absolve a team from pursuing their own entry point research for their specific contexts and tasks. The current project was carried out in the specific context of Section Translation with Bengali editors and readers, and as such detailed findings should not be assumed to necessarily generalize to every editor and reader. These broader takeaways (eight in total) fall into three general categories - understanding - motivation - action.[2]
Understanding / Discovery
- Don’t underestimate the power of embedded menu items to conceal themselves. The further an entry point is buried, the more layers of categorization logic required.
- Different types of platform engagement are associated with different mental models. As an example of why this matters, we observed that readers interpret ‘start/get translation’ as ‘get a translation to read’, whereas editors understood the same CTA to mean ‘edit a translation’.
- The ‘why’ provides context relevant for evaluating a decision around engagement. Contributors want to know why they’re seeing a new option.
Motivation
- A successful entry point offers a match in terms of interest, ability, and time. The potential user must feel motivated by the topic and confident in their ability to perform the task, as well as able to perform the task in the time they have available at the moment.
- Perceived readership and editor interaction is a strong motivator. “People love to expand articles that are read by many people.”
- Addicted to errors (of the right size). For someone inclined to participate, an entry point presented at a moment of perceived need for improvement is a strong pull.
Action
- Getting around a mismatch in timing (lack of time). The ‘problem’ of not enough time may really be a mismatch in timing between an editor’s availability and the entry point opportunity. Editors want the option to bookmark tasks and ‘save for later’.
- Surface relevant follow-ups. Tasks feel easier if they’re logical continuations of a related task just completed. For example, translating an edit to another language or expanding an article for which the editor has just improved the translation of another section.
Footnotes: