Task 3.

3 Query App Match (YouTube Searches)

Mobile App and Query Evaluation for YouTube Searches

Version: 2017-04-04

In this task, you will research a mobile app and a query entered in YouTube. You will
then answer a question about how well the app fulfills what the user is looking for.

Terms

• You already know what a query is: it is a word or phrase that a real person typed
into a Google search. However, in this evaluation, we will be talking about
queries within YouTube.
• A mobile device is a small, handheld computing device, such as a smartphone or
tablet.
• A mobile app is a smartphone or tablet application.

How to rate this task

Step 1: Review and understand the user query.

You will see a query that a user entered. Just as in normal search ads rating, it will be
your responsibility to do your best to understand what the user was looking for.
However, unlike the normal search ads rating, it is important to keep in mind that the
query was entered on YouTube.

Click the query to open the YouTube results page, and research to understand what the
user was looking for. You may perform additional research outside of the YouTube as
well.
Unrateable Query

If the query is in a foreign language, is incomprehensible, or otherwise can’t be rated,

select the “Unrateable query” flag and submit the task.

Step 2: Familiarize yourself with the mobile app.

Click the button labeled “research mobile app” to open the app’s Google Play page,
then review its content. You should consider all the information available to you in order
to inform your understanding of the app, including the text that describes the app,
ratings, any relevant comments from users, and even your overall impression of how
professional and trustworthy the app looks. Some additional notes:

• Do not download the app.

• The Play Store page may include some additional content, such as links to other
parts of the Play Store, or lists of “Similar” mobile apps. Please ignore this and
only focus on the content that is related to the main app on the page.
• You may, however, follow links and do additional research outside of the Play
Store if it helps you understand the app.

Unrateable Apps

If the Play Store page or the app it describes is in a foreign language, is

incomprehensible, or otherwise can’t be rated, select the “Unrateable app” flag and
submit the task.
Personal Knowledge of Apps

If you've personally installed and used the app, or believe you have additional
knowledge about the app that would affect your ratings in this task, select the flag
labeled "I am personally familiar with this app." You will still be required to complete the
task.

Step 3: Answer the following question: Would installing and using the app satisfy the
user intent?

Think about what the mobile app is used for and what the user was looking for. There
will inherently be a mismatch between searching on a video site and what apps can
offer. For example, if the user is searching for a [baked alaska recipe], the user wants to
see video tutorials on making Baked Alaska. An app of dessert recipes would probably
be Satisfaction Possible/Satisfaction Likely, whether or not it includes videos. Pay
attention to the app’s quality as well. A good quality app looks trustworthy, has a well-
written description that makes the app’s purpose clear, and gets positive reviews from
users. The user intent is less likely be satisfied by a poor quality app.

Taking all of these factors into consideration, select a rating from [-100] to [100] to
indicate the likelihood that a specific user would be satisfied by installing and using the
app.

Rating Description Examples

Satisfaction Likely The app provides exactly the User query: [how to file taxes]
functionality that the user is looking
for, even if the app is a competitor App: TurboTax
app. The app is of good quality, and
 the user intent would be satisfied by The user is looking for a
installing and using it. tutorial/information on how to file
taxes, and is provided an app that
guides a user through filing taxes.

User query: [formation beyonce]

App: Spotify

The user's YouTube search is

ambiguous because the user may or
may not care about watching the
music video. If the user is trying to
find a way to listen to the song, the
Spotify app can satisfy that need.

User query: [learn spanish]

App: Duolingo

The user is trying to find a Spanish

language tutorial and the app
provides resources to learn Spanish.

Satisfaction Possible The app provides similar functionality User query: [full episode of fargo
to what the user is looking for. The s1e4]
user intent could be satisfied by
installing and using the app because App: Hulu Plus
it is of generally good quality, even
though some of its features don’t There is no app that could possibly
match what they are looking for. receive "Satisfaction Likely" for this
query since the user is likely looking
for illegal free content on YouTube.
Even though the user's query cannot
fully be satisfied in a legal way,
installing and using the Hulu Plus
app will mostly satisfy the intent.

User query: [weight loss tips]

App: Lose It!

The user is looking for a video that

contains weight loss tips and the app
has strategies for setting and
accomplishing weight loss goals, but
is more of a fitness/calorie tracker
than a repository of tips.

Dissatisfaction The app provides features that are User query: [macaroni and cheese
Possible related to what the user is looking recipe]
 for, but they’re unlikely to be satisfied App: Whole Foods

by installing and using it because it’s The user is looking for visual tutorial
not a very good match and/or of how to make macaroni and
because the app appears to be of cheese. The Whole Foods app has
low quality. some recipes and coupons for the
grocery chain that could help the
user move closer to his/her goal, but
only a small subset of users would be
satisfied with this app
recommendation.

User query: [vinyasa yoga class]

App: MindBody

The user is looking for a yoga class

to follow along with, and the app is a
yoga class reservation system. While
the yoga reservation system can help
the user find a yoga class, only a
small subset of users would be
satisfied with this app.

User query: [eyebrow shaping

tutorial]

App: Makeup Genius

The user is looking for a tutorial on

shaping eyebrows and the app
allows the user to apply different
makeup looks on him/herself. Only a
small subset of users would be
satisfied with this app.

Dissatisfaction The app is unrelated to the user User query: [dogfight plane]
Likely intent and/or the app is of obviously
bad quality. The user intent will App: Modern Dog Fighting
definitely not be satisfied by installing
and using the app. The user is looking for videos of
aerial combat, not a gaming app.

User query: [donald trump press

conference]

App: Driving President Trump

The user is looking for a press

conference video of Donald Trump,
not a gaming app.
If you have questions about this project that are not answered by the instructions above,
please review the Rater Hub first, which contains additional content about rating. If you
encounter a technical problem with this rating task, use the “Report A Problem" link in
the lower-right hand corner of the rating page.

Task 3.3 Screenshot