Exploring the Diversity of Music Experiences for Deaf and Hard of Hearing People

Abstract.

Sensory substitution or enhancement techniques have been proposed to enable deaf or hard of hearing (DHH) people to listen to and even compose music. However, little is known about how such techniques enhance DHH people’s music experience. Since deafness is a spectrum – as are DHH people’s preferences and perceptions of music – a more situated understanding of their interaction with music is needed. To understand the music experience of this population, we conducted social media analyses, both qualitatively and quantitatively, in the deaf and hard of hearing Reddit communities. Our content analysis revealed that DHH people leveraged sign language and visual/haptic cues to feel the music and preferred familiar, non-lyrical, instrument-heavy, and loud music. In addition, hearing aids were not customized for music, and the visual/haptic techniques developed were not widely adopted by DHH people, leading to their suboptimal music experiences. The DHH community embodied mutual support among music lovers, evidenced by active information sharing and Q&A around music and hearing loss. We reflect on design justice for DHH people’s music experience and propose practical design implications to create a more accessible music experience for them.

1. Introduction

Music provides rich affordances, such as delicate emotional experience, enhanced connections during social events (Elor et al., 2021), and the potential for enhancing cognitive abilities (Schellenberg, 2005). The main barriers to an optimal music experience for deaf or hard of hearing (DHH) people are their limited access to the audio features of music (Silva et al., 2020). Children with cochlear implants (CIs) showed the ability to perceive emotions (e.g., happy vs sad) in music but did so less accurately than typically hearing peers (Hopyan et al., 2011). Interviews with musicians with hearing loss revealed a low satisfaction with digital hearing aids (HAs) – pitch and timbre distortion was often reported to compromise music listening experiences (Fulford et al., 2015).

Music and music education should be made more accessible for DHH people (Wright, 2014; Kyriakou, 2022), who can benefit from music – music has been shown to improve the cognitive performance of deaf children (Rochette et al., 2014) and help them acquire new vocabularies through music therapy (Bassiouny et al., 2017). Accessibility for DHH people is also an important consideration in live music events such as music festivals (Bossey, 2020). DHH people expressed difficulty joining sound-based communal activities such as fan chanting and suggested utilizing visual stimuli such as AR glasses to provide contextual information (Han and Kim, 2023). Assistive music technologies, mostly drawing on haptic (Fahey and Birkenshaw, 1972; Cavdir and Wang, 2020; Jack et al., 2015) and visual (Deja et al., 2020; Grierson, 2011; Kim et al., 2015) cues have been designed for DHH people. However, how DHH people experience music, potentially with the help of such assistive technologies, is not well understood.

Toward bridging this research gap, we analyzed 535 music-related posts/comments in the r/deaf and r/hardofhearing communities on Reddit. The haptic and visual music technologies developed in academia were not widely adopted in our sample. The Reddit users mostly drew on their own experiences and skills, as well as external help from the community, to optimize their music experience, e.g., drawing on sign language and visual/vibrational cues, listening to familiar and specific types of music, and sharing information about accessible music and devices. They still experienced a wide range of accessibility challenges with music such as mishearing lyrics and a lack of captions in online videos and movies. After presenting a situated understanding of the experiences and challenges of DHH people in music through social media analyses, we provide suggestions for contextual, affordable, technology-assisted designs to create a more accessible music experience.

2. Related Work: Assistive Music Technologies for DHH People

DHH people have limited access to auditory information, thus they need to rely on other senses to feel the music. Researchers have called for multimodal music streaming experiences for DHH people (McHugh et al., 2021). Visual and vibratory stimuli are the most cited methods of experiencing music by DHH people themselves (Watkins, 2017), sometimes used together (e.g., (Petry et al., 2016a)). Below, we elaborate on assistive music technologies that leveraged these two senses.

2.1. Visual Music

DHH people use assistive technologies such as sound visualization and speech-to-text to improve sound perception via a combination of auditory and visual means (Ohshiro and Cartwright, 2022). Fourney and Fels explored music visualization to help hard of hearing (HoH) music consumers understand the emotions conveyed by music (Fourney and Fels, 2009) – they used a piano roll-up to represent notes and a moving ball to represent note lengths. Deja et al. presented an on-screen visualizer that generated effects alongside music – it was shown to effectively enhance the musical experiences of DHH people (Deja et al., 2020). Grierson developed an interactive audio-visual performance system for deaf children’s music interaction (Grierson, 2011). Kim et al. demonstrated tangible music visualizations that leveraged items children saw in their daily lives, such as flowers and plants (Kim et al., 2015).

Lyrics were perceived by d/Deaf people as one of the central elements in a song – song signing augmented their musical experiences with an additional level of accessibility (Yoo et al., 2023). Maler argued that “d/Deaf song signers embody music differently than the hearing by creating a visual, kinetic form of music in sign language, rather than using sign language to express something about sound” – deafness may thus become a source of musical ability rather than a source of impairment (Maler, 2015).

2.2. Haptic Music

Perceiving music by touch has been extensively investigated (Fahey and Birkenshaw, 1972; Cavdir and Wang, 2020; Jack et al., 2015; JACK et al., [n. d.]; Tranchant et al., 2017; Araújo and Batista, 2016; Yan and Liu, 2022; Chao et al., 2018; Remache-Vinueza et al., 2021; Aker et al., 2022; Baijal et al., 2012), with pitch, rhythm, loudness, and timbre mapped to tactile vibrations. Displaying emotionally resonant sounds as vibrotactile feedback could produce distinct emotional responses (Sion et al., 2023).

Rhythm is often the first musical concept DHH people learn in music classes. Florian et al. developed a prototype to generate light and vibrotactile outputs from music rhythm for deaf people (Florian et al., 2017). Providing ad-hoc, real-time rhythm information to a deaf user was deemed important (Petry et al., 2016b). Deaf people were even able to collaborate with hearing people to create short beats with the help of sound visualization and haptic feedback (Søderberg et al., 2016).

Haptic sense is also helpful in delivering music as a whole to DHH people. Karam et al. presented a sensory substitution technique that delivered music as discrete channels of vibrotactile stimuli, perceived positively by Deaf senior citizens (Karam et al., 2009). Nanayakkara et al. combined a “Haptic Chair” and a computer display of visual effects to enhance DHH people’s music experience (Nanayakkara et al., 2013, 2009). Trivedi et al. aimed to develop an affordable wearable haptic device, which turned out to be vibrotactile sleeves with bone conduction speakers (Trivedi et al., 2019). Audio-tactile music interactions were further integrated into immersive VR music videos (Young et al., 2023). Deaf children felt more confident when wearing a visual and vibrotactile music-sensory substitution device (Petry et al., 2018). Musical vibrations also played an important role in music therapy with D/deaf clients (Palmer and Ojala, 2022).

Further, accessible Digital Musical Instruments (ADMIs) have been extensively explored in inclusive music practice, with an emphasis on tangible or physical controllers (Frid, 2019). Iijima et al. prototyped a “smartphone drum,” a smartphone application that presented a drum-like vibrotactile sensation when users used smartphones as drumsticks and made drumming motions in the air (Iijima et al., 2021). End users’ preferred gestures when using smartphones to simulate instrumental experience were identified (Iijima et al., 2022).

2.3. Research Gap

Despite the fruitful designs of assistive music technologies, as shown above, they were not widely adopted by DHH people – their expressed barriers to technology included unknown availability and limited options (Ohshiro and Cartwright, 2022). Limited research has been conducted to understand how DHH people practically experience music and leverage assistive technologies (e.g., hearing aids, headphones, advanced haptic/visual devices) to enhance their music experience. In this paper, we explore the diverse, rich discussion in DHH communities on Reddit to add to this literature.

3. Methodology

3.1. Data Collection

We searched both the r/deaf and r/hardofhearing subreddits in October 2023 using the keyword “music.” We examined the top 40 posts (by relevance), 20 in each subreddit, ranging from 8 years old to 1 month old by the time of the search. The posts concerned general music experience (e.g., “How do you feel about music?”), music recommendations for DHH people (e.g., “Music for deaf kids?”), device recommendations (e.g., “Deaf on left ear looking for BT in-ear headphone for iPhone and music”), ways to better experience music (e.g., “How do you feel music through vibrations?”), accessibility challenges or features (e.g., “App that adds subtitles to many apps”), and medical questions (e.g., “hearing loss from loud music?”). Comments under each post were collected, leading to 352 and 183 comments, respectively.

3.2. Quantitative Analysis

To understand the sentiments and themes prevalent in the DHH communities’ discussions about music, we applied topic modeling and sentiment analysis to the collected data. Comments that were off-topic, contained spam, or lacked substantive content were not constructive to the analysis. To ensure the relevance and quality of the quantitative analysis, we implemented a filtering process. We evaluated the comments’ relevance to the DHH community’s interaction with music. Meaningless or low-effort posts, such as those with minimal textual content or unrelated to DHH people’s music experience, were filtered out by the authors. After this filtering process, our final dataset for the quantitative analysis comprised 291 posts.

3.2.1. Sentiment Analysis

Sentiment analysis was conducted using TextBlob, a library offering a straightforward API for natural language processing tasks (Diyasa et al., 2021). TextBlob computes sentiment scores by evaluating the polarity of words in the text, with scores ranging from -1 (highly negative) to +1 (highly positive). These scores facilitated the classification of sentiments into three categories: negative (scores less than -0.01), neutral (scores between -0.01 and 0.01), and positive (scores greater than +0.01).

• Negative Sentiment: Text with scores within $[-0.1,-0.01)$, reflecting expressions of dissatisfaction or concern. Examples are shown below,

  “I don’t listen to songs, I don’t have even one song on my iPhone, I listen to other people’s songs probably once or twice a year. I roll my eyes at those who sign songs on YouTube (though there’s a gem sometimes). I sometimes find music annoying, actually. It’s just nonexistent in my life. The same cannot be said for other Deaf people – I’ve seen a lot of Gallaudet students listening to music, they probably have milder hearing loss and can relate to music, whereas I am profoundly deaf and I see no point in music.”

  “My dad always blasted the radio. I don’t fully understand how this is marked ’deaf/hoh with questions’ if you don’t understand how this works?”

• Neutral Sentiment: Text with scores within $[-0.01,+0.01]$, denoting a balanced or indifferent emotional tone. Examples are shown below,

  “HoH here. I dont hear bass and I cant make out lyrics. Recorded music i use bone conductors, and live music i listen by feeling vibrations. If i am in a restaurant or bar or something with music in the background, i probably cant make any of it out, if i detect it at all”

  “I’m profoundly deaf, however, I like to turn on the volume to its max and blast my music. I love feeling the bass. I prefer to use music videos from YouTube so I don’t look like an idiot blasting radio. Funny story - when I was a teenager riding in a deaf friend’s car blasting ‘music,’ and pulled up to my hearing parents, they came out and said we were listening to Oprah’s Talk Show. I like dance/house because it has such a strong bass.”

• Positive Sentiment: Text with scores within $(+0.01,+0.1]$, indicating a positive or affirmative disposition. Examples are shown below,

  “Damn thats awesome! I have no info about it but its really cool and nice and i hope you find help for this :)”

  “I listen to music quite a lot, in different genres and have done for most of my life. It makes me feel a variety of emotions like hearing people.”

To visually assess the sentiment distribution, we utilized histograms and count plots. Histograms showcased the overall sentiment tendency, while count plots provided a frequency comparison of the sentiment categories.

3.2.2. Topic Modeling

We began by pre-processing the text data from our dataset, including tokenization, normalization, and lemmatization using NLTK’s WordNetLemmatizer. This process also involved the removal of stopwords to enhance the relevance of our feature set. We then vectorized the processed content using TfidfVectorizer, which transformed the text into a TF-IDF matrix – a numerical representation that reflected the importance of words within the corpus. K-Means clustering was chosen for its effectiveness in partitioning the words into ten distinct clusters, each potentially representing a topic. By fitting the TF-IDF matrix into the K-Means model, we identified centroids for each cluster, which were indicative of the central themes of each topic. To interpret these clusters, we extracted the top ten keywords for each, using the centroids to locate the most defining words within our TF-IDF feature space. These keywords served as a proxy for the topics discussed within the community.

3.3. Qualitative Analysis

After uncovering the nuanced and diverse nature of DHH communities’ discussion about music, music technology, and hearing impairments, we further employed a collaborative thematic analysis (Braun and Clarke, 2012) to gain more in-depth insights into the posts and comments. Two researchers performed the analysis independently, generating themes and subthemes, and regularly discussed to reach a consensus. The three overarching themes we summarized were “Understanding DHH People’s Music Experience” (Section 4.2.1), “How DHH People Enhance Their Music Experience” (Section 4.2.2), and “Information Sharing Within DHH Communities” (Section 4.2.3). Within these themes, subthemes emerged. For example, under “How DHH People Enhance Their Music Experience,” subthemes included “familiar music,” “non-lyrical, instrument-heavy, and loud music,” “ASL (American Sign Language),” “visual music,” and “haptic music.” A theoretical saturation was reached after we coded 10 posts in each community, i.e., no new themes emerged. We used XMind, a mind-mapping tool, to organize different levels of themes and quotes into a hierarchical structure. In the paper, we use raw quotes from Reddit users to illustrate our findings. Notably, HoH people tended to refer to their hearing impairments as “hearing loss.” Otherwise, the differences in the discussions between the deaf and HoH communities were not obvious. Unless explicitly differentiated, our findings apply to DHH people in general.

4. Findings

4.1. Quantitative Results

4.1.1. Sentiment Analysis Results

We utilized a histogram to demonstrate the distribution of sentiment scores, as shown in Figure 0(a). The analysis uncovered a slight right-skewness in the sentiment distribution, with the bulk of sentiment scores clustering around the neutral to slightly positive range. This skewness towards a more positive sentiment indicated a generally favorable dialogue within the DHH community regarding their music experiences and their use of assistive technologies. Despite the potential challenges that the DHH community may face, the prevailing sentiment captured in the dataset suggested a predominance of positive expressions, which could be reflective of positive personal experiences, supportive initiatives, or progress in technology that enhanced music accessibility.

Moreover, as depicted in Figure 0(b), we categorized the sentiment of the posts and comments into three distinct groups: negative, neutral, and positive. The count plot illustrated a predominant number of texts with positive sentiment (N=225), significantly outnumbering those classified as neutral (N=23) or negative (N=43). This distribution again suggested that the majority of the community’s discourse was optimistic or constructive when discussing music experiences. A smaller, yet notable number of texts were categorized as neutral, indicating a balanced or impartial view. Meanwhile, the posts with negative sentiment represented a less common sentiment category, which may point to the few instances of dissatisfaction or adverse experiences within the dataset. This visual representation reinforced the notion that, while challenges existed, the overall sentiment within the community conversations was largely positive.

The sentiment analysis underscored the complexity of the emotional responses within the DHH community. While the histogram in Figure 0(a) revealed a slight right-skewness in sentiment scores, indicating a tendency towards positivity, the count plot in Figure 0(b) further corroborated this finding by showing a predominant number of posts with positive sentiment. These optimistic expressions significantly outnumbered the neutral and negative categories, suggesting that the majority of the community’s discourse was constructive when discussing music experiences and the use of assistive technologies. Despite this, the presence of negative sentiment, albeit as a less common sentiment category, highlighted that certain aspects of the DHH community’s experiences did warrant attention.

(a) Histogram of sentiment scores.

(b) Count plot for different sentiments.

Figure 1. Visualization for sentiment analysis results.

4.1.2. Topic Modeling Results

We employed a K-Means clustering approach on the TF-IDF vectorized content of the DHH community’s Reddit discussions to discern underlying themes. Ten clusters were identified, with each cluster representing a distinct topic prevalent within the DHH communities’ discussion of music.

The top keywords from each cluster offered a glimpse into the thematic focus areas, as shown in Table 1. For example, Cluster 0 highlighted discussions centered around hope, understanding of the brain’s interaction with music, and general knowledge about deafness and hearing; contrastingly, Cluster 1 was characterized by practical discussions about hearing devices such as headphone and AirPods. In clusters focusing on “Music Experience” (Cluster 3 and Cluster 7), terms such as “enjoy,” “loud,” “song,” “lyric,” and “bass” indicated a rich dialogue on personal interactions with music, highlighting both the sensory aspects of music and the emotional connections to music. Clusters like Cluster 2 and Cluster 6, which were enriched with keywords like “hearing,” “aid,” “loss,” “audiologist,” and ’‘Bluetooth,” reflected a technological slant, discussing the use of hearing aids and other assistive technologies, as well as the implications of hearing loss. Cluster 4 and Cluster 8 delved into the emotional and social dimensions, with words such as “care,” “appreciate,” “health,” “deaf,” “people,” “love,” “friend,” and “story.” These clusters seemed to capture the community’s sentiment towards hearing health and the social aspects of being DHH, including the shared stories and relationships that formed around music. The remaining clusters, such as Cluster 5 and Cluster 9, contained terms that suggested a focus on practical solutions and the sensory experience of music, respectively. Words like “mono,” “sound,” “channel,” “feel,” and “loss” seemed to point towards a dialogue on the technical adaptations for mono hearing and the tactile experience of music.

Overall, the K-Means clustering and the examination of the top keywords in each cluster provided a comprehensive thematic map of the discussions within the DHH communities on Reddit.

Table 1. Top 10 keywords in each cluster from topic modeling.

Cluster	Top 10 Keywords
0	would, brain, music, hope, think, know, deaf, hear, find, hearing
1	headphone, call, airpods, pretty, charge, clarity, well, single, might, work
2	hearing, aid, music, bluetooth, sound, loss, without, ear, car, use
3	music, hoh, im, still, always, enjoy, loud, thing, loved, lyric
4	know, hearing, dont, care, sometimes, appreciate, music, better, health, loss
5	left, right, check, solution, bit, mono, sound, perhaps, combined, channel
6	time, get, audiologist, better, make, noise, bad, going, one, good
7	song, hear, bass, lyric, like, music, playing, instrument, play, im
8	music, deaf, listen, people, dont, love, friend, story, party, find
9	music, sound, like, im, hearing, ear, feel, would, help, loss

4.1.3. Interpretation

The combined sentiment and topic analyses provided a concise yet comprehensive portrayal of the DHH community’s online discourse around music. Sentiment scores varied yet showed an inclination towards positive emotions when discussing music experiences. The topic modeling unearthed a nuanced facet of the community – a collective that not only confronted difficulties but also celebrated the joys of music and music technologies. The presence of both positive and negative sentiments underscored the complexity of the DHH experience and highlighted essential considerations for the design of assistive technologies. The results from the sentiment analysis and topic modeling painted a picture of a community actively navigating the intersection of music, technology, and hearing impairments.

4.2. Qualitative Results

4.2.1. Understanding DHH People’s Music Experience

Hearing people may have biased assumptions about DHH people’s music experiences, which annoyed them constantly in the Reddit discussions. It is important to learn and understand DHH people’s music experience instead of imposing assumptions. Both DHH people’s diverse music experiences and their accessibility challenges with music are discussed below.

DHH Individuals’ Diverse Music Experiences. The DHH community was annoyed when people without hearing impairments held biased assumptions about DHH people’s experiences, preferences, or attitudes toward music. Assuming every DHH person liked music by default was rude and regarded as a hearing-centric perspective. For example, when a hearing user asked what album to buy for their deaf niece as a gift, a deaf user argued that not every deaf person would eventually listen to music, and it was nicer to understand deaf people’s world, instead of trying to integrate them into the world of people without hearing loss, “You have an opportunity to enter into a new understanding of a world and a culture that is hers by rights. It’s a rare and truly beautiful thing if you let it be. Rather than rushing to integrate her into your world (one in which she will always be an outsider), strive to first learn hers.” When a Reddit user asked if deaf people listened to music, a member of the deaf community argued that music experiences and preferences varied among people and that deaf people should not be treated as one, “We are not a monolith. You should just ask them! I for one don’t enjoy music at all. I know there are deaf people out there [who enjoy music], and one in particular has a YouTube channel. You should find an actual deaf person and ask them what a deaf space might look like.” Similar annoyance occurred when hearing people assumed all DHH people had the same music preferences, “I kinda get what you are asking, but would ever pose the question ‘what sort of music do POC [people of color] like? Or what kinds of music do Jewish people like?’ Can you see how it reduces us to the sun of all people, rather than letting us feel like individuals which is a big part of the reason we are in these spaces?”

Deafness is a spectrum, which may lead to different music experiences. One deaf user elaborated on this opinion, “It’s important to note that deafness is a spectrum, and not everyone who is deaf has no hearing. Everyone has different levels of how much and what they can hear… I have heard of profoundly deaf people who enjoy the vibrations of music. I am deaf or hard of hearing, and I can hear music as long as it’s loud enough.” Another user echoed this view, “There’s a huge range of hearing levels – ranging from a mild hearing loss to hearing nothing. Deaf (and hard-of-hearing) ppl are not a monolithic group. We are regular individuals with different tastes and preferences.” Compared to deaf people, HoH people tended to enjoy music more. Some of them even became professional musicians, with help as simple as good headphones, “I’m an active musician despite my hearing loss! I’ve released a few albums and I play bass. I find that with technology I don’t have much of a problem; I just have to keep the volume of things up pretty high and have good headphones. And learning about things like waveforms so I can visually identify any issues too :)”

DHH people’s preferences for music can be contextual. For example, music was not wanted at parties, where people needed to talk, “Basically, listening to what people are saying takes a lot of energy but it doesn’t discriminate easily if a radio is playing in the background. I can’t just tune it out, it’s just as present as what someone is saying.” Several DHH people described listening as work for them. Having music as the background made listening and conversing much more difficult. When a hearing user planned to hold a party for deaf people and asked about music choices, a deaf user explained how music made the party “deaf-UNfriendly”: “ If we wanted to use speech to text to talk with someone, the music would interfere. If we wear assistive listening devices, the music will be distorted, staticky, etc. You came at this from the wrong direction. You’re having a party, not a concert, and party suggests socializing more than enjoying music. You have to make the socializing aspect deaf-friendly, not the music taste.”

Further, some DHH people preferred to enjoy music alone instead of treating it as a group activity, given their extra challenge with music, “I seldom enjoy music with others. I’m okay with bopping along to a familiar song, but I loathe feeling like I have to perform. It’s likely because as a kid, I was forced to participate in music class when I knew I couldn’t follow along and sang gibberish. If I feel motivated to pretend I’m Slash with the air guitar, then I will, but if not, I’m not budging. Music is not supposed to be a chore.”

Accessibility Challenges with Music. Hearing loss was a barrier to effectively listening to or enjoying music. Some DHH people gave up music which became random noise to them. This user was an example: “After losing my hearing and getting a cochlear implant, music has become dull, mechanical, and most of all, indistinguishable from a wall of aluminum cans crashing to the floor. None of it holds any meaning to me anymore. So, I don’t bother with music anymore.” Those who were profoundly deaf may find even more difficulty listening to music, “I don’t listen to songs. I don’t have even one song on my iPhone. I listen to other people’s songs probably once or twice a year. I roll my eyes at these who sign songs on YouTube (though there’s a gem sometimes). I sometimes find music annoying, actually. It’s just nonexistent in my life.” DHH people who did not enjoy music may nevertheless use music for other personal purposes, e.g., to block noise from colleagues at work, “I don’t give a shit [about music]. While I think some songs can be lit af, I can’t pick out one particular song by sound alone. I do listen to it a lot at work, but that’s to block out the people laughing too loudly near my cubicle.”

Accessibility infrastructure did not always exist to help DHH people listen to music. Some people complained that contextual caption was often missing in movies or online videos when music was played as the background or as a hint for the atmosphere. Without contextual, visual (i.e., textual) descriptions of music, DHH people may find difficulty understanding the messages conveyed by music, especially if the music was not loud, “Just [music playing] is lazy. Especially since most of the time doesn’t differentiate between music in the story and out of the story and doesn’t tell you what kind of music is being played… If a character enters a house and hears piano, then it’s valuable information. There’s probably someone in the living room playing piano. If it’s loud music then you get why the character is annoyed. Even atmospheric music is useful if the kind is given.” An HoH user talked about interpretation challenges brought by missing captions for YouTube and TikTok videos, “If a YouTube video has no captions, I usually can’t watch it :( I also just don’t use TikTok because it’s so hard to understand most of the videos.”

Even slightly more detailed captions for music might be of help, “Well, I like ‘ominous music plays’ better than ‘music playing”’ Netflix was mentioned as a good example of providing contextual captions of music, “If the song has any importance to the plot, I think descriptions are awesome. Grey’s Anatomy is especially good about that, they even make sure to put the title of the song, the artist, and the lyrics. But if the song is only there for background music, a simple ‘Soft music starts’ is fine if it seems like it is anticipating something, or just omitted if it’s just there to fill in silence.”

Several HoH users recommended the live captioning function in iOS and Google Live Transcribe. One of them raved about the accessibility features provided by Google Pixel and Apple, “The Pixel were the very first to come out with Live Captioning, Live Transcribe, Sound Notifications (let you know if a baby is crying or an alarm is going off), Call Hold (Alert you when the call hold is done), and Direct My Call (listen to the automated voice menu and lists the options for you). All of these features are on the Pixels first and then sent out to other Android phones in later OS updates. Apple will copy them and improve on the features. Win/win for all of us! I think the Pixels are the best for the Deaf/HoH who use their voice to communicate with hearing people. A huge kudos for Apple to caption FaceTime and Google to caption Duo & Meet!” Another HoH person shared how Airpods compensated for their hearing capability, “I have around 40dB loss, 60/70 as you get towards the higher frequencies, and I find that the airpods are fantastic at filling in the gaps, and the pros make phone calls way easier.” More advanced smartphones tended to provide better captioning performance, “For Android users, I’ll recommend getting the Google Pixel 6 or higher. The new Tensor chip really pulls its weight in providing a more accurate captioning even if the phone is in airplane mode.” This might impose additional challenges on socioeconomically disadvantaged smartphone users when they try to access captions for music.

4.2.2. How DHH People Enhance Their Music Experience

DHH people in our sample did not extensively use music technologies developed in academia but used their own heuristics to listen to music, such as sticking to familiar music, actively choosing non-lyrical, instrumental-heavy, and loud music, relying on ASL, and leveraging visual and haptic cues.

Familiar Music. DHH people often had to listen to music multiple times to pick up the tune, “I have to listen to something multiple times before I start to pick up the tune. So live music still sucks.” Due to the difficulty of picking up new music, they tended to listen to music that they were familiar with, “If I have heard the song before and know the lyrics and the music, I can hear it better.” Even a semi-pro musician shared this habit, “I’m HoH (moderate to severe). I have my bachelor’s degree in music theory and am a semi-pro musician. However, I have a complicated relationship with music. People are often surprised, given my involvement with and knowledge of music, that I don’t listen to that much music… If I do have music on in the background, it’s almost always music I’ve heard many many times and is extremely familiar.”

Non-lyrical, Instrument-heavy, and Loud Music Is More Accessible. Instrument-heavy and loud music was favored by most DHH people, and deaf people in particular liked non-lyrical music.

Recognizing words was challenging for deaf people. Thus they preferred non-lyrical, non-vocal music such as electronica, “Well with hearing aids most of the time you could hear it like anyone else but to me as near deaf I can still hear the beat and I enjoy the music words not very much and maybe instrument of loud enough but for me I just hear the beat and honestly like that.” Another deaf person also expressed that lyrics were intelligible and they preferred instrumental types of music, “ In my case, I can hear most instruments fairly clearly, but the lyrics of course are unintelligible. I tend to prefer instrumental types of music for that reason - mostly chiptunes/synthwave, but also some jazz or lofi hiphop at times.”

Instrument-heavy music could be better felt by DHH people, e.g., “I tend to prefer bass-heavy music since I can actually feel it and move along when in a club or party setting.” “I find that music with strong percussion sounds best in my CI.” “Rap music will have good bass. What are you planning on using for streaming it? Increase the bass and it will vibrate with most songs.” One person preferred instrumental music and noted that they were only able to fully enjoy lyrical music if the lyrics were provided, “I listen to mainly instrumental music myself. Have to listen in a quiet room since my ‘ears’ will pick up outside sounds… I do like some pop/country/bluegrass but need the lyrics in front of me to fully enjoy.” Another person directly asked for suggestions for music with bass, “Hey, my friend has some music clips that plays when chilling with his friends. I was thinking, is there some music that’s just loaded with good rhythmic bass? Sorry if this is a dumb question, and thanks if you could share a playlist or some music.”

Loud music was more comprehensible, thus DHH people tended to “blast music” or “blast speakers/radios.” One user described their preferred level of loudness, “This is individual, of course. But generally loud. When I was at Gallaudet, the parties in a specific dorm were so loud hearing students often would not live there. The walls and furniture would shake from the sound.” Some played music loud to feel the bass, “I’m profoundly deaf, however, I like to turn on the volume to its max and blast my music. I love feeling the bass.” Loudness helped HoH people better understand lyrics, “The more vibrations from the music the better if I cannot make out the lyrics.” Another HoH person attributed the effect of loudness to the association between vibrations and sound in memory, “What I do now is annoy anyone around me by insisting on turning up the bass as high as I can because although I can’t hear it, I can feel it and it is nearly the same to me now. I feel like I’m hearing it because the vibrations take the place where the sound once would have been in my brain, is my theory, but most people tell me it ruins the song and vocals. To me, it makes it sound like it used to sound!” Even those who did not like loud music acknowledged that easy-listening music made subtle changes in pith undistinguishable, “I seem to have no tolerance for loud fast songs with a lot of clutter and noise. Most of the stuff I listen to is easy listening. But it can’t be too easy because I don’t distinguish between more subtle changes in pitch…”

Playing music loud often annoyed family members or neighbors. The following example illustrated how a HoH photographer negotiated between having superior music experience and not bothering her husband, “I almost only get to listen when I drive and then I get to have it at a level that I can hear it. My husband says it’s too loud for him. I have one speaker and it sits on a part of my desk about ten inches from my hard-of-hearing side. I always ask if it’s too loud, because our dining room (where my desk is) and the living room are open to each other. I keep my music quiet enough to not drown out his TV and he keeps the TV low enough that it doesn’t drown out my music.” The “rude loud deaf neighbor” discussion often occurred. DHH people agreed that it was okay to request a volume adjustment when one’s deaf neighbor played music too loud. They also suggested headphones that reserved optimal music experience while not annoying neighbors. Some showcased how they dealt with this dilemma, e.g., reaching a consensus on socially acceptable “loud hours” with neighbors, “Completely deaf here. I have some very loud hobbies like DIY, metal bashing, cutting up wood with power saws for my carpentry/repair hobbies, drilling holes in walls, etc. I ALWAYS check with my neighbors over what they consider socially acceptable hours for noisy hobbies and try to stick to them.”

The Reddit users enthusiastically shared songs, singers/bands, albums, and Spotify playlists that were accessible to deaf people, often with the above-mentioned features, e.g., “Most songs by RHCP since Flea is an absolute legend on the bass.”

ASL. DHH people tended to use sign language to interact with music and each other, “I often will tell people that ASL is loud…visually. When there is a party or an event with a large number of Deaf people, I don’t ‘miss out’ on music because there is music playing with people’s hands.” Some even hired sign language interpreters to sign music for them at home.

Musical venues were obligated to provide interpreters for DHH people. When a deaf user asked how they could enjoy music at a rock show, another user suggested that the venue was responsible for finding an interpreter for them. The National Association of the Deaf (NAD) had an advocacy letter to ask stubborn venues to comply: “If you’re in the US, the responsibility of obtaining an interpreter rests entirely on the venue and they cannot charge you extra for your request. The NAD also has an advocacy letter for more stubborn venues that try to push back that clearly explains the relevant laws they need to follow. I have a lot of experience with these sorts of requests and the pushback venues will give to us.” Someone else also mentioned that providing qualified interpreters for deaf people was a federal law, “This! And they have to give you a QUALIFIED interpreter. You can’t have your kids/spouse/friend just interpret for you. It’s federal law. I know it’s scary but you’re in the right here. You’ve got this :)” Dropping the law bomb was often seen as a last resort when DHH people went to musical venues.

Visual Music. A lot of DHH users on Reddit mentioned the combination of audio music with visual elements, such as colors and lighting: “I feel like HOH or deaf people are more focused on visual cues since one of your senses is not working or working less. Also I’ve experienced and got to know that HOH/dead people are often way more sensible to things such as light.” “I love when music is paired with lighting personally, or strong bass” One HoH user used visual features to enhance their viola playing experience, “I have bilateral cochlear implants and take viola lessons. I think keeping time involves developing an inner pulse, and learning to use a metronome - especially one with visual feature, is one way to develop that inner pulse.”

In the HoH community, people commonly expressed challenges with lyrics, e.g., mishearing lyrics, “Mishearing song lyrics is a common thing, even for those who don’t have hearing loss. Of course, it’s worse for us. There are even websites devoted to misheard lyrics. Some of the misheard lyrics are hysterical.” For many, the challenges did not disappear even when using hearing aids or cochlear implants. One user gave a vivid example of how lyrics could be misheard, “*Cheese is on the shelf in there.* Then I finally looked at the satellite radio display which was playing the Elvin Bishop song ‘Fooled Around And Fell In Love’.” Many people referred to such mishearing experiences as hysterical or hilarious. Videos showing lyrics with music in the background enabled them to interpret lyrics, although they could not be watched anywhere, anytime, “I’m hard of hearing so with my hearing aids I can hear mostly everything. With them I just enjoy the beats but can’t understand most of the lyrics. But I read the lyrics on YouTube lyrics videos a lot while I’m listening to songs. Obviously I can’t watch the lyrics videos while driving tho.”

Other favorable visual elements included captions and body language. Some people even solely relied on visuals of music, “I find Deaf musicians and watch (not listen) their work.” A Reddit user shared a music video created by a deaf artist, who told the story of social isolation and the ultimate “coming home” to the shared Deaf/HoH community, “It is a testament to our prevailing power and the strength of community.”

Haptic Music. Haptic/vibrational/tactile music was mentioned by multiple people. One HoH user listened to music through vibrations: “HoH here. I don’t hear bass and I can’t make out lyrics. Recorded music I use bone conductors, and live music I listen by feeling vibrations. If I am in a restaurant or bar or something with music in the background, I probably can’t make any of it out, if I detect it at all.” Some were amazed by how others could sing by looking at the mouth and feeling vibrations, “I want to know how to enjoy music after losing my hearing, as a person who once enjoyed it a lot. A curious part I know, there are a few people who actually learn how to speak and sing looking at the mouth and feeling vibrations equivalent to some notes.”

Several academic or industrial projects on haptic music technologies for DHH people were mentioned and linked. One user elaborated on a research project conducted by the University of Liverpool: “Your friend may be interested to know that the University of Liverpool is undertaking research into using Vibrotactile technology to support deaf people in music education, performance, appreciation and production.” An HoH person talked about their satisfactory experience of using a vibrotextile wearable to listen to music at a musical, “I’m HoH and was at a small free event last year put on by Opera Philadelphia that had Vibrotextile wearables from Music: Not Impossible. It was pretty cool.. a vest and a strap on each arm which had many different vibration patterns to experience the sound. I’m not sure where they’re at as far as lending out for use at events but they’re @musicnotimpossible on Instagram and if you Google ‘Music Not Impossible’ you’ll find their site.” However, such devices have not been produced at scale and are not accessible to the DHH community, as evidenced in the Reddit discussions.

Without advanced devices, the music experience created by vibration could hardly be optimal, “No, I can’t clearly comprehend music through vibrations of speakers. If I want to understand a song, I put on my hearing aid or turn up the volume until I can hear it. Vibrations are just the bass, beat, tempo, general sound, or whatever the musical terms are (I’m an idiot when it comes to music).” Higher frequencies were harder to feel via vibrations, “Higher frequencies need to be several orders of magnitude louder/stronger for them to be felt. So while a person can get more from touching the speaker it’s nothing like hearing the music, but if you’ve been deaf your whole life, you wouldn’t know the difference.”

4.2.3. Information Sharing Within DHH Communities

Information sharing and mental support regarding both music and hearing loss are common in Reddit communities.

Information Sharing for Better Music Experience. DHH people frankly shared their suboptimal music experience even with hearing aids or cochlear implants: “Does it help? Yeah, it satisfies my need. Do I hear as well as a hearing person? Nah, definitely not, but I don’t need to hear all the little nuances to enjoy music. If I want to understand lyrics, I read the lyrics (I can’t understand words without seeing them first). I can’t tell instruments apart from each other, but that’s fine. I don’t need to know that.” One HoH person noted that hearing aids were not tailored for music experience, “The main program in hearing aids is adapted to increase speech comprehension, which means the variety of frequencies in music isn’t represented well.” CIs also did not bring perfect music experience for some, “I’m deaf since I was 2 and I also had CI ever since I was 3. Thanks to those 2 years I can speak normally and everything, and my CI’s are absolutely great which makes me able to listen to music almost the same way. I do have trouble knowing what instrument is playing and what notes and if those are high or low. I can actually understand some lyrics but never a lot of it.” Another Reddit user who was not born deaf and had listened to music before they became deaf was frustrated by the current suboptimal music experience, “ Hearing and understanding music, or pretty much anything are 2 different things. If you mean hearing hearing, then yes, I do, if you mean understanding what I’m hearing, then no I don’t. I hear the sound, I just can’t make out any music out of it. Then again I wasn’t born deaf or hearing impaired, so I remember what music sounded like, which is an added difficulty.”

In response, DHH people often shared accessible songs with the community, “I tend to listen with strong vocal songs or using some sort of equalizer to make the vocal stronger. I tend to read the lyrics, seeing if I like it or not. Other times I listen just purely for the combination of the sounds.” In addition to equalizers mentioned in the quote above, DHH people also actively shared other assistive technologies such as hearing aids and music devices like headphones. The sharing happened in either Q&A posts or standalone posts for information sharing. For example, one user shared their special setup on the hearing aid programming that helped make music more sharp and crisp. Another user shared the combination of Apple earbuds and hearing aids. Specific hearing aids such as Phonak Audéo Paradise and Oticon S1 were also recommended. Big-DJ-type over-ear headphones, Bluetooth headphones with ambient sound settings, and specific brands and models such as Dome, 2E1, and sony mx5 were suggested.

Medical Q&A. When one user asked if loud music could worsen deafness, several others provided their opinions and suggestions. One shared their experience of worsened hearing due to loud music, “I didn’t lose my hearing because of loud music but it’s 100% gotten worse because of loud music. Hearing loss is also well known to cause hearing loss according to the Center of Disease Control. I’d find an alternative to the loud music sooner over later if that’s possible.” Another person suggested protecting hearing by using noise-canceling headphones to remove background noise and listening to music at more normal levels. A user who was genetically HoH worried less about loud music, “I am still losing my hearing, but I get plenty of enjoyment from music now just as I did before. I have trouble understanding lyrics and high pitched guitar/piano/punchy drums don’t come through but that’s ok I just turn up the bass.”

A HoH person just got their first hearing aid and experienced noises, preventing them from focusing. Other users provided suggestions based on their own experience, e.g., “The audiologist will likely tell you to use your hearing aid as much as you can, but you can definitely take it off when you’re overwhelmed, or at least lower the volume.” Possible reasons were also provided, “You’re hearing tons of stuff you haven’t heard in a long time, and your brain is going to take some time to adjust. Car noise, the turn signal, light switches, your feet, paper rustling. All of this is going to sound overwhelming. It will get better with time. The more you wear them, the more you’ll get used to it, and it won’t seem so overwhelming. Cars can be particularly bad. They are noisier than most resize.” Someone shared a similar experience, “You’re one day in, so this type of reaction is completely normal. I think it took me a solid 4 months until I could reliably wear my HAs all day long, and I still have days, over 3 years later, that I just have to take them off in some situations.”

5. Discussion

Our analysis revealed diverse experiences and challenges from DHH communities’ discussions around music. DHH people had diverse preferences about music and expressed various accessibility challenges. Assistive technologies developed in academia were not common in DHH people’s discussions; they alternatively used their own heuristics to approach music, such as choosing specific music types, using sign language to interpret lyrics, and relying on haptic or visual cues. Information sharing was common in the DHH community regarding both music experiences and hearing loss. The insights drawn are crucial for developers of assistive technologies and for advocates who strive to cultivate more inclusive music environments. Addressing the challenges reflected in the negative sentiments and discourses around music while continuing to support and enhance the positive discourse that prevails within the DHH community is important. This balanced approach is essential for fostering an ecosystem that not only acknowledges and resolves difficulties but also celebrates and builds upon the positive experiences of the DHH community.

5.1. Design Justice

DHH people’s music experience can be best understood in the context of deaf culture, guided by such questions as “How does society perceive the deaf?” “How do they see themselves in society?” and “How do they identify themselves?” (De Paula and Pederiva, 2017). Both similarities and differences exist between the music experience of deaf people and hearing people – according to Darrow, deaf people interact with music in ways similar to hearing people but less often engage in ritual uses of music (Darrow, 1993).

Having a situated understanding of how DHH people listen to and interact with music is important, and possible. Music students’ knowledge about deaf children’s music experience increased after social, musical, and educational interactions with them – they were able to better relate to the deaf population and showed interest in more knowledge acquisition (Kaiser and Johnson, 2000). Without an adequate understanding, hearing people tended to impose wrong and biased assumptions about DHH people’s music experience, as evidenced in the Reddit discussions. For example, hearing people may assume that all DHH people listen to music by default or that every DHH individual has the same preference for music. The DHH community resisted such assumptions, stating that they had diverse choices and preferences about music and should not be treated as a “monolith.” In addition, deafness was commonly referred to as a spectrum, which led to different music experiences. DHH people’s preferences for music were also contextual – even music lovers preferred not to have music when conversing with others to avoid interference.

Understanding DHH people’s diverse experiences and expectations of music can lead to designs that truly serve their needs. Several DHH people mentioned that they preferred to enjoy music alone; when they were included in group musical activities with other hearing people, they had to pretend to follow along – leading to awkward and frustrating experiences. Thus they may not be willing to collaborate with hearing people to create short beats, as proposed in a previous study (Søderberg et al., 2016). There has only been a small body of literature on understanding DHH people’s music experience. Ohshiro and Cartwright surveyed 50 DHH people to understand their use of technology and the barriers they faced in creative sound activities (Ohshiro and Cartwright, 2022). Their barriers to technology included unknown availability and limited options, echoing our results. Watkins presented a content analysis of online weblogs, vlogs, videos, and articles written by DHH people regarding music (Watkins, 2017) – most discussed topics were music experienced through visual/vibratory methods, music and Deaf Culture, personal fulfillment through music, the importance of music, and preference for musical instruments. We further provide a contextual understanding of DHH people’s music discussions in Reddit deaf and HoH sub-communities, revealing both individualized experiences and community dynamics.

5.2. Design Implications

Hearing loss creates barriers to DHH people’s optimal music experience. Drawing on the literature and our findings, we propose design implications to accommodate their challenges with music.

Building Accessibility Infrastructures. Emerging technologies are often found inaccessible (Zhou et al., 2023). Deafness-related accessibility features are similarly lacking in many popular platforms and media. On Reddit, DHH people often complained about movies or online videos that did not provide captions for music. For example, they found difficulty receiving YouTube music videos and TikTok videos with background music if captioning was not provided. Several DHH users raved about the captioning function provided by some smartphones like iPhone and Google Pixel. According to them, Netflix was a good example of providing contextual captions for music in movies. Other streaming and video-based social platforms are encouraged to follow suit. AI-based music captioning technologies (Manco et al., 2021) may be of help. Music streaming platforms such as Spotify and Apple Music should make sure lyrics are always available for lyrical songs and consider accompanying music with ASL videos.

Tailored Music Recommendation and Generation. DHH people have unique preferences compared to hearing people. For example, they tend to listen to music that they are familiar with. DHH people commonly express a preference for non-lyrical, instrument-heavy, and loud music, which is more accessible to them. Music streaming apps may consider providing customized playlists and music recommendations for DHH users. Further, generative AI models (Dong et al., 2018) can be leveraged to generate music tailored to DHH people’s needs, or allow DHH people to compose music for themselves.

Democratizing Assistive Technologies. Similar to (Watkins, 2017), our analysis revealed the importance of haptic and visual cues for enhancing DHH users’ music experience. Haptic or visual-based assistive technologies were shown effective in research (Cavdir and Wang, 2020; Jack et al., 2015; Deja et al., 2020; Grierson, 2011), but have not been widely adopted by the DHH community (Ohshiro and Cartwright, 2022). The academic or industrial projects the Reddit users mentioned have not translated into easily purchasable products, and few in the DHH community mentioned using them. Affordability of assistive wearable devices should be given more consideration in the design process (Trivedi et al., 2019). Assistive technologies are important since DHH people’s own heuristics did not always work – playing music loud may bother family members or neighbors, especially in quiet hours – and CIs and hearing aids did not always produce optimal music experience, according to them.

Leveraging The DHH Community For Better Music Experience. Marginalized communities are often found to nurture mutual support among their members (Tang et al., 2023b, a). In our case, DHH people tended to share accessible music, assistive technologies, and music devices with the community. They also frankly shared their suboptimal music experience and answered medical questions regarding hearing loss and hearing aids based on their own experience. Empathetic responses helped DHH people achieve a better music experience or adapt to hearing loss. Designers should actively refer to social media discussions to gauge DHH people’s unique needs with music.

6. Conclusion

We presented a mixed-methods analysis of DHH people’s online music discussions in their communities. They drew on their own heuristics for better music experience, such as ASL, visual/haptic cues, defaulting to familiar music, listening to non-lyrical, instrument-heavy music, and playing music loud. A reciprocal community was formed around music, where the members actively shared information and answered questions about music and hearing loss. Based on the findings, we discuss the design justice for enhancing DHH people’s music experience through a more thorough understanding of their real, diverse musical needs. We conclude by providing concrete design implications. More research is encouraged to understand DHH people’s music needs and how assistive music technologies help them in practice.