Week 10: CALL and

Speech & Language Technology

The Phonetics & Speech Lab. and the www.ABAIR.ie Initiative for Irish

Neasa Ní Chiaráin
[email protected]
29/03/22
 What is CALL?

Computer-Assisted Language Learning

Most general terms:

A specialised field in Applied Linguistics
concerned with the use of technology in
language teaching and learning
 What is CALL?

• New: Formal beginnings in 1983

– adoption of the acronym at the 1983 TESOL convention
• Very diverse:
– multidisciplinary: OLL; iCALL; CMC; distance learning;
autonomous learning; social networking; language games
• Very wide range of technologies and software:
– new software coming on stream every week
• Wide range of principles and theories:
– AL; Ed.; SLA; TBLL; CALL
 Role of CALL?
Computer as Tutor Computer as Tool

• vocabulary flashcards • language learning

• set of online grammar activity involving social
exercises media, email, or a search
engine (e.g. Google)

the computer has no overt

the computer in some way teaching function
has a teaching function
 iCALL

“a field within CALL that applies concepts,

techniques, algorithms and technologies from
artificial intelligence to CALL”

(Gamper & Knapp, 2002; Heift & Schulze, 2007; Nerbonne, 2003; Schulze, 2008a)
 iCALL

• Intelligent CALL (iCALL) …adjective ‘intelligent’ from area

of Artificial Intelligence
 the sub-discipline of CALL which applies research findings
from natural language processing (NLP) and student modeling
– both areas of Artificial Intelligence – to CALL
• Connections with SLA theory – particularly in relation to
noticing & interaction in second language learning
(Gass, 1997; Long, 1996; Schmidt, 1990, 1994)
• Diverse range of activities, e.g.
 Vocabulary acquisition -> grammar development ->
comprehension skills in relation to reading & writing in the
second language classroom
 The Irish language

Q-Celtic P-Celtic
Irish Welsh
Scots Gaelic Breton
Manx Cornish
“ceann”

“penn”
 Decline over 150 years

1800 1851 1891 1926 1956

 Decline over 150 years
Gaeltacht pockets:
3 main dialects

1800 1851 1891 1926 1956

Phonetics & Speech Lab.
 ABAIR: The Technology
• Building speech and language technologies for
the dialects of Irish
– text-to-speech (TTS) voices
– speech recognition (ASR)
• Plan to expand to other areas
– dialogue (interactive language learning context)
Speech Technology: abair.ie
Multidialect synthetic voices
 ABAIR: speech technologies for Irish
• places the living, spoken language centre stage in the
digital world
– a public utility for the wider Irish language community,
including the diaspora
– disability/access
– education
– language maintenance and documentation
– Linguistic research on the Irish dialects
• by embedding speech and language technologies into
applications, we extend the everyday domains where we
use and are exposed to spoken Irish
The ABAIR initiative
 The ABAIR initiative

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

Speech Synthesis
Speech Recognition
II
Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

Speech Synthesis
Speech Recognition
II
Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

General Public
III
Education
Applications
Access/Disability
Speech Synthesis
Speech Recognition
II Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

General Public
III
Education
Applications
Access/Disability
Speech Synthesis
Speech Recognition
II Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

General Public
III
Education
Applications
Access/Disability
Speech Synthesis
Speech Recognition
II Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

General Public
III
Education
Applications
Access/Disability
Speech Synthesis
Speech Recognition
II Spoken Dialogue systems
Core (Machine Translation)
Technologies (Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

General Public
III
Education
Applications
Access
Speech Synthesis
II Speech Recognition
Core Spoken Dialogue systems
(Machine Translation)
Technologies
(Information Retrieval)

I Speech & text corpora

Resources (Pronunciation) dictionaries
Language models
Acoustic models
 The ABAIR initiative

III
Applications

II
Core
Technologies

I
Resources
 The ABAIR initiative

III
Applications

II
Core
Technologies

I
Resources
II. Core Technologies

Sidney Harris Cartoon

 Introductions…
TTS
• Andrew Murphy, postdoctoral researcher, ABAIR
ASR
• Liam Lonergan, PhD researcher, ABAIR
 General Public:
www.abair.ie open to everyone
 Accessibility: Inclusion
Crossing the communication gap for those with
disabilities
• Speaking for those who have no voice
– Augmentative and Alternative Communication (AAC)
• Hearing text for those who can’t read
(blind/literacy difficulties)
– Screenreaders (NVDA/eSpeak)
– Literacy aids for those with dyslexia
– Web reader
Accessibility: Inclusion
 Language Learning (iCALL)
A Flavour… 3 applications we’re working on to
promote language proficiency among learners
• Lón don Leon: teaching the sounds of Irish to
preliterate children
• Taidhgín: focusing on computer(teacher)-
student interaction
• An Scéalaí: teaching proficiency through
reading and storytelling, listening & speaking
Aim: contextualised, repetitive and compelling interactive materials
Lón don Leon
Taidhgín
 An Scéalaí (‘the Storyteller’)
Tarraingíonn scéal scéal eile (One Story Begets Another!)

• iCALL platform – more advanced literacy skills

• Improving all language skills with spoken language centre stage
– Story writing – learners create their own story/diary entry –
writing/telling and retelling stories: critical practice for acquisition of
Irish
– Prevalent use of speech output in reading materials means the linkage
of pronunciation & written forms are being reinforced while reading ->
assist learners to gain an intuitive grasp of the phonic regularities of
the written language, crucial to literacy acquisition

• Marries Research, Practice & Technology Development

 An Scéalaí: For whom?
CALL; Research & Irish Speech and Language Technology

Learners: autonomous Researchers: offers Future speech technology

reflective language insight into autonomous development: to cater
learning practice language learning effectively for learners ->
integrating the 4 processes need to gather large
language skills amount of learner data
The Language
Skills ->

A holistic
approach…
Title/Dialect Feedback
Compose title & Communication

6 core features… Select preferred

dialect
with the teacher

38
Dictionary Audio check Grammar check Recordings
Use Listen back to your Correct any Listen to the TTS
dictionary/thesau composition and highlighted and record
rus as you write correct any errors grammar errors yourself. Compare
you hear the pronunciation

39
An Scéalaí
Storytelling: composition time!
Corrective feedback: audio checks
Corrective feedback: grammar checks
 An Scéalaí
https://abair.ie/scealai
An Scéalaí–LARA: rich multimodal content

Université de Genève CALLECTOR project

 An Scéalaí: the learner
• A tool to support creative writing:
– Learners supported by the ‘wisdom’ that can be incorporated through ‘intelligent’ Speech
and Language technologies
» Encouraging revisions and rewritings as part of the process

• A holistic approach:
– constant reinforcing of the sound system and in turn that link between the sound and the
spelling

• Immediate corrective feedback:

– to encourage learners users to improve and enrich their text during the writing process

• Encouraging self-monitoring:
– learners keep track of their own actions and performance
 An Scéalaí: the teacher

• Help teachers promote an awareness of the phonics-basis of the

language
• Provide a motivational environment for learners to practice
writing
• Brings the spoken language into every aspect of the language
learning and helps compensate for the fact that native speakers
are not readily to hand for most learners
• Learners engage in self-correction before handing in written
tasks
• Opportunity to give feedback to individual learners via voice/text
 SLA-guided framework

• Based on ‘experiential’ or ‘hands-on’ learning where the task is

meaningful and students engage in reflective activity.
– The use of the target language is key to the learning process as learning
happens by doing (and by making errors and learning from these).
• With a view to facilitating autonomy (intentional learning), the
present application entails a personalised diary-style writing task
• Learners’ reflective contributions instantiate meaningful
language use, serving as a vehicle for making/correcting written
errors in a way that links the written to oral/aural skills
 An Scéalaí: the researcher

• a controllable learning environment, configurable towards

optimal conditions for learning
» feedback, learner modeling and adaptivity, motivational support
• monitored conditions for conducting empirical research on L2
learning
» learner interactions are logged (‘event’ data)
» valuable insight into what students are actually doing
 Society and Impact
• The Irish context
– About much more than the technology
– Endangered languages conferences and fora
– Language -> most precious cultural artefact
Out of the Lab -> Into Technologies -> Impact on a Language
Situation
Public usage

Inclusion

Education
 Documentation and Preservation
• All endangered languages and dialects

• Every language community can create voices & have

access to native speaker models
– Preserves virtual native speakers

• NOTE– not enough to just develop a voice … we (the

language communities) need to own the voice and
have the flexibility to adapt it –i.e. be able to use it and
modify for real-life uses...
 Endless possibilities…
Language …as a spoken communication system

• Now technology is allowing us to capture and

create virtual speakers
 TTS… working towards…
• expressivity in the voice
– characters, e.g. in the games need to talk with
expression, affect and attitude
• voice transformation: create multiple voices
from a single dialect voice
• speech science: state-of-the-art research in the
field
– require fundamental research on the voice
– a nut that hasn’t been cracked yet!
 ABAIR
General Public Access Education
 ASR... Working towards...
• General recognition system
– Multidialect
– Codeswitching
– Learner’s speech How you can help…
– Children’s speech
 SPEECH RECOGNITION
General Public Inclusion Education

- Dragon dictate for Joe Soap - Blind talking - CAPT

- Famous author
 Endless possibilities…
General Public Inclusion Education

- Museum - Personal assistant for - Taidhgín

- Siri/Alexa independent living - Talking Heads
Go raibh míle maith agaibh

Neasa Ní Chiaráin
[email protected]