Abstract
While we are capable of modeling the shape, e.g. face, arms, etc. of humanoid robots in a nearly natural or human-like way, it is much more difficult to generate human-like facial or body movements and human-like behavior like e.g. speaking and co-speech gesturing. In this paper it will be argued for a developmental robotics approach for learning to speak. On the basis of current literature a blueprint of a brain model will be outlined for this kind of robots and preliminary scenarios for knowledge acquisition will be described. Furthermore it will be illustrated that natural speech acquisition mainly results from learning during face-to-face communication and it will be argued that learning to speak should be based on human-robot face-to-face communication. Here the human acts like a caretaker or teacher and the robot acts like a speech-acquiring toddler. This is a fruitful basic scenario not only for learning to speak, but also for learning to communicate in general, including to produce co-verbal manual gestures and to produce co-verbal facial expressions.
References
Asada M, Hosoda K, Kuniyoshi Y, Ishiguro H, Inui T, Yoshikawa Y, Ogino M, Yoshida C, 2009. Cognitive developmental robotics: A survey. IEEE transactions on Autonomous Mental Development 1, 12-34.10.1109/TAMD.2009.2021702Search in Google Scholar
Aziz-Sadeh L, Damasio A, 2008. Embodied semantics for actions: Findings from functional brain imaging. Journal of Physiology-Paris 102, 35-39.10.1016/j.jphysparis.2008.03.012Search in Google Scholar
Bailly G, Raidt S, Elisei F, 2010. Gaze, conversational agents and face-to-face communication. Speech Communication 52, 598-612.10.1016/j.specom.2010.02.015Search in Google Scholar
Bergmann K, Kopp S, 2009. Increasing the Expressiveness of Virtual Agents – Autonomous Generation of Speech and Gesture for Spatial Description Tasks. In: Decker K, Sichman J, Sierra C, Castelfranchi C (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009), pp. 361-368.Search in Google Scholar
Birkholz P, Kröger BJ, 2006. Vocal tract model adaptation using magnetic resonance imaging. Proceedings of the 7th International Seminar on Speech Production (Belo Horizonte, Brazil) pp. 493-500.Search in Google Scholar
Birkholz P, Kröger BJ, Neuschaefer-Rube C, in press. Model-based reproduction of articulatory trajectories for consonant-vowel sequences. IEEE Transactions on Audio, Speech and Language Processing. DOI:10.1109/TASL.2010.209163210.1109/TASL.2010.2091632Search in Google Scholar
Brandl H, 2009. A computational model for unsupervised child-like speech acquisition. Unpublished Doctoral Thesis (University of Bielefeld, Bielefeld, Germany)Search in Google Scholar
Breazeal C, 2003. Towards sociable robots. Robotics and Autonomous Systems 42, 167-175.10.1016/S0921-8890(02)00373-1Search in Google Scholar
Breazeal C, 2004. Function meets style: Insights from emotion theory applied to HRI. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 34, 187-194.10.1109/TSMCC.2004.826270Search in Google Scholar
Brooks RA, Breazeal C, Marjanovic M, Scassellati B, Williamson MM, 1999. The cog project: building a humanoid robot. In: Nehaniv CL (ed.) Computation for metaphors, analogy, and agents (Springer Verlag, Berlin), pp. 52-87.10.1007/3-540-48834-0_5Search in Google Scholar
Caligiore D, Ferrauto T, Parisi D, Accornero N, Capozza M, Baldassarre G, 2008. Using motor babbling and Hebb rules for modeling the development of reaching with obstacles and grasping. In: Dillmann R, Maloney C, Sandini G, Asfour T, Cheng G, Metta G, Ude A (eds.) International Conference on Cognitive Systems, CogSys 2008 (University of Karlsruhe, Karlsruhe, Germany)Search in Google Scholar
Cangelosi A, Riga T, 2006. An embodied model for sensorimotor grounding and grounding transfer: experiments with epigenetic robots. Cognitive Science 30, 673-689.10.1207/s15516709cog0000_72Search in Google Scholar
Coleman J, 1999. Cognitive reality and the phonological lexicon: A review. Journal of Neurolinguistics 11, 295-320.10.1016/S0911-6044(97)00014-6Search in Google Scholar
Dehaene-Lambertz G, Hertz-Pannier L, Dubois J, Dehaene S, 2008. How Does Early Brain Organization Promote Language Acquisition in Humans? European Review 16, 399-411.10.1017/S1062798708000513Search in Google Scholar
Demiris Y, Dearden A, 2005. From motor babbling to hierarchical learning by imitation: a robot developmental pathway. In: Berthouze L, Kaplan F, Kozima H, Yano H, Konczak J, Metta G, Nadel J, Sandini G, Stojanov G, Balkenius C (eds.) Proceedings of the Fifth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems (Lund University Cognitive Studies 123, Lund), pp. 31-37.Search in Google Scholar
Desmurget M, Grafton ST, 2000. Forward modeling allows feedback control for fast reaching movements. Trends in Cognitive Sciences 4, 423-431. Dohen M, Schwartz, JL, Bailly G, 2010. Speech and face-to-face communication – An introduction. Speech Communication 52, 477-480.10.1016/S1364-6613(00)01537-0Search in Google Scholar
Fehr E, Fischbacher U, Gächter S, 2002. Strong reciprocity, human cooperation, and the enforcement of social norms. Human Nature 13, 1-25.10.1007/s12110-002-1012-7Search in Google Scholar PubMed
Fujie S, Fukushima K, Kobayashi T, 2004. A conversation robot with backchanel feedback function based on linguistic and nonlinguistic information. Proceedings of the 2nd International conference on Autonomous Robots and Agents (Palmerston North, New Zealand), pp. 379-384.Search in Google Scholar
Fukui K, Nishikawa K, Ikeo S, Shintaku E, Takada K, Takanobu H, Honda M, Takanishi A, 2005. Development of a talking robot with vocal cords and lips having human-like biological structures. Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (Edmonton, Alberta, Canada), pp. 2023-2028.10.1109/IROS.2005.1545356Search in Google Scholar
Galantucci B, Steels L, 2008. The embodied communication in artificial agents and humans. In: Wachsmuth I, Lenzen M, Knoblich G (eds.), Embodied Communication in Humans and Machines (Oxford University Press, Oxford) pp. 229-256.10.1093/acprof:oso/9780199231751.003.0011Search in Google Scholar
Goldstein MH, Schwade J, 2008. Social Feedback to Infants’ Babbling Facilitates Rapid Phonological Learning. Psychological Science 19, 515-523.10.1111/j.1467-9280.2008.02117.xSearch in Google Scholar PubMed
Goldstein MH, Schwade J, Briesch J, Syal S, 2010. Learning While Babbling: Prelinguistic Object-Directed Vocalizations Indicate a Readiness to Learn. Infancy 15, 362-391.10.1111/j.1532-7078.2009.00020.xSearch in Google Scholar PubMed
Golfinopoulos E, Tourville JA, Bohland JW, Ghosh SS, Nieto-Castanon A, Guenther FH, 2011. fMRI investigation of unexpected somatosensory feedback perturbation during speech. NeuroImage 55, 1324-1338.10.1016/j.neuroimage.2010.12.065Search in Google Scholar PubMed PubMed Central
Grossberg S, 2010. The link between brain learning, attention, and consciousness. In: Carsetti A (ed.) Causality, Meaningful Complexity and Embodied Cognition (Springer, Dordrecht), pp. 3-45.10.1007/978-90-481-3529-5_1Search in Google Scholar
Grossmann T, Johnson MH, Lloyd-Fox S, Blasi A, Deligianni F, Elwell C, Csibra G, 2008. Early cortical specialization for face-to-face communication in human infants. Proceedings of the Royal Society B: Biological Sciences 275, 2803-2811.10.1098/rspb.2008.0986Search in Google Scholar PubMed PubMed Central
Guenther FH, Ghosh SS, Tourville JA, 2006. Neural modeling and imaging of the cortical interactions underlying syllable production. Brain and Language 96, 280–301.10.1016/j.bandl.2005.06.001Search in Google Scholar PubMed PubMed Central
Haikonen POA, 2009. The role of associative processing in cognitive computing. Cognitive Computation 1, 42-49.10.1007/s12559-009-9006-ySearch in Google Scholar
Hashimoto T, Kato N, Kobayashi H, 2010. Study on educational application of android robot SAYA: Field trial and evaluation at elementary school. In: Lui H, Ding H, Xiong Z, Zhu X (eds.) Intelligent Robotics and Applications. LNCS 6425 (Springer, Berlin), pp. 505-516.10.1007/978-3-642-16587-0_47Search in Google Scholar
Hickok G, Poeppel D, 2007. Towards a functional neuroanatomy of speech perception. Trends in Cognitive Sciences 4, 131–138.10.1016/S1364-6613(00)01463-7Search in Google Scholar
Indefrey P, Levelt WJM, 2004. The spatial and temporal signatures of word production components. Cognition 92, 101-144.10.1016/j.cognition.2002.06.001Search in Google Scholar
Iverson JM, Capirci O, Longobardi E, Caselli MC, 1999. Gesturing in mother-child interactions. Cognitive Development 14, 57-75.10.1016/S0885-2014(99)80018-5Search in Google Scholar
Kanda T, Hirano T, Eaton D, 2004. Interactive robots as social partners and peer tutors for children: a field trial. Human-Computer Interaction 19, 61-84.10.1207/s15327051hci1901&2_4Search in Google Scholar
Kanda T, Kamasima M, Imai M, Ono T, Sakamoto D, Ishiguro H, Anzai Y, 2007. A humanoid robot that pretends to listen to route guidance from a human. Journal of Autonomous Robots 22, 87-100.10.1007/s10514-006-9007-6Search in Google Scholar
Kanda T, Miyashita T, Osada T, Haikawa Y, Ishiguro H, 2008. Analysis of humanoid appearance in human-robot interaction. IEEE Transactions on Robotics 24, 725-735.10.1109/TRO.2008.921566Search in Google Scholar
Kandel ER, Schwartz JH, Jessell TM, 2000. Principles of Neural Science. 4th edition (McGraw-Hill, New York).Search in Google Scholar
Kiebel SJ, Daunizeau J, Friston KJ, 2008. A Hierarchy of Time-Scales and the Brain. PLoS Comput Biol 4(11): e1000209. doi:10.1371/journal.pcbi.1000209.10.1371/journal.pcbi.1000209Search in Google Scholar PubMed PubMed Central
Kipp M, Neff M, Kipp KH, Albrecht I, 2007. Towards Natural Gesture Synthesis: Evaluating gesture units in a data-driven approach to gesture synthesis. In: Pellachaud C, Martin JC, Andre E, Chollet G, Karpouzis K, Pele D (eds.), Intelligent Virtual Agents. LNAI 4722 (Springer, Berlin), pp. 15-28.10.1007/978-3-540-74997-4_2Search in Google Scholar
Kohonen T, 2001. Self-Organizing Maps (Springer, Berlin).10.1007/978-3-642-56927-2Search in Google Scholar
Kopp S, Bergmann K, Buschmeier H, Sadeghipour A, 2009. Requirements and Building Blocks for Sociable Embodied Agents. In: Mertsching B, Hund M, Aziz Z (eds.) Advances in Artificial Intelligence. LNCS 5803 (Springer, Berlin), pp. 508-515.10.1007/978-3-642-04617-9_64Search in Google Scholar
Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I, 2005. A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Oliver P, Rist T (eds.), Intelligent Virtual Agents. LNCS 3661 (Springer, Berlin), pp. 329-343.10.1007/11550617_28Search in Google Scholar
Kosuge K, Hirata Y, 2004. Human-robot interaction. Proceedings of the 2004 IEEE International Conference on Robotics and Biometrics (Xhenyang, China), pp. 8-11.Search in Google Scholar
Kröger BJ, Birkholz P, 2007. A gesture-based concept for speech movement control in articulatory speech synthesis. In: Esposito A, Faundez-Zanuy M, Keller E, Marinaro M (eds.) Verbal and Nonverbal Communication Behaviours. LNAI 4775 (Springer, Berlin), pp. 174-189.10.1007/978-3-540-76442-7_16Search in Google Scholar
Kröger BJ, Birkholz P, 2009. Articulatory Synthesis of Speech and Singing: State of the Art and Suggestions for Future Research. In: Esposito A, Hussain A, Marinaro M (eds) Multimodal Signals: Cognitive and Algorithmic Issues. LNAI 5398 (Springer, Berlin), pp. 306-319.10.1007/978-3-642-00525-1_31Search in Google Scholar
Kröger BJ, Kannampuzha J, Neuschaefer-Rube C, 2009. Towards a neurocomputational model of speech production and perception. Speech Communication 51, 793-809.10.1016/j.specom.2008.08.002Search in Google Scholar
Kröger BJ, Birkholz P, Lowit A, 2010. Phonemic, sensory, and motor representations in an action-based neurocomputational model of speech production (ACT). In: Maassen B, van Lieshout P (eds.), Speech Motor Control: New developments in basic and applied research. (Oxford University Press, New York), pp. 23-36.10.1093/acprof:oso/9780199235797.003.0002Search in Google Scholar
Kröger BJ, Kopp S, Lowit A, 2010. A model for production, perception, and acquisition of actions in face-to-face communication. Cognitive Processing 11, 187-205.10.1007/s10339-009-0351-2Search in Google Scholar PubMed
Kuhl PK, 2004. Early language acquisition: cracking the speech code. Nature Reviews Neuroscience 5, 831-843.10.1038/nrn1533Search in Google Scholar PubMed
Kuhl PK, 2007. Is speech learning „gated by the social brain? Developmental Science 10, 110-120.10.1111/j.1467-7687.2007.00572.xSearch in Google Scholar PubMed
Lau EF, Phillips C, Poeppel D, 2008. A cortical network for semantics: (de)constructing the N400. Nature Reviews Neuroscience 9, 920-933.10.1038/nrn2532Search in Google Scholar PubMed
Levelt WJM, Roelofs A, Meyer A, 1999. A theory of lexical access in speech production. Behavioral and Brain Sciences 22, 1-75.10.1017/S0140525X99451775Search in Google Scholar
Li P, Fakas I, MacWhinney B, 2004. Early lexical development in a self-organizing neural network. Neural Networks 17, 1345-1362.10.1016/j.neunet.2004.07.004Search in Google Scholar PubMed
Li Y, Kurata S, Morita S, Shimizu S, Munetaka D, Nara S, 2008. Application of chaotic dynamics in a recurrent neural network to control: hardware implementation into a novel autonomous roving robot. Biological Cybernetics 99, 185-196.10.1007/s00422-008-0249-6Search in Google Scholar PubMed
Lindblom J, Ziemke T, 2003. Social situatedness of natural and artificial intelligence: Vygotsky and beyond. Adaptive Behavior 11, 79-96.10.1177/10597123030112002Search in Google Scholar
Lungarella M, Metta G, Pfeiffer R, Sandini, 2003. Developmental robotics: a survey. Connection Science 15, 151-190.10.1080/09540090310001655110Search in Google Scholar
Madden C, Hoen M, Dominey PF, 2010. A cognitive neuroscience perspective on embodied language for human-robot cooperation. Brain and Language 112, 180-188.10.1016/j.bandl.2009.07.001Search in Google Scholar PubMed
McGurk H, MacDonald J, 1976. Hearing lips and seeing voices. Nature 264, 746-748.10.1038/264746a0Search in Google Scholar PubMed
Mitchell CJ, De Houwer J, Lovibond PF, 2009. The propositional nature of human associative learning. Behavioral and Brain Sciences 32, 183-198. Ogawa H, Watanabe T, 2000. Interrobot: A speech driven embodied interaction robot. Proceedings of the 2000 IEEE International Workshop on Robot and Human Interactive Communication (Osaka, Japan), pp. 322-327.Search in Google Scholar
Özçalkan S, Goldin-Meadow S, 2005. Gesture is at the cutting edge of early language development. Cognition 96, B101-B113.10.1016/j.cognition.2005.01.001Search in Google Scholar PubMed
Parisi D, 2010. Robots with language. Frontiers in Neurorobotics 4. DOI: 10.3389/fnbot.2010.0001010.3389/fnbot.2010.00010Search in Google Scholar PubMed PubMed Central
Patterson K, Nestor PJ, Rogers TT, 2007. Where do you know what you know? The representation of semantic knowledge in the human brain. Nature Reviews Neuroscience 8, 976-987.10.1038/nrn2277Search in Google Scholar
Pelachaud C, Poggi I, 2002. Subtleties of facial expressions in embodied agents. The Journal of Visualization and Computer Animation 13, 301–312. Pierrehumbert JB, 2003. Phonetic diversity, statistical learning, and acquisition of phonology. Language and Speech 46, 115-154.10.1002/vis.299Search in Google Scholar
Plebe A, Mazzone M, de la Cruz V, 2010. First word learning: a cortical model. Cognitive Computation 2, 217-229.10.1007/s12559-010-9044-5Search in Google Scholar
Prince CG, Demiris Y, 2003. Introduction to the special issue on epigenetic robotics. Adaptive Behavior 11, 75-77.10.1177/10597123030112001Search in Google Scholar
Rich C, Ponsler B, Holroyd A, Sidner CL, 2010. Recognizing engagement in human-robot interaction. Proceedings of the 5th ACM/IEEE International conference on Human-Robot Interaction (Osaka, Japan), pp. 375-382.10.1109/HRI.2010.5453163Search in Google Scholar
Riecker A, Mathiak K, Wildgruber D, Erb A, Hertrich I, Grodd W, Ackermann H, 2005. fMRI reveals two distinct cerebral networks subserving speech motor control. Neurology 64, 700-706.10.1212/01.WNL.0000152156.90779.89Search in Google Scholar
Rizzolatti G, 2005. The mirror neuron system and its function in humans. Anatomy and Embryology 210, 419-421.10.1007/s00429-005-0039-zSearch in Google Scholar
Roy AC, Craighero L, Fabbri-Destro, M, Fadiga L, 2008. Phonological and lexical motor facilitation during speech listening: A transcranial magnetic stimulation study. Journal of Physiology-Paris 102, 101-105.10.1016/j.jphysparis.2008.03.006Search in Google Scholar
Saunders JA, Knill DC, 2004. Visual Feedback Control of Hand Movements. The Journal of Neuroscience 24, 3223-3234.10.1523/JNEUROSCI.4319-03.2004Search in Google Scholar
Schaal S, 1999. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3, 233-242.10.1016/S1364-6613(99)01327-3Search in Google Scholar
Shiomi M, Kanda T, Miralles N, Miyashita T, 2004. Face-to-face interactive humanoid robot. Proceedings of the 2004 IEEE International Conference on Intelligent Robots and Systems (Sendai, Japan), pp. 1340-1346.Search in Google Scholar
Shiwa T, Kanda T, Imai M, Ishiguro H, Hagita N, 2008. How quickly should communication robots respond? Proceedings of 2008 ACM Conference of Human Robot Interaction (Amsterdam, Netherlands), pp. 153-160.10.1145/1349822.1349843Search in Google Scholar
Sidner CL, Lee C, Kidd CD, Lesh N, Rich C, 2005. Explorations in engagement for humans and robots. Artificial Intelligence 166, 140-164. Steels L, 2003. Evolving grounded communication for robots. Trends in Cognitive Sciences 7, 308-312.10.1016/j.artint.2005.03.005Search in Google Scholar
Tani J, Ito M, 2003. Self-organization of behavioral primitives as multiple attractor dynamics: a robot experiment. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans 33, 481-488.10.1109/TSMCA.2003.809171Search in Google Scholar
Tani J, Nishimoto R, Namikawa J, Ito M, 2008. Codevelopmental learning between human and humanoid robot using a dynamic neural network model. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics 38, 43-59.10.1109/TSMCB.2007.907738Search in Google Scholar PubMed
Thompson RF, 1986. The neurobiology of learning and memory. Science 233, 941-947.10.1126/science.3738519Search in Google Scholar PubMed
Tomasello M, 2000. First steps towards a usage-based theory of language acquisition. Cognitive Linguistics 11, 61-82.10.1515/cogl.2001.012Search in Google Scholar
Trappenberg T, Hartono P, Rasmusson D, 2009. Top-Down Control of Learning in Biological Self-Organizing Maps. In: Principe JC, Miikkulainen R (eds.), Advances in Self-Organizing Maps. LNCS 5629 (Springer, Berlin), pp. 316-324.10.1007/978-3-642-02397-2_36Search in Google Scholar
Yoshikawa Y, Shinozawa K, Ishiguro H, Hagita N, Miyamoto T, 2006. The effects of responsive eye movement and blinking behavior in a communication robot. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (Beijing, China), pp. 4564-4569.10.1109/IROS.2006.282160Search in Google Scholar
Vaz M, Brandl H, Joublin F, Goerick C, 2009. Learning from a tutor: Embodied speech acquisition and imitation learning. Proceedings of the IEEE 8th International Conference on Development and Learning (Shanghai, China), pp. 1-6.10.1109/DEVLRN.2009.5175543Search in Google Scholar
Vilhjálmsson H, 2009. Representing communicative function and behavior in multimodal communication. In: Esposito A, Hussain A, Marinaro M, Martone R (eds.) Multimodal Signals: Cognitive and Algorithmic Issues. LNCS 5398 (Springer, Berlin), pp. 47-59.10.1007/978-3-642-00525-1_4Search in Google Scholar
Weng J, 2004. Developmental robotics: Theory and experiments. International Journal of Humanoid Robotics 1, 199-236.10.1142/S0219843604000149Search in Google Scholar
Weng J, McClelland J, Pentland A, Sporns O, Stockman I, Sur M, Thelen E, 2001. Autonomous mental development by robots and animals. Science 291, 599-600.10.1126/science.291.5504.599Search in Google Scholar PubMed
© Bernd J. Kröger et al.
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
