Skip to main content
SpringerLink
Log in

Multimodal Dialogue for Ambient Intelligence and Smart Environments

  • Chapter
Handbook of Ambient Intelligence and Smart Environments

Abstract

Ambient Intelligence (AmI) and Smart Environments (SmE) are based on three foundations: ubiquitous computing, ubiquitous communication and intelligent adaptive interfaces [41]. This type of systems consists of a series of interconnected computing and sensing devices which surround the user pervasively in his environment and are invisible to him, providing a service that is dynamically adapted to the interaction context, so that users can naturally interact with the system and thus perceive it as intelligent.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Batliner A, Hacker C, Steidl S, Nöth E, D’Arcy S, Russel M, Wong M (2004) Towards multilingual speech recognition using data driven source/target acoustical units association. In: Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’04), Montreal, Quebec, Canada, pp 521–524

    Google Scholar 

  2. Beringer N, Karal U, Louka K, Schiel F, Türk U (2002) PROMISE A procedure for multimodal interactive system evaluation. In: Proc. of the LREC Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas, Spain, pp 77–80

    Google Scholar 

  3. Beringer N, Louka K, Penide-Lopez V, Türk U (2002) End-to-end evaluation of multimodal dialogue systems - can we transfer established methods? In: Proc. of the LREC Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas, Spain, pp 558–563

    Google Scholar 

  4. Bernsen N (2003) User modelling in the car. Lecture Notes in Artificial Intelligence pp 378–382

    Google Scholar 

  5. Berre AJ, Marzo GD, Khadraoui D, Charoy F, Athanasopoulos G, Pantazoglou M, Morin JH, Moraitis P, Spanoudakis N (2007) SAMBA - an agent architecture for ambient intelligence elements interoperability. In: Proc. of Third International Conference on Interoperability of Enterprise Software and Applications, Funchal, Madeira, Portugal

    Google Scholar 

  6. Bickmore T, Mauer D, Brown T (2008) Context awareness in a handheld exercise agent. Pervasive and Mobile Computing Doi:10.1016/j.pmcj.2008.05.004. In press

    Google Scholar 

  7. Bouzy B, Cazenave T (1997) Using the object oriented paradigm to model context in computer Go. In: Proc. of Context’97, Rio, Brazil

    Google Scholar 

  8. Bricon-Souf N, Newman CR (2007) Context awareness in health care: A review. International journal of medical informatics 76:2–12

    Article  Google Scholar 

  9. Callejas Z, López-Cózar R (2008) Influence of contextual information in emotion annotation for spoken dialogue systems. Speech Communication 50(5):416–433

    Article  Google Scholar 

  10. Carpenter R (1992) The logic of typed feature structures. Cambridge University Press, Cambridge, England

    Book  MATH  Google Scholar 

  11. Corradini A, Mehta M, Bernsen N, Martin J, Abrilian S (2003) Multimodal input fusion in human-computer interaction. In: Proc. of the NATO-ASI Conference on Data Fusion for Situation Monitoring, Incident Detection, Alert and Response Management, Yerevan, Armenia

    Google Scholar 

  12. Dale R, Moisl H, Somers H (eds) (2000) Handbook of natural language processing. Dekker Publishers

    Google Scholar 

  13. Daubias P, Deléglise P (2002) Lip-reading based on a fully automatic statistical model. In: Proc. of International Conference on Speech and Language Processing, Denver, Colorado, US, pp 209–212

    Google Scholar 

  14. Dey A, Abowd G (1999) The context toolkit: Aiding the development of context-enabled applications. In: Proc. of the SIGCHI conference on Human factors in computing systems (CHI 99), Pittsburgh, Pennsylvania, US, pp 434–441

    Google Scholar 

  15. Dey A, Abowd G (2000) Towards a better understanding of context and context-awareness. In: Proc. of the 2000 Conference on Human Factors in Computer Systems (CHI’00), pp 304–307

    Google Scholar 

  16. Doulkeridis C, Vazirgiannis M (2008) CASD: Management of a context-aware service directory. Pervasive and mobile computing Doi:10.1016/j.pmcj.2008.05.001. In press

    Google Scholar 

  17. Dutoit T (1996) An introduction to text-to-speech synthesis. Kluwer Academic Publishers

    Google Scholar 

  18. Dybkjaer L, Bernsen N, Minker W (2004) Evaluation and usbility of multimodal spoken language dialogue systems. Speech Communication 43:33–54

    Article  Google Scholar 

  19. Encarnaçao J, Kirste T (2005) Ambient intelligence: Towards smart applicance ensembles. In: From Integrated Publication and Information Systems to Virtual Information and Knowledge Environments, pp 261–270

    Google Scholar 

  20. Engelmore R, Mogan T (1988) Blackboard systems. Addison-Wesley

    Google Scholar 

  21. Forbes-Riley K, Litman D (2004) Modelling user satisfaction and student learning in a spoken dialogue tutoring system with generic, tutoring, and user affect parameters. In: Proc. of the Human Language Technology Conference - North American chapter of the Association for Computational Linguistics annual meeting (HLT-NAACL’06), New York, US, pp 264–271

    Google Scholar 

  22. Fraser M, Gilbert G (1991) Simulating speech systems. Computer Speech and Language 5:81–99

    Article  Google Scholar 

  23. Gárate A, Herrasti N, López A (2005) Genio: An ambient intelligence application in home automatation and entertainment environment. In: Proc. of Joint soc-EUSI Conference, pp 241–245

    Google Scholar 

  24. Gaver WW (1992) Using and creating auditory icons. SFI studies in the sciences of complexity, Addison Wesley Longman, URL Proceedings/1992/Gaver1992.pdf

    Google Scholar 

  25. Georgalas N, Ou S, Azmoodeh M, Yang K (2007) Towards a model-driven approach for ontology-based context-aware application development: a case study. In: Proc. of the fourth International Workshop on Model-Based Methodologies for Pervasive and Embedded Software (MOMPES ’07), Braga, Portugal, pp 21–32

    Google Scholar 

  26. Gustafson J, Bell L, Beskow J, Boye J, Carlson R, Edlund J, Granstrom B, House D, Wirén M (2000) Adapt - a multimodal conversational dialogue system in an apartment domain. In: Proc. of International Conference on Speech and Language Processing, Beijing, China, pp 134–137

    Google Scholar 

  27. Haseel L, Hagen E (2005) Adaptation of an automotive dialogue system to users’ expertise. In: Proc. of 9th International Conference on Spoken Language Processing (Interspeech’05-Eurospeech), Lisbon, Portugal, pp 222–226

    Google Scholar 

  28. Heim J, Nilsson E, Havard J (2007) User Profiles for Adapting Speech Support in the Opera Web Browser to Disabled Users. Lecture Notes in Computer Science 4397:154–172

    Article  Google Scholar 

  29. Hengartner U, Steenkiste P (2006) Avoiding privacy violations caused by context-sensitive services. Pervasive and mobile computing 2:427–452

    Article  Google Scholar 

  30. Henricksen K, Indulska J (2006) Developing context-aware pervasive computing applications: models and approach. Pervasive and mobile computing 2:37–64

    Article  Google Scholar 

  31. Henricksen K, Indulska J, Rakotonirainy A (2002) Modeling context information in pervasive computing systems. In: Proc. of the First International Conference on Pervasive Computing, pp 167–180

    Google Scholar 

  32. Ho J, Intille S (2005) Using context-aware computing to reduce the perceived burden of interruptions from mobile devices. In: Proc. of the 2005 Conference on Human Factors in Computer Systems (CHI’05), Portland, US, pp 909–918

    Google Scholar 

  33. Hovy EH (1993) Automated discourse generation using discourse relations. Artificial Intelligence, Special Issue on Natural Language Processing 63:341–385

    Google Scholar 

  34. Intille S, Larson K, Munguia E (2003) Designing and evaluating technology for independent aging in the home. In: Proc. of the International Conference on Aging, Disability and Independence

    Google Scholar 

  35. Intille S, Larson K, Beaudin J, Nawyn J, Tapia EM, Kaushik P (2005) A living laboratory for the design and evaluation of ubiquitous computing technologies. In: Proc. of the 2005 Conference on Human Factors in Computer Systems (CHI’05), Portland, Oregon, US, pp 1941–1944

    Google Scholar 

  36. Johnston M, Bangalore S, Vasireddy G, Stent A, Ehlen P, Walker M, Whittaker S, Maloor P (2002) Match: An architecture for multimodal dialogue systems. In: Proc. of Association for Computational Linguistics, Pennsylvania, Philadelphia, US, pp 376–383

    Google Scholar 

  37. Jokinen K (2003) Natural interaction in spoken dialogue systems. In: Proc. of the Workshop Ontologies and Multilinguality in User Interfaces, Crete, Greece, pp 730–734

    Google Scholar 

  38. Kang H, Suh E, Yoo K (2008) Packet-based context aware system to determine information system user’s context. Expert systems with applications 35:286–300

    Google Scholar 

  39. Kettebekov S, Sharma R (2000) Understanding gestures in multimodal human computer interaction. Int Journal on Artificial Intelligence Tools 9(2):205–223

    Article  Google Scholar 

  40. Korth A, Plumbaum T (2007) A framework for ubiquitous user modelling. In: Proc. of IEEE International Conference on Information Reuse and Integration, Las Vegas, Nevada, US, pp 291–297

    Google Scholar 

  41. Kovács GL, Kopácsi S (2006) Some aspects of ambient intelligence. Acta Polytechnica Hungarica 3(1):35–60

    Google Scholar 

  42. Kwon O, Sadeh N (2004) Applying case-based reasoning and multi-agent intelligent system to context-aware comparative shopping. Decision Support Systems 37:199–213

    Google Scholar 

  43. Langner B, Black A (2005) Using speech in noise to improve understandability for elderly listeners. In: Proc. of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU’05), San Juan, Puerto Rico, pp 392–396

    Google Scholar 

  44. Lemon O, Bracy A, Gruenstein A, Peters S (2001) The WITAS Multi-Modal Dialogue System I. In: Proc. of Interspeech, Aalborg, Denmark, pp 1559–1562

    Google Scholar 

  45. Levin E, Levin A (2006) Dialog design for user adaptation. In: Proc. of the International Conference on Acoustics Speech Processing, Toulouse, France, pp 57–60

    Google Scholar 

  46. López-Cózar R, Araki M (2005) Spoken, Multilingual and Multimodal Dialogue Systems: Development and Assessment. John Wiley Sons

    Google Scholar 

  47. López-Cózar R, Callejas Z, Montoro G (2006) DS-UCAT: A new multimodal dialogue system for an academic application. In: Proc. of Interspeech ’06 - Satellite Workshop Dialogue on Dialogues, Multidisciplinary Evaluation of Advanced Speech-Based Interactive Systems, Pittsburgh, Pennsylvania, US, pp 47–50

    Google Scholar 

  48. Markopoulos P, de Ruyter B, Privender S, van Breemen A (2005) Case study: bringing social intelligence into home dialogue systems. Interactions 12(4):37–44

    Article  Google Scholar 

  49. Martínez AE, Cabello R, Gómez FJ, Martínez J (2003) INTERACT-DM. A solution for the integration of domestic devices on network management platforms. In: Proc. of IFIP/IEEE International Symposium on Integrated Network Management, Colorado Springs, Colorado, US, pp 360–370

    Google Scholar 

  50. Martinovski B, Traum D (2003) Breakdown in human-machine interaction: the error is the clue. In: Proc. of the ISCA Tutorial and Research Workshop on Error Handling in Dialogue Systems, Chateau d’Oex, Vaud, Switzerland, pp 11–16

    Google Scholar 

  51. McAllister D, Rodman R, Bitzer D, Freeman A (1997) Lip synchronization of speech. In: Proc. of ESCA Workshop on Audio-Visual Speech Processing (AVSP’97), Kasteel Groenendael, Hilvarenbeek, The Netherlands, pp 133–136

    Google Scholar 

  52. Möller S, Krebber J, Raake A, Smeele P, Rajman M, Melichar M, Pallotta V, Tsakou G, Kladis B, Vovos A, Hoonhout J, Schuchardt D, Fakotakis N, Ganchev T, Potamitis I (2004) INSPIRE: Evaluation of a Smart-Home System for Infotainment Management and Device Control. In: Proc. of the International Conference on Language Resources and Evaluation (LREC), Lisbon, Portugal, pp 1603–1606

    Google Scholar 

  53. Möller S, Krebber J, Smeele P (2006) Evaluating the speech output component of a smart-home system. Speech Communication 48:1–27

    Article  Google Scholar 

  54. Montoro G, Alamán X, Haya P (2004) A plug and play spoken dialogue interface for smart environments. In: Proc. of Fifth International Conference on Intelligent Text Processing and Computational Linguistics (CICLing’04), Seoul, South Korea, pp 360–370

    Google Scholar 

  55. Nazari AA (2005) A Generic UPnP Architecture for Ambient Intelligence Meeting Rooms and a Control Point allowing for Integrated 2D and 3D Interaction. In: Proc. of Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services, USges and Technologies, pp 207–212

    Google Scholar 

  56. Ndiaye A, Gebhard P, Kipp M, Klessen M, Schneider M, Wahlster W (2005) Ambient intelligence in edutainment: Tangible interaction with life-like exhibit guides. Lecture Notes in Artificial Intelligence 3814:104–113

    Google Scholar 

  57. Nigay L, Coutaz J (1995) A generic platform for addressing the multimodal challenge. In: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Denver, Colorado, US, pp 98–105

    Google Scholar 

  58. Ohno T, Mukawa N, Kawato S (2003) Just blink your eyes: A head-free gaze tracking system. In: Proc. of Computer-Human Interaction, Fort Lauderdale, Florida, pp 950–951

    Google Scholar 

  59. Pascoe J (1997) The Stick-e note architecture: Extending the interface beyond the user. In: Proc. of the International Conference on Intelligent User Interfaces, Orlando, Florida, US, pp 261–264

    Google Scholar 

  60. Porzel R, Gurevych I (2002) Towards context-sensitive utterance interpretation. In: Proc. of the 3rd SIGdial Workshop on Discourse and Dialogue, Philadelphia, US, pp 154–161

    Google Scholar 

  61. Prendinger H, Mayer S, Mori J, Ishizuka M (2003) Persona effect revisited. using bio-signals to measure and reflect the impact of character-based interfaces. In: Proc. of the 4th International Working Conference on Intelligent Virtual Agents (IVA’03), Kloster Irsee, Germany, pp 283–291

    Google Scholar 

  62. Rabiner LR, Juang BH (1993) Fundamentals of Speech Recognition. Prentice-Hall

    Google Scholar 

  63. Reithinger N, Lauer C, Romary L (2002) MIAMM - Multidimensional information access using multiple modalities. In: Proc. of International CLASS workshop on natural intelligent and effective interaction in multimodal dialogue systems

    Google Scholar 

  64. de Rosis F, Novielli N, Carofiglio V, Cavalluzzi A, de Carolis B (2006) User modeling and adaptation in health promotion dialogs with an animated character. Journal of Biomedical Informatics 39:514–531

    Article  Google Scholar 

  65. Sachetti D, Chibout R, Issarny V, Cerisara C, Landragin F (2004) Seamless access to mobile services for the mobile user. In: Proc. of IEEE Int. Conference on Software Engineering, Beijing, China, pp 801–804

    Google Scholar 

  66. Saini P, de Ruyter B, Markopoulos P, Breemen AV (2005) Benefits of social intelligence in home dialogue systems. In: Proc. of 11th International Conference on Human-Computer Interaction, Las Vegas, Nevada, US, pp 510–521

    Google Scholar 

  67. Satyanarayanan M (2002) Challenges in implementing a context-aware system. IEEE Distributed Systems Online 3(9)

    Google Scholar 

  68. Schmidt A (2002) Ubiquitous computing - computing in context. PhD thesis, Lancaster University

    Google Scholar 

  69. Schneider M (2004) Towards a Transparent Proactive User Interface for a Shopping Assistant. In: Proc. of Workshop on Multi-User and Ubiquitous User Interfaces (MU3I), Funchal, Madeira, Portugal, vol 3, pp 10–15

    Google Scholar 

  70. Shimoga KB (1993) A survey of perceptual feedback issues in Dexterous telemanipulation: Part II. Finger Touch Feedback. In: Proc. of the IEEE Virtual Reality Annual International Symposium, Piscataway, NJ, IEEE Service Center

    Google Scholar 

  71. Strang T, Linnhoff-popien C (2004) A context modeling survey. In: Proc. of Workshop on Advanced Context Modelling, Reasoning and Management, UbiComp 2004, Nottingham, England

    Google Scholar 

  72. Wahlster W (2002) Smartkom: Fusion and fission of speech, gestures, and facial expressions. In: Proc. of First International Workshop on Man-Machine Symbiotic Systems, pp 213–225

    Google Scholar 

  73. Wahlster W (ed) (2006) SmartKom: Foundations of Multimodal Dialogue Systems. Springer

    Google Scholar 

  74. Walker M, Cahn J, Whittaker S (1997) Improvising linguistic style: Social and affective bases of agent personality. In: Proc. of the 1st International Conference on Autonomous Agents (Agents’97), Marina del Rey, CA, US, pp 96–105

    Chapter  Google Scholar 

  75. Walker M, Litman D, Kamm C, Abella A (1998) Evaluating spoken dialogue agents with PARADISE: Two case studies. Computer Speech and Language 12(4):317–347

    Article  Google Scholar 

  76. Whittaker S, Walker M (2005) Evaluating dialogue strategies in multimodal dialogue systems. In: Minker W, Buehler D, Dybkjaer L (eds) Spoken Multimodal Human-Computer Dialogue in Mobile Environments, Kluwer

    Google Scholar 

  77. Yang J, Stiefelhagen R, Meier U, Waibel A (1998) Real-time face and facial feature tracking and applications. In: Proc. of Workshop on audio-visual speech processing, pp 79–84

    Google Scholar 

  78. Yasuda H, Takahashi K, Matsumoto T (2000) A discrete HMM for online handwriting recognition. Pattern Recognition and Artificial Intelligence 14(5):675–689

    Article  Google Scholar 

  79. Zuckerman O, Maes P (2005) Awareness system for children in distributed families. In: Proc. of the 2005 International Conference on Interaction design for children (IDC 2005), Boulder, Colorado, US

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Languages and Computer Systems, Faculty of Computer Science and Telecommunications, University of Granada, Spain

    Ramón López-Cózar & Zoraida Callejas

Authors
  1. Ramón López-Cózar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zoraida Callejas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ramón López-Cózar or Zoraida Callejas .

Editor information

Editors and Affiliations

  1. Future University Hakodate, Kameda-Nakano 116-2, Hakodate, Hokkaido, 041-8655, Japan

    Hideyuki Nakashima

  2. Department of Electrical Engineering, Stanford University, 350 Serra Mall, Stanford, CA, 94305-9515, USA

    Hamid Aghajan

  3. School of Computing & Mathematics, University of Ulster at Jordanstown, Shore Road, Newtownabbey, Co. Antrim, UK, BT37 0QB

    Juan Carlos Augusto

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

López-Cózar, R., Callejas, Z. (2010). Multimodal Dialogue for Ambient Intelligence and Smart Environments. In: Nakashima, H., Aghajan, H., Augusto, J.C. (eds) Handbook of Ambient Intelligence and Smart Environments. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-93808-0_21

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-93808-0_21

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-93807-3

  • Online ISBN: 978-0-387-93808-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation