Skip to content
BY-NC-ND 3.0 license Open Access Published by De Gruyter Open Access December 27, 2011

iCompose: context-aware physical user interface for application composition

  • Oleg Davidyuk EMAIL logo , Ekaterina Gilman , Iván Milara , Jussi Mäkipelto , Mikko Pyykkönen and Jukka Riekki
From the journal Open Computer Science

Abstract

Ubiquitous computing environments advocate creating applications by plugging together various resources (mobile devices, displays, augmented everyday objects, and so on) and Web Services to support the user’s everyday activities and needs. This approach is referred to as application composition, and such applications are called composite. Due to the dynamic nature of ubiquitous environments, application composition has to be supported at runtime, so that the applications are able to adapt to the situation within the environment and other contexts. Application composition is usually performed by autonomous context-aware mechanisms that provide limited or no control for users. Still, users need to be aware of their environments and be able to control and configure applications when they are composed and executed. Towards this goal we present in this article a context-aware application composition system based on the iCompose interface for composing and controlling applications at runtime. Users compose applications by simply touching the resources in the environment with their mobile devices, while the iCompose interface provides feedback and assists users by suggesting them possible further actions. The interface relies on a rule-based reasoner and utilizes various context sources to support users dynamically, according to the situation in which they compose applications. We present a complete implementation of the system and report the results of a user evaluation study conducted with 21 participants. This study assesses the issues of control, usability, feasibility and user acceptance of the iCompose interface for context-aware application composition and the prototype as a whole.

[1] Abowd G.D., Dey A.K., Brown P.J., Davies N., Smith M., Steggles P., Towards a better understanding of context and context-awareness, In: Proceedings of the 1st International Symposium on Handheld and Ubiquitous Computing, Gellersen H-W. (Ed.), London, UK, 304–307, 1999 10.1007/3-540-48157-5_29Search in Google Scholar

[2] Ailisto H., Matinmikko T., Ylisaukko-Oja A., Strommer E., Hillukkala M., Wallin A., Siira E., Poyry A., Tormanen V., Huomo T., Physical browsing with NFC technology, VTT Tiedotteita, 2400, 2007 Search in Google Scholar

[3] Beauche S., Poizat P., Automated service composition with adaptive planning, In: Proceedings of the 6th International Conference on Service-Oriented Computing (ICSOC’08), Bouguettaya A., Kruger I., Margaria T. (Eds.), Sydney, Australia, 530–537, 2008 10.1007/978-3-540-89652-4_42Search in Google Scholar

[4] Ben Mokhtar S., Georgantas N., Issarny V., COCOA: COnversation-based service composition in pervasive computing environments with QoS support, J. Syst. Software, 80(12), 1941–1955, 2007 http://dx.doi.org/10.1016/j.jss.2007.03.00210.1016/j.jss.2007.03.002Search in Google Scholar

[5] Bertolino A., Angelis G., Frantzen L., Polini A., The PLASTIC framework and tools for testing service-oriented applications, In: Proceedings of the International Summer School on Software Engineering (ISSSE 2006–2008), De Lucia A., Ferrucci F. (Eds.), Springer, 106–139, 2009 10.1007/978-3-540-95888-8_5Search in Google Scholar

[6] Bikakis A., Antoniou G., Distributed defeasible contextual reasoning in ambient computing, In: Proceedings of the 2008 European Conference on Ambient Intelligence (AmI’08), Aarts E., Crowley J., Gerhauser H., Pflaum A., Schmidt J., Wichert R. (Eds.), Nuremberg, Germany 10.1007/978-3-540-89617-3_20Search in Google Scholar

[7] Bottaro A., Bourcier J., Escoffier C., Lalanda P., Context-aware service composition in a home control gateway, In: Proceedings IEEE International Conference on Pervasive Services, Istanbul, Turkey Search in Google Scholar

[8] Broll G., Rukzio E., Paolucci M., Wagner M., Schmidt A., Hussmann H., PERCI: Pervasive service interaction with the Internet of things, IEEE Internet Comput., 13(6), 74–81, 2009 http://dx.doi.org/10.1109/MIC.2009.12010.1109/MIC.2009.120Search in Google Scholar

[9] Brønsted J., Hansen K.M., Ingstrup M., Service composition issues in pervasive computing, IEEE Pervas. Comput., 9(1), 62–70, 2010 http://dx.doi.org/10.1109/MPRV.2010.1110.1109/MPRV.2010.11Search in Google Scholar

[10] Buford J., Kumar R., Perkins G., Composition trust bindings in pervasive computing service composition, In: Proceedings of the 4th Annual IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOMW’06), Werner B. (Ed.), Pisa, Italy, 261–266, 2006 http://dx.doi.org/10.1109/PERCOMW.2006.3110.1109/PERCOMW.2006.31Search in Google Scholar

[11] Cadenas A., Ruiz C., Larizgoitia I., García-Castro R., Lamsfus C., Vázquez I., González M., Martín D., Poveda M., Context management in mobile environments: a semantic approach, In: Proceedings of the 1st Workshop on Context, Information and Ontologies, Gomez-Perez J., Haase P., Tilly M., Warren P. (Eds.), Heraklion, Greece, 2:1–2:8, 2009 Search in Google Scholar

[12] Chang S.C., Liao C.F., Liu Y.C., Fu L.C., A spontaneous preference aware service composition framework for messageoriented pervasive systems, In: Proceedings of the 4th International Conference on Pervasive Computing and Applications (ICPCA’09), Tamsui, Taipei, 441–446, 2009 Search in Google Scholar

[13] Chantzara M., Anagnostou M., Sykas E., Designing a quality-aware discovery mechanism for acquiring context information, In: Proceedings of the 20th International Conference on Advanced Information Networking and Applications (AINA’06), Vienna, Austria, 211–216, 2006 10.1109/AINA.2006.142Search in Google Scholar

[14] Chin J., Callaghan V., Clarke G., An end-user tool for customising personal spaces in ubiquitous computing environments, In: Proceedings of the 3rd International Conference on Ubiquitous Intelligence and Computing (UIC’06), Jin H., Laurence T.T., Jeffrey J.-P. (Eds.), Three Gorges, China, 1080–1089, 2006 10.1007/11833529_109Search in Google Scholar

[15] Davidyuk O., Sánchez I., Duran J.I., Riekki J., Autonomic composition of ubiquitous multimedia applications in REACHES, In: Proceedings of the 7th International ACM Conference on Mobile and Ubiquitous Multimedia (MUM’08), Wiberg M., Zaslavsky A. (Eds.), Umea, Sweden, 105–108, 2008 http://dx.doi.org/10.1145/1543137.154315910.1145/1543137.1543159Search in Google Scholar

[16] Davidyuk O., Sánchez I., Riekki J., CADEAU: supporting autonomic and user-controlled application composition in ubiquitous environments, Pervasive Computing and Communications Design and Deployment: Technologies, Trends, and Applications, Apostolos M. (Ed.), 74–103, 2011 10.4018/978-1-60960-611-4.ch004Search in Google Scholar

[17] Davidyuk O., Selek I., Duran J.I., Riekki J., Algorithms for composing pervasive applications, International Journal of Software Engineering and Its Applications, 2(2), 71–94, 2008 Search in Google Scholar

[18] de Mey V., Visual composition of software applications, In: Object-oriented software composition, Nierstrasz O., Tsichritzis, D., (Eds.), Prentice Hall, UK, 275–303, 1995 Search in Google Scholar

[19] Dey A.K., Newberger A., Support for context-aware intelligibility and control, In: Proceedings of the 27th international conference on Human factors in computing systems, Olsen D. et al (Eds.), Boston, MA, USA, 859–868, 2009 10.1145/1518701.1518832Search in Google Scholar

[20] Ghiani G., Paternï F., Spano L.D., Cicero Designer: an environment for end-user development of multi-device museum guides, In: Proceedings of the 2nd International Symposium on End-User Development (IS-EUD’09), Volkmar P. et al (Eds.), Siegen, Germany, 265–274, 2009 10.1007/978-3-642-00427-8_15Search in Google Scholar

[21] Gilman E., Su X., Davidyuk O., Zhou J., Riekki J., Perception framework for supporting development of context-aware web services, International Journal of Pervasive Computing and Communications, 7(4), 339–364, 2011 http://dx.doi.org/10.1108/1742737111118966510.1108/17427371111189665Search in Google Scholar

[22] Gross T., Marquardt N., Creating, editing and sharing complex ubiquitous computing environment configurations with CollaborationBus, J. Parallel. Distr. Com., 11(3), 289–303, 2010 Search in Google Scholar

[23] Hardian B., Indulska J., Henricksen K., Exposing contextual information for balancing software autonomy and user control in context-aware systems, In: Proceedings of the Workshop on Context-Aware Pervasive Communities: Infrastructures, Services and Applications, Sydney, Australia, 2008 Search in Google Scholar

[24] Hesselman C., Tokmakoff A., Pawar P., Iacob S., Discovery and composition of services for context-aware systems, In: Smart sensing and context, Havinga P., Lijding M.E.M., Meratnia N., Wegdam M. (Eds.), Heidelberg, Berlin, 67–81, 2006 10.1007/11907503_6Search in Google Scholar

[25] Ibrahim N., Le Mouel F., A survey on service composition middleware in pervasive environments, International Journal of Computer Science Issues, 1, 1–12, 2009 Search in Google Scholar

[26] Kalasapur S., Kumar M., Shirazi B., Dynamic service composition in pervasive computing, IEEE T. Parall. Distr., 18(7), 907–918, 2007 http://dx.doi.org/10.1109/TPDS.2007.103910.1109/TPDS.2007.1039Search in Google Scholar

[27] Kawsar F., Nakajima T., Fujinami K., Deploy spontaneously: supporting end-users in building and enhancing a smart home, In: Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp’08), Seoul, Korea, 282–291, 2008 10.1145/1409635.1409673Search in Google Scholar

[28] Koch F., Meyer J.J.C., Dignum F., Rahwan I., Programming deliberative agents for mobile services: the 3APLM platform, In: Proceedings of the AAMAS’05 Workshop on Programming Multi-Agent Systems (ProMAS’05), Utrecht, Netherlands, 222–235, 2005 10.1007/11678823_14Search in Google Scholar

[29] Laranjeira M.V., Social interaction in asynchronous learning environments, Anglogermanica Online, 13–26, 2006 Search in Google Scholar

[30] Lindenberg J., Pasman W., Kranenborg K., Stegeman J., Neerincx M.A., Improving service matching and selection in ubiquitous computing environments: a user study, Pers. Ubiquit. Comput., 11(1), 59–68, 2006 http://dx.doi.org/10.1007/s00779-006-0066-710.1007/s00779-006-0066-7Search in Google Scholar

[31] Masuoka R., Parsia B., Labrou Y., Task computing — the semantic web meets pervasive computing, In: Proceedings of the 2nd International Semantic Web Conference (ISWC’03), Sanibel Island, Florida, USA, 866–881, 2003 10.1007/978-3-540-39718-2_55Search in Google Scholar

[32] Mavrommati I., Kameas A., Markopoulos P., An editing tool that manages device associations in an in-home environment, Pers. Ubiquit. Comput., 8(3–4), 255–263, 2004 10.1007/s00779-004-0286-7Search in Google Scholar

[33] Meloche J., Hasan H., Papakosmas A., Support for asynchronous interaction in group experiential learning, Outlines. Critical Practice Studies, 6(2), 47–62, 2004 10.7146/ocps.v6i2.2144Search in Google Scholar

[34] Messer A., Kunjithapatham A., Sheshagiri M., Song H., Kumar P., Nguyen P., Yi K.H., InterPlay: a middleware for seamless device integration and task orchestration in a networked home, In: Proceedings of the 4th Annual IEEE Conference on Pervasive Computing and Communications, Pisa, Italy, 296–307, 2006 Search in Google Scholar

[35] Mokhtar S.B., Fournier D., Georgantas N., Issarny V., Context-aware service composition in pervasive computing environments, In: 2nd International Workshop on Rapid Integration of Software Engineering Techniques (RISE’05), Guelfi N., Savidis A. (Eds.), Heraklion, Crete, Greece, 129–144, 2005 10.1007/11751113_10Search in Google Scholar

[36] Mugellini E., Abou Khaled O., Pierroz S., Carrino S., Chabbi Drissi H., Generic framework for transforming everyday objects into interactive surfaces, In: Proceedings of the 13th International Conference on Human-Computer Interaction, Part III, Jacko J.A. (Ed.), San Diego, CA, USA, 473–482, 2009 10.1007/978-3-642-02580-8_52Search in Google Scholar

[37] Mugellini E., Rubegni E., Gerardi S., Khaled O.A., Using personal bjects as tangible interfaces for memory recollection and sharing, In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI’07), Ullmer B., Schmidt A. (Eds.), Baton Rouge, Louisiana, USA, 231–238, 2007 10.1145/1226969.1227016Search in Google Scholar

[38] Mukhtar H., Belaïd D., Bernard G., Dynamic user task composition based on user preferences, ACM Trans. Auton. Adap., 6(1), 4:1–4:17, 2011 10.1145/1921641.1921645Search in Google Scholar

[39] Nakazawa J., Yura J., Tokuda H., Galaxy: a service shaping approach for addressing the hidden service problem, In: Proceedings of the 2nd IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, Vienna, Austria, 35–39, 2004 Search in Google Scholar

[40] Newman M., Ackerman M., Pervasive Help @ Home: connecting people who connect devices, In: Proceedings of the International Workshop on Pervasive Computing at Home (PC@Home), Sydney, Australia, 28–36, 2008 Search in Google Scholar

[41] Newman M., Elliott A., Smith T., Providing an integrated user experience of networked media, devices, and services through end-user composition, In: Proceedings of the 6th International Conference on Pervasive Computing (Pervasive’08), Sydney, Australia, 213–227, 2008 10.1007/978-3-540-79576-6_13Search in Google Scholar

[42] Paluska J.M., Pham H., Saif U., Chau G., Terman C., Ward S., Structured decomposition of adaptive applications, Pervasive and Mobile Computing, 4(6), 791–806, 2008 http://dx.doi.org/10.1016/j.pmcj.2008.04.00610.1016/j.pmcj.2008.04.006Search in Google Scholar

[43] Paluska J.M., Pham H., Saif U., Chau G., Terman C., Ward S., Structured decomposition of adaptive applications, In: Proceedings of the 6th Annual IEEE International Conference on Pervasive Computing and Communications (PerCom’08), Hong Kong, 1–10, 2008 10.1109/PERCOM.2008.55Search in Google Scholar

[44] Preuveneers D., Berbers Y., Automated context-driven composition of pervasive services to alleviate non-functional concerns, International Journal of Computing and Information Sciences, 3(2), 19–28, 2005 Search in Google Scholar

[45] Przybilski M., Distributed context reasoning for proactive systems, In: Proceedings of the 2005 Workshop on Context-Awareness for Proactive Systems (CAPS’05), Helsinki, Finland, 43–53, 2005 Search in Google Scholar

[46] Ranganathan A., Campbell R.H., Autonomic pervasive computing based on planning, In: Proceedings of the International Conference on Autonomic Computing, Los Alamitos, CA, USA, 80–87, 2004 Search in Google Scholar

[47] Rantapuska O., Lahteenmaki M., Task-based user experience for home networks and smart spaces, In: Proceedings of the International Workshop on Pervasive Mobile Interaction Devices, Sydney, Australia, 188–191, 2008 Search in Google Scholar

[48] Rich C., Sidner C., DiamondHelp: a generic collaborative task guidance system, AI Mag., 28(2), 33–46, 2007 Search in Google Scholar

[49] Riekki J., RFID and smart spaces, International Journal of Internet Protocol Technology, 2(3–4), 143–152, 2007 http://dx.doi.org/10.1504/IJIPT.2007.01621610.1504/IJIPT.2007.016216Search in Google Scholar

[50] Riekki J., Sánchez I., Pyykkonen M., Remote control for pervasive services, International Journal of Autonomous and Adaptive Communications Systems, 3(1), 39–58, 2010 http://dx.doi.org/10.1504/IJAACS.2010.03031110.1504/IJAACS.2010.030311Search in Google Scholar

[51] Rigole P., Clerckx T., Berbers Y., Coninx K., Task-driven automated component deployment for ambient intelligence environments, Pervasive and Mobile Computing, 3(3), 276–299, 2007 http://dx.doi.org/10.1016/j.pmcj.2007.01.00110.1016/j.pmcj.2007.01.001Search in Google Scholar

[52] Rigole P., Vandervelpen C., Luyten K., Berbers Y., Vandewoude Y., Coninx K., A component-based infrastructure for pervasive user interaction, In: Proceedings of Software Techniques for Embedded and Pervasive Systems, Varea M., Cortes L. (Eds.), Munich, Germany, 1–16, 2005 Search in Google Scholar

[53] Riva O., Contory: a middleware for the provisioning of context information on smart phones, In: Proceedings of the ACM/IFIP/USENIX 7th International Middleware Conference (Middleware’06), van Steen M., Henning M. (Eds.), Melbourne, Australia, 219–239, 2006 10.1007/11925071_12Search in Google Scholar

[54] Rizou S., Haussermann K., Durr F., Cipriani N., Rothermel K., A System for distributed context reasoning, In: Proceedings of the 6th International Conference on Autonomic and Autonomous Systems (ICAS’10), Los Alamitos, CA, USA, 84–89, 2010 10.1109/ICAS.2010.21Search in Google Scholar

[55] Rouvoy R., Barone P., Ding Y., Eliassen F., Hallsteinsen S.O., Lorenzo J., Mamelli A., Scholz U., MUSIC: middleware support for self-adaptation in ubiquitous and service-oriented environments, In: Proceedings of Dagstuhl Seminar: Software Engineering for Self-Adaptive Systems, Cheng B., Lemos R., de Inverardi P., Magee J. (Eds.), Schloss Dagstuhl, Germany, 164–182, 2009 http://dx.doi.org/10.1007/978-3-642-02161-9_910.1007/978-3-642-02161-9_9Search in Google Scholar

[56] Rukzio E., Leichtenstern K., Callaghan V., Holleis P., Schmidt A., Chin J.S.Y., An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning, In: Proceedings on the 8th International Conference on Ubiquitous Computing (UbiComp’06), Dourish P., Friday A. (Eds.), Orange County, CA, USA, 87–104, 2006 10.1007/11853565_6Search in Google Scholar

[57] Saarenpää H., Korhonen H., Paavilainen J., Asynchronous gameplay in pervasive multiplayer mobile games, In: Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems. Olsen D. Jr. et al (Eds.), Boston, MA, USA, 4213–4218, 2009 10.1145/1520340.1520642Search in Google Scholar

[58] Sánchez I., Riekki J., Pyykkonen M., Touch&Compose: physical user interface for application composition in smart environments, In: Proceedings of the International Workshop on Near Field Communication, Hagenberg, Austria, 61–66, 2009 10.1109/NFC.2009.11Search in Google Scholar

[59] Schuster M., Domene A., Vaidya R., Arbanowski S., Kim S.M., Lee J.W., Lim H., Virtual device composition, In: Proceedings of the 8th International Symposium on Autonomous Decentralized Systems (ISADS’07), Sedona, USA, 270–278, 2007 10.1109/ISADS.2007.78Search in Google Scholar

[60] Shneiderman B., Plaisant C., Designing the user interface: strategies for effective human-computer interaction, (4th edition), Pearson Addison Wesley, Reading, MA, 2004 Search in Google Scholar

[61] Sousa J.P., Poladian V., Garlan D., Schmerl B., Shaw M., Task-based adaptation for ubiquitous computing, IEEE T. Syst. Man Cy. C, 36, 328–340, 2006 http://dx.doi.org/10.1109/TSMCC.2006.87158810.1109/TSMCC.2006.871588Search in Google Scholar

[62] Sousa J.P., Schmerl B., Poladian V., Brodsky A., uDesign: end-user design applied to monitoring and control applications for smart spaces, In: Proceedings of the Working IEEE/IFIP Conference on Software Architecture, Kruchten P., Garlan D., Woods E. (Eds.), Vancouver, Canada, 71–80, 2008 10.1109/WICSA.2008.46Search in Google Scholar

[63] Takemoto M., Oh-ishi T., Iwata T., Yamato Y., Tanaka Y., Shinno K., Tokumoto S., Shimamoto N., A service-composition and service-emergence framework for ubiquitous-computing environments, In: Proceedings of the 2004 Workshop on Applications and the Internet (as part of SAINT’04), Tokyo, Japan, 313–318, 2004 10.1109/SAINTW.2004.1268653Search in Google Scholar

[64] Urbieta A., Barrutieta G., Parra J., Uribarren A., A survey of dynamic service composition approaches for ambient systems, In: Proceedings on Ambi-Sys Workshop on Software Organisation and Monitoring of Ambient Systems (SOMITAS’08), Quebec City, Canada, 1–8, 2008 10.4108/ICST.AMBISYS2008.2842Search in Google Scholar

[65] Välkkynen P., Niemelä M., Tuomisto T., Evaluating touching and pointing with a mobile terminal for physical browsing, In: Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (NordiCHI’06), Morch A., Morgan K., Bratteteig T., Ghosh G., Svanaes D. (Eds.), Oslo, Norway, 28–37, 2006 10.1145/1182475.1182479Search in Google Scholar

[66] Vanderhulst G., Luyten K., Coninx K., Put the user in control: ontology-driven meta-level interaction for pervasive environments, In: Proceedings of the 1st International Workshop on Ontologies in Interactive Systems, Bouamrane M. et al (Eds.), Liverpool, UK Search in Google Scholar

[67] Vastenburg M., Keyson D., de Ridder H., Measuring user experiences of prototypical autonomous products in a simulated home environment, Human Computer Interaction (HCI), 2, 998–1007, 2007 10.1007/978-3-540-73107-8_110Search in Google Scholar

[68] Viterbo J., Endler M., Baptista G., A two-tiered approach for decentralized reasoning in ambient intelligence, IEEE Intell. Syst., 25(5), 54–60, 2010 http://dx.doi.org/10.1109/MIS.2010.2910.1109/MIS.2010.29Search in Google Scholar

[69] Watt J., Walther J., Nowak K., Asynchronous videoconferencing: a hybrid communication prototype, In: Proceedings of the 35th Annual Hawaii International Conference on System Sciences, Sprague R.H. (Ed.), Big Island, HI, USA, 97–105, 2002 Search in Google Scholar

[70] Weiser M., The computer for the 21st century, Sci. Am., 265, 94–104, 1991 http://dx.doi.org/10.1038/scientificamerican0991-9410.1038/scientificamerican0991-94Search in Google Scholar

[71] Yang Y., Mahon F., Williams M., Pfeifer T., Context-aware dynamic personalised service re-composition in a pervasive service environment, Lect. Notes. Comput. Sc., 4159, 724–735, 2006 http://dx.doi.org/10.1007/11833529_7410.1007/11833529_74Search in Google Scholar

[72] Zhang B., Shi Y., Xiao X., A policy-driven service composition method for adaptation in pervasive computing environment, Comput. J., 53(2), 152–165, 2007 http://dx.doi.org/10.1093/comjnl/bxm10310.1093/comjnl/bxm103Search in Google Scholar

[73] Zhou J., Gilman E., Palola J., Riekki J., Ylianttila M., Sun J., Context-aware pervasive service composition and its implementation, Pers. Ubiquit. Comput., 15, 291–303, 2011 http://dx.doi.org/10.1007/s00779-010-0333-510.1007/s00779-010-0333-5Search in Google Scholar

Published Online: 2011-12-27
Published in Print: 2011-12-1

© 2011 Versita Warsaw

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Downloaded on 1.10.2024 from https://www.degruyter.com/document/doi/10.2478/s13537-011-0031-z/html
Scroll to top button