Guests in my multimodal interaction class

Today I had brought 3 more professors with me to teach the class on multimodal interaction (I learned from Hans). As we have the pd-net project meeting Nigel Davies, Marc Langheirich, and Rui Jose were in Stuttgart and ‘volunteered’ to give a talk.

Nigel talked about the work in Lancaster on the use of mobile computing technology to support sustainable travel. He explained the experiments they conducted for collecting and sharing travel related information. In the 6th Sense Transport project they look beyond looking at understanding the current context into predictions and eventually ‘time travel’ 😉

Marc presented a one hour version of his tutorial on privacy introducing the terminology and explaining the many facets this topic has. We discussed the ‘NTHNTF’ argument (Nothing To Hide Nothing To Fear) and Marc used an example of to show the weaknesses of this argument. Marc suggested some reading if you want to dive into the topic, see [1,2,3,4].

Rui focused in his lecture on pervasive public displays. He gave an overview of typical architectures for digital signage systems and the resulting limitation. The pd-net approach aims at creating an open platform that allows many different applications and use cased. He showed once concept of using virtual pin-badges to trigger content and to express interest in a certain topic.

There is more information on the pd-net project on

[1] David Brin. The Transparent Society. Perseus Publishing, 1999.
[2] Simson Garfinkel: Database Nation – The Death of Privacy in the 21st Century. O’Reilly, 2001.
[3] Lawrence Lessig: Code and Other Laws of Cyberspace. Basic Books, 2006.
[4] Waldo, Lin, Millett (eds.): Engaging Privacy and Information Technologygy in a Digital Age. National Academies Press, 2007.

Call for Papers: Symposium on Pervasive Display Networks

Rui José and Elaine Huang are chairing an international symposium on pervasive displays in Portugal. The conference will be held June 4-5 2012 in Porto. The submission deadline for full papers is January 16th, 2012.

With our research in the PD-net project we encounter many interesting research questions and met with many other researchers interested in the topic. It seems that the many real deployments of electronic displays is fueling ideas and makes it obvious that research is required to understand the properties of this new upcoming media. The call states: “As digital displays become pervasive, they become increasingly relevant in many areas, including advertising, art, sociology, engineering, computer science, interaction design, and entertainment.

We hope with this symposium we will bring together researchers and practitioners as well as users to share research results and generate new ideas.

Submissions that report on cutting-edge research in the broad spectrum of pervasive digital displays are invited, ranging from large interactive walls to personal projection, from tablets and mobile phone screens to 3-D displays and tabletops. Topics include:

  • Novel technologies
  • Architecture
  • Applications
  • Domains and formative studies studies
  • Evaluations and deployments
  • Interfaces and interaction techniques
  • Content design

Have a look at the webpage and the call for paper at

Large pixels along the underpass

In the refurbished railway station (not yet finished) there is an interesting new pixel display in one main underpass. One wall is covered with a display. It is about 10 pixel (probably about 4 meters) high and several hundred pixels long (have not counted/measured them). It changes colors and shows writing (so far not really exciting).

How cool would it be if there is a freely accessible programmable web-service to control these pixel? I would guess people could create all sorts of interesting content… Perhaps people would start to bargain to get their 5 minutes of virtual graffiti shown…

More surface interaction using audio: Scratch input

After my talk at the Minerva School Roy Weinberg pointed me to a paper by Chris Harrison and Scott Hudson [1] – it also uses audio for creating an interactive surface. The novelty on the technical side is limited but nevertheless the approach is interesting and appealing because of its simplicity and its potential (e.g. just think beyond a fingernail on a table to any contact movement on surfaces – pushing toy cars, walking, pushing a shopping trolley…). Perhaps having a closer look at this approach a generic location system could be created (e.g. using special shoe soles that make a certain noise).

There is a youtube movie:

Besides his studies Roy develops software for the Symbian platform and he sells a set of interesting applications.

[1] Harrison, C. and Hudson, S. E. 2008. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. In Proceedings of the 21st Annual ACM Symposium on User interface Software and Technology (Monterey, CA, USA, October 19 – 22, 2008). UIST ’08. ACM, New York, NY, 205-208. DOI=

Interaction technologies for display environments

I was invited to give a talk on „Embedded interaction with display environments“ to discuss human computer interaction and technology issue for creating interactive display systems. The summer school has very diverse program! and I have enjoyed listening to my colleagues as much as presenting myself 🙂

In the talk I have a (more or less random) selection of technologies for making display environments interactive. There are the obvious vision based approaches (see the talk for the references) but I think there are many interesting approaches that are not yet fully explored. – including spatial audio location [1], eye tracking, and physiological sensors. Sebastian Boring create a focus and context input by combing different input technologies [2] – this can be especially interesting when scaling interaction up to larger surfaces. Additionally I think looking at the floor and the ceiling is worthwhile…

Please feel free to add further technologies and approaches for creating interactive displays in the comment.

[1] James Scott, Boris Dragovic: Audio Location: Accurate Low-Cost Location Sensing. Pervasive Computing: Third International Conference, PERVASIVE 2005, Munich, Germany, May 8-13, 2005. Springer LNCS 3468/2005. pp 1-18.

[2] S. Boring, O. Hilliges, A. Butz. A Wall-sized Focus plus Context Display. In Proceedings of the Fifth Annual IEEE Conference on Pervasive Computing and Communications (PerCom), New York, NY, USA, Mar. 2007

Reading material for summer school.

Tsvi Kuflik and Antonio Krüger organize from August 30th – September 3rd a German-Israeli Minerva School for Ubiquitous Display Environments: Intelligent Group Interaction, Foundations and Implementation of Pervasive Multimodal Interfaces. I will teach a session on: „Embedded interaction with display environments“ and here is the list of recommended readings for the participants – if you are short on time only read the first one and glance over the other two.

Mahato, H., Kern, D., Holleis, P., and Schmidt, A. 2008. Implicit personalization of public environments using bluetooth. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 3093-3098. DOI= (if you do not have access to ACM here is a copy on the web)

Schmidt, A.; van Laerhoven, K. 2001. How to build smart appliances? Personal Communications, IEEE. Volume 8, Issue 4, Aug 2001:66 – 71. DOI: 10.1109/98.944006. (if you do not have access to IEEE here is a copy on the web)

Villar N.; Schmidt A.; Kortuem G.; Gellersen H.-W. 2003. Interacting with proactive public displays. Computers and Graphics, Elsevier. Volume 27, Number 6, December 2003 , pp. 849-857(9). Draft Version „Interacting with proactive community displays“ online available.

Visit to NEC labs in Heidelberg

In the afternoon I gave a talk at NEC labs in Heidelberg on ubiquitous display networks. Over the last year we developed and number of ideas and prototypes of interactive public display systems. We run a lab class (Fallstudien) on pervasive computing technologies and advertising together with colleagues from marketing. In another class (Projektseminar) we investigated how to facilitate interaction between interactive surfaces (e.g. multi touch table) and mobile devices. One of the prototypes will be shown as poster at mobile HCI 2009 in Bonn. In some thesis projects we introduced the notion of mobile contextual displays and their potential applications in advertising, see [1] and [2].

Seeing the work at NEC and based on the discussion I really think there is a great of potential for ubiquitous display networks – at the same time there are many challenges – including privacy that allways ensures discussion 😉 It would be great to have another bachelor or master thesis to address some of them – perhaps jointly with people from NEC. To understand the information needs in a particular display environment (at the University of Duisburg-Essen) we currently run a survey to better understand requirements. If you read German you are welcome to participate in the survey.

Predicting the future usually features in my talks – and interestingly I go a recommendation from Miquel Martin for a book that takes its own angle on that: Predictably Irrational by Dan Ariely (the stack of book gets slowly to large – time for holidays).

[1] Florian Alt, Albrecht Schmidt, Christoph Evers: Mobile Contextual Displays. Pervasive Advertising Workshop @ Pervasive 2009. Nara, Japan 2009.

[2] Florian Alt, Christoph Evers, Albrecht Schmidt: Users‘ View on Car Advertisements. In: Proceedings of the Seventh International Conference on Pervasive Computing, Pervasive’09. Springer Berlin / Heidelberg Nara, Japan 2009.

Our Publications at Pervasive – Public Displays, Car Adverts, and Tactile Output for Navigation

Our group was involved in 3 papers that are published at Pervasive 2009 in Nara.

The first contribution is a study on public display that was presented by Jörg Müller from Münster. The paper explores display blindness that can be observed in the real world (similarly to banner blindness) and concludes that the extent to which people look at displays is very much correlated to the users expectation of the content of a display in a certain location [1].

The second short paper is a survey on car advertising and has been conducted in the context of the master thesis of Christoph Evers. The central question is about the design space of dynamic advertising on cars and how the users perceive such a technology [2].

Dagmar presented a paper on vibra-tactile output integrated in the steering wheel for navigation systems in cars. The studies explored how multi-modal presentation of information impact driving performance and what modalities are preferred by users. The general conclusion is that combining visual information with vibra-tactile output is the best option and that people prefer multi-modal output over a single modality [3].

[1] Jörg Müller, Dennis Wilmsmann, Juliane Exeler, Markus Buzeck, Albrecht Schmidt, Tim Jay, Antonio Krüger. Display Blindness: The Effect of Expectations on Attention towards Digital Signage. 7th International Conference on Pervasive Computing 2009. Nara, Japan. Springer LNCS 5538, pp 1-8.

[2] Florian Alt, Christoph Evers, Albrecht Schmidt. User’s view on Context-Aware Car Advertisement. 7th International Conference on Pervasive Computing 2009. Nara, Japan. Springer LNCS 5538, pp 9-16.

[3] Dagmar Kern, Paul Marshall, Eva Hornecker, Yvonne Rogers, Albrecht Schmidt. Enhancing Navigation Information with tactile Output Embedded into the Steering Wheel. 7th International Conference on Pervasive Computing 2009. Nara, Japan. Springer LNCS 5538, pp 42-58.

Meeting on public display networks

Sunday night I travelled to Lugano for a meeting public display networks. I figured out that going there by night train is the best option – leaving midnight in Karlsruhe and arriving at 6am there. As I planned to sleep all the time my assumption was that the felt travel time would be zero. Made my plan without the rail company… the train was 2 hours late and I walked up and down for 2 hours in Karlsruhe at the track – and interestingly the problem would have been less annoying if public displays would provide the relevant information … The most annoying thing was passengers had no information if or when the train will come and no one could tell (neither was anyone at the station nor was anyone taking calls at the hotline).
The public display – really nice state of the art hardware – showed for 1 hour nothing and that it showed that the train is one hour late (was already more than 1 hour after the scheduled time) and finally the train arrived 2 hours late (the display still showing 1 hour delay). How hard can it be to provide this information? It seems with current approaches it is too hard…

On my way back I could observe a further example of short comings with content on public display. In the bus office they had a really nice 40-50 inch screen showing teletext of the departure. The problem was it was the teletext for the evening as the staff has to manually switch the pages. Here too it is very clear the information is available but current delivery systems are not well integrated.

In summary it is really a pity how poorly the public display infrastructures are used. It seems there are a lot of advances in the hardware but little on the content delivery, software and system side.

Trip to Dublin, Aaron’s Display Project

Visiting Dublin is always a pleasure – even if the weather is rainy. Most of the day I was at Trinity College reading master theses (which is the second best part of being external examiner, best part is to have lunch at the 1592 😉
In the evening I met with Aaron Quigley and we talked about some ongoing display and advertsing projects in our groups. He told me about one of their recent workshop papers [1] on public displays where they investigated what people take in and what people remember of the content on displays in an academic environment. It is online available in the workshop proceedings of AIS08 [2]. I found it worthwhile to browse the whole workshop proceedings.
[1] Rashid U. and Quigley A., „Ambient Displays in Academic Settings: Avoiding their Underutilization“, Ambient Information Systems Workshop at UbiComp 2008, September 21, Seoul, South Korea (download [2], see page 26 ff)