Breakfast reading: tabgible UIs, brands, and user centred design

The current issue interactions (http://interactions.acm.org) features some interesting articles – at least scanning through it and reading some of them prolonged my breakfast today considerably 😉

The cover story gives an already in its title a very interesting definition of tangible interaction (Tangible Interaction = Form + Computing) [1]. Their view is very much from a design perspective, but provides a good introduction to the topic.

The article by Jay Chaeyong Yi on a success story from Korea is a very good case study of how to apply UI research [2]. It shows (1) a comprehensive example of user driven research and (2) the value and importance it contributes to a product/service. Even though the article describes a project from 2005 I find the topic of messaging on phone still exciting and the example of IM vs. SMS can still tell a lot. Personally I am really interested in where messaging on the phone goes – perhaps there is some time next week in Tampere to discuss this.

Not central on user interfaces but still quite interesting is the article on operationalizing brands with Web 2.0 technologies [3]. The example of eBags.com shows interestingly what is currently possible. Thinking and imagine a bit further and considering the opportunities that arise from the web/internet of things (e.g. where you know from the bag how it is used and can communicate this) some real change will be ahead. And some companies are slowly getting to the point of bringing first products that go into this direction (Ali pointed me to http://greengoose.com).

After setting up some IKEA furniture with my daughter (model BENNO as bookshelf for paper backs and not for CD/DVDs) I found Don Norman’s article where he discusses the issue consuming vs. producing or spectator vs. creator quite interesting [4].

[1] Baskinger, M. and Gross, M. 2010. Tangible interaction = form + computing. interactions 17, 1 (Jan. 2010), 6-11. DOI= http://doi.acm.org/10.1145/1649475.1649477

[2] Yi, J. C. 2010. User-research-driven mobile user interface innovation: a success story from Seoul. interactions 17, 1 (Jan. 2010), 48-51. DOI= http://doi.acm.org/10.1145/1649475.1649487

[3] Yohn, D. L. 2010. Operationalizing brands with new technologies. interactions 17, 1 (Jan. 2010), 24-27. DOI= http://doi.acm.org/10.1145/1649475.1649481

[4] Norman, D. A. 2010. The transmedia design challenge: technology that is pleasurable and satisfying. interactions 17, 1 (Jan. 2010), 12-15. DOI= http://doi.acm.org/10.1145/1649475.1649478

Ubiquitous Computing – Ever wondered if we are there yet?

Given the technologies around us I sometimes wonder how close we are to a vision of ubiquitous computing. In this month IEEE Computer Invisible Computing column I had the pleasure to ask this question and share my view on it.

The short answer is: many technologies are ubiquitous but there is a lot more to come. In particular we see that many technologies (public displays, people centric sensing, and personal memory devices) are just around the corner and that they may have a large impact on how we perceive computing. For the long answer have a look at my article: ubiquitous computing – are we there yet? [1]. I have taken over responsibility for the invisible computing column from Bill Schilit who introduced the Invisible Computing column in 2003 [2].

Some years ago in 2006 Yvonne Rogers presented her view on how Ubicomp is going forward [3] contrasting it to Weiser’s Vision of calm computing. In her paper she introduces an alternative agenda that argue that we should engage people by ubicomp technologies rather than to make life easy, convenient and calm. Yvonne’s paper is an interesting starting point for getting students into this topic.

[1] Schmidt, A. 2010. Ubiquitous Computing: Are We There Yet? Computer 43, 2 (Feb. 2010), 95-97. DOI= http://dx.doi.org/10.1109/MC.2010.54

[2] Schilit, B. N. 2003. Mega-Utilities Drive Invisible Technologies. Computer 36, 2 (Feb. 2003), 97-99. DOI= http://dx.doi.org/10.1109/MC.2003.1178056

[3] Yvonne Rogers: Moving on from Weiser’s Vision of Calm Computing: Engaging UbiComp Experiences. Ubicomp 2006: 404-421

Books – Christmas break reading

While travelling I came across two very different books. On one of the airports I came through I came across Superfreakonimics by Levitt and Dubner [1] I had also read their earlier book (Freakonomics) as well as The Undercover Economist from Tim Harford – so I got this one. It is funny to read and I enjoyed most of it. The Geo-Engineering statements in the book received quite some critism on the net. So don’t by it for its discussion on climate ;-). Reading the books it seems one gets a good explanation of certain things in the world (and economics) – not sure if this is really true, but it is great fun to read nevertheless. I particular like the argument why emancipation leads to a lower quality of teaching in schools 🙂

A very different book (also with regard to the price; its more on a library budget than a casual airport buy), but not less interesting, is „Awareness systems“ by Panos Markopolous, Boris de Royter and Wendy Mackay [2]. So far I have had only a quick look at the book but this could be the basis for a seminar in a term to come.

[1] Steven D. Levitt, Stephen J. Dubner. Superfreakonomics. 2009. http://freakonomicsbook.com/ ISBN 978-0-7139-9991-4

[2] Markopoulos, Panos; De Ruyter, Boris; Mackay, Wendy (Eds.). Awareness Systems. Advances in Theory, Methodology and Design. 2009, ISBN: 978-1-84882-476-8

PS: I came across another book that takes an extreme – but still to some extent interesting – perspective on the German society. The book is called „Die verblödete Republik“ (the republic that went gaga). In some parts I was reminded of the movie „wag the dog“ – but the book is much more serious about it – providing a lot of references…

From the Internet of Things to the Web of Things

The central role of ICT becomes very visible when it does not work. Sometimes for the good as I was late arriving at DĂŒsseldorf airport but the Airberlin check-in system was down for a few minutes – just enough that I was still in time 🙂

In the evening I met Prof. Lorenz Hilty, who gave a talk in the afternoon at ETH Zurich. I missed the talk but after the interesting and though provoking dinner conversation I decided I should finally really read his book [1] – perhaps over Christmas. Meeting with Friedemann Mattern and Hans Gellersen was very inspiring and I hope we get a change to have future joint projects.

Looking out over ZĂŒrich we talked about the transformation from the internet of things to the world wide web of things. The use of prototcol seems a little technical detail, but in my eye it may have a major impact. The WWW of things is creating a world of networked artefacts (much like the internet of things) but is completely based on Web protocols (e.g. http, RESTful web services). By working with web protocols the objects can easily become part of the web and interact with web-platforms and applications on the www (e.g. facebook, twitter, etc.). I expect by having a WWW of things we enable many more developers to create new and exciting applications on top of the internet of things. There are many challenging research questions. I am particularly interested in how will a good platform look like that empowers web programmers to create and distribute applications on the Web of things. I think we should run a workshop on this in the near future!

[1] Information Technology and Sustainability: Essays on the Relationship between Information Technology and Sustainable Development. Lorenz M. Hilty. 2008.

The computer mouse – next generation?

In my lecture on user interface engineering I start out with a short history of human computer interaction. I like to discuss ideas and inventions in the context of the people who did it, besides others I take about Vannevar Bush and his vision of information processing [1], Ivan Sutherland’s sketchpad [2], Doug Engelbart’s CSCW demo (including the mouse) [3], and Alan Kay’s vision of the Dynabook [4].

One aspect of looking at the history is to better understand the future of interaction with computers. One typical question I ask in class is „what is the ultimate user interface“ and typical answers are „direct interface to my brain – the computer will do what I think“ and „mouse and keyboard“ – both answers showing some insight


As the mouse is still a very import input device (and probably for some time to come) there is a recent paper that I find really interesting. It looks at how the mouse could be enhanced – Nicolas Villar and his colleagues put really a lot of ideas together [5]. The paper is worthwhile to read – but if you don’t have time at least watch it on youtube.

[1] Vannevar Bush, As we may think, Atlantic monthly, July 1945.
[2] Ivan Sutherland, „Sketchpad: A Man-Machine Graphical Communication System“ Technical Report No. 296, Lincoln Laboratory, Massachusetts Institute of Technology via Defense Technical Information Center January 1963. (PDF, youtube).
[3] Douglas Engelbart, the demo 1968. (Overview, youtube)
[4] John Lees. The World In Your Own Notebook (Alan Kay’s Dynabook project at Xerox PARC). The Best of Creative Computing. Volume 3 (1980)
[5] Villar, N., Izadi, S., Rosenfeld, D., Benko, H., Helmes, J., Westhues, J., Hodges, S., Ofek, E., Butler, A., Cao, X., and Chen, B. 2009. Mouse 2.0: multi-touch meets the mouse. In Proceedings of the 22nd Annual ACM Symposium on User interface Software and Technology (Victoria, BC, Canada, October 04 – 07, 2009). UIST ’09. ACM, New York, NY, 33-42. DOI= http://doi.acm.org/10.1145/1622176.1622184

Engineering Thrill – Why is a fairground fun?

On the weekend I went with Vivien to a fairground at the Volksfest in Crailsheim (smaller Version of the Oktoberfest). Vivien is now old enough for some of the attractions – giving me an excuse to try them out 🙂 The forces on the body are pretty exciting when you feel them for the first time or after 20 years again…

It is amazing how much (mechanical) engineering is in these attractions, even though many are still the same as when I was a child (about 30 years ago). It seems that computer science plays a very minor role (besides controlling the mechanics). Virtual reality does not feature at all. What people are attracted to is physical (e.g. a fraction of a second of zero G, great heights, and a life boxing fight).

When I was in Lancaster working in the Equator project I worked with Brendan Walker – and he is the world’s only thrill engineer. He is an aeronautical engineer (Imperial College, London) and Industrial Design Engineer (Royal College of Art, London) by training and had a research grant to investigate how thrill works and how to create thrill. Some of his results are published in a booklet [1] and the work is continued in the Thrilllaboratory [2]. Earlier this year he gave an interview to that gives an intro of his work – a quick and fun read. There are some youtube videos of his work, e.g. http://www.youtube.com/watch?v=0v1M59Aaa2A

[1] Brendan Walker. The Taxonomy of Thrill and Thrilling Designs: Chromo11 (Volumes One and Two). Aerial Publishing (Jan 2005)

[2] http://www.thrilllaboratory.com/

Best papers at MobileHCI 2009

At the evening event of MobileHCI2009 the best paper awards were presented. The best short paper was „User expectations and user experience with different modalities in a mobile phone controlled home entertainment system“ [1]. There were two full papers that got a best paper award: „Sweep-Shake: finding digital resources in physical environments“ [2] and „PhotoMap: using spontaneously taken images of public maps for pedestrian navigation tasks on mobile devices“ [3]. We often look at best papers of a conference to better understand what makes a good paper for this community. All of the 3 papers above are really well done and worthwhile to read.

PhotoMap [3] is a simple but very cool idea. Many of you have probably taken photos of public maps with your mobile phone (e.g. at a park, city map) and PhotoMap explores how to link them to realtime location data from the GPS on the device. The goal is that you can move around in the real space and you have a dot marking where you are on the taken photo. The implementation however seems not completely simple
 There is a youtube movie on PhotoMap (there would be more movies from the evening event – but I do not link them here – the photo above gives you an idea…)

Since last year there is also a history best paper award (most influential paper from 10 years ago). Being at the beginning of a new field sometimes pays of
 I got this award for the paper on implicit interaction [4] I presented in Edinburgh at MobileHCI 1999.

[1] Turunen, M., Melto, A., Hella, J., Heimonen, T., Hakulinen, J., MĂ€kinen, E., Laivo, T., and Soronen, H. 2009. User expectations and user experience with different modalities in a mobile phone controlled home entertainment system. In Proceedings of the 11th international Conference on Human-Computer interaction with Mobile Devices and Services (Bonn, Germany, September 15 – 18, 2009). MobileHCI ’09. ACM, New York, NY, 1-4. DOI= http://doi.acm.org/10.1145/1613858.1613898

[2] Robinson, S., Eslambolchilar, P., and Jones, M. 2009. Sweep-Shake: finding digital resources in physical environments. In Proceedings of the 11th international Conference on Human-Computer interaction with Mobile Devices and Services (Bonn, Germany, September 15 – 18, 2009). MobileHCI ’09. ACM, New York, NY, 1-10. DOI= http://doi.acm.org/10.1145/1613858.1613874

[3] Schöning, J., KrĂŒger, A., Cheverst, K., Rohs, M., Löchtefeld, M., and Taher, F. 2009. PhotoMap: using spontaneously taken images of public maps for pedestrian navigation tasks on mobile devices. In Proceedings of the 11th international Conference on Human-Computer interaction with Mobile Devices and Services (Bonn, Germany, September 15 – 18, 2009). MobileHCI ’09. ACM, New York, NY, 1-10. DOI= http://doi.acm.org/10.1145/1613858.1613876

[4] Albrecht Schmidt. Implicit human computer interaction through context. Personal and Ubiquitous Computing Journal, Springer Verlag London, ISSN:1617-4909, Volume 4, Numbers 2-3 / Juni 2000. DOI:10.1007/BF01324126, pp. 191-199 (initial version presented at MobileHCI1999). http://www.springerlink.com/content/u3q14156h6r648h8/

It is better to look beautiful… Aesthetics and HCI

During the summerschool in Haifa Prof. Noam Tractinsky from Ben-Gurion University of the Negev gave a presentation about Aesthetics in Human-Computer Interaction. It was good to meet him in person and get some more insight in his work – as I refer to it typically in my HCI class.


In short his finding can be summarized by: What is Beautiful is Usable [1], [2]. In his talk he had some interesting example – you can look at a web page for one second only and you will figure out if it is a good design or not. There has been previous work in Japan [3] similar results – suggesting that this may be universial. Methodical I think the research approaches are not straightforward and may be disputed in parts – but the basic findings are very intuitive and should be taken more into account.

[1] Tractinsky, N. 1997. Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, United States, March 22 – 27, 1997). S. Pemberton, Ed. CHI ’97. ACM, New York, NY, 115-122. DOI= http://doi.acm.org/10.1145/258549.258626

[2] Tractinsky, N., Shoval-Katz A. and Ikar, D. (2000) What is Beautiful is Usable. Interacting with Computers, 13(2): 127-145.

[3] Kurosu, M. and Kashimura, K. 1995. Apparent usability vs. inherent usability: experimental analysis on the determinants of the apparent usability. In Conference Companion on Human Factors in Computing Systems (Denver, Colorado, United States, May 07 – 11, 1995). I. Katz, R. Mack, and L. Marks, Eds. CHI ’95. ACM, New York, NY, 292-293. DOI= http://doi.acm.org/10.1145/223355.223680

Meeting Prof. Brian Randell, book recommendations

Yesterday after my talk I met briefly Prof. Brian Randell from the School of Computing Science at Newcastle University. Today we had a chat over a coffee and it is really interesting to think more about dependability implications of ubicomp technologies.

Besides many other points I got a set of interesting pointers to books:

Reading material for summer school.

Tsvi Kuflik and Antonio KrĂŒger organize from August 30th – September 3rd a German-Israeli Minerva School for Ubiquitous Display Environments: Intelligent Group Interaction, Foundations and Implementation of Pervasive Multimodal Interfaces. I will teach a session on: „Embedded interaction with display environments“ and here is the list of recommended readings for the participants – if you are short on time only read the first one and glance over the other two.

Mahato, H., Kern, D., Holleis, P., and Schmidt, A. 2008. Implicit personalization of public environments using bluetooth. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 3093-3098. DOI= http://doi.acm.org/10.1145/1358628.1358813 (if you do not have access to ACM here is a copy on the web)

Schmidt, A.; van Laerhoven, K. 2001. How to build smart appliances? Personal Communications, IEEE. Volume 8, Issue 4, Aug 2001:66 – 71. DOI: 10.1109/98.944006. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=944006 (if you do not have access to IEEE here is a copy on the web)

Villar N.; Schmidt A.; Kortuem G.; Gellersen H.-W. 2003. Interacting with proactive public displays. Computers and Graphics, Elsevier. Volume 27, Number 6, December 2003 , pp. 849-857(9). Draft Version „Interacting with proactive community displays“ online available.