History and Future of Computing and Interaction

Today I was teaching my class on user interface engineering and we covered a selected history of HCI and looked at the same time at a potential future. We discussed how user interface evolved and where UI revolutions have happed. To my question „What is the ultimate user interface?“ I got three very interesting answers (1) a keyboard, (2) mind reading, and (3) a system that anticipates what I want. 
With regard to history in HCI one of my favorite texts is the PhD dissertation of Ivan Sutherland [1]. The work described was done in 1960-1963 when the idea of personal computing was very far from main stream. Even just browsing some of the pages gives an impression of the impact the work had…
For future user interfaces we talked about brain computer interfaces (BCI) and how they very much differ from the idea of mind reading. I came across a game controller – Mindlink – developed by Atari (1984) and that was never released [2]. It was drawing on the notion of linking to the mind but in fact it only measured muscle activity above the eye brows and apparently did not perform very well. However there is a new round coming up for such devices, see [3] for a critical article on consumer BCI.
On the fun side I found a number of older videos that look at future technology predictions- see the videos for yourself:
http://www.paleofuture.com one is a site that has an amazing (and largely funny) selection of predictions. There is a more serious – but nevertheless – very entertaining article on predictions for computing and ICT by Friedemann Mattern: Hundert Jahre Zukunft – Visionen zum Computer- und Informationszeitalter (hundred years future – predictions of the computing and information age) [4].
[1] Sutherland’s Ph.D. Thesis, Sketchpad, A Man-Machine Graphical Communication System. 1963 http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf
[3] Emmet Cole. Direct Brain-to-Game Interface Worries Scientists. Wired. 09.05.07. http://www.wired.com/medtech/health/news/2007/09/bci_games
[4] Friedemann Mattern.Hundert Jahre Zukunft – Visionen zum Computer- und Informationszeitalter. Die Informatisierung des Alltags – Leben in smarten Umgebungen, Springer Verlag 2007. http://www.vs.inf.ethz.ch/publ/papers/mattern2007-zukunft.pdf

Ideas in Advertisment, Privacy, German Law

In our master course we offer a project on pervasive advertisement (it is an interdisciplinary course project from computer science and marketing) where we look at future forms of advertisement that become possible by new technologies.
The students presented a set of really exciting ideas – an I would expect (if they get some of their ideas implemented) that advertising will be more entertaining and fun in the future. For some of the ideas we discussed potential privacy issues and I promised to provide later the reference to the German privacy law that restricts the use of optical/camera devices in public spaces. The German law is at a first glance very restrictive with regard to using cameras in public spaces. In short it can be summarized that data can only obtained for a legitimate, concrete and defined purpose and that the privacy interest of the people are not higher to value as the purpose. Additionally it has to be clear to the person observed that he or she is observed. (We probably need a lawyer to figure out what is allowed 😉 In [1] the text of the law (in German) can be found.

Lucia and Thomas from Vodafone R&D visiting

Lucia Terrenghi and Thomas Lang from Vodafone R&D in Munich visited our lab. We talked at lot about the future role of mobile devices and in particular how they may change personal computing in the near future. 

After lunch they gave a talk for our students describing Vodafone research and their particular research interests. Using a nice visualization of train lines they showed the research themes and introduced some of their research foci. One area of interest is electronic paper, resulting future devices and potential applications and services. In the discussion I briefly mentioned that I had a look at some of the displays with my microscope – have a look in the previous blog post if you are interested.

Privacy – will our understanding change radically?

As one issue this morning we came across issues related to privacy. In particular it seems that social network analysis based on behavior in the real world (e.g. the reality mining project [1]) is creating serious interest beyond the technology people. Beyond measuring the frequency of encounters qualifying the way people interact (dominance, emotion, …) will reveal even more about social networks… 

In our discussion I made a reference to a book: „The Transparent Society“ by David Brin. Even Though it is now nearly 10 years since it was first published I still think it is an interesting starting point for a privacy discussion.

[1] Eagle, N. and (Sandy) Pentland, A. 2006. Reality mining: sensing complex social systems. Personal Ubiquitous Comput. 10, 4 (Mar. 2006), 255-268. DOI= http://dx.doi.org/10.1007/s00779-005-0046-3 

[2] The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom? David Brin, Basic Books (June 1, 1999). At Amazon

2 day faculty meeting in Essen

The last 2 day we had a meeting – with all faculty of the ICB – discussing the future challenges of work at universities in the context of our current situation. For me the this time was very well spent as I got to know many of my colleagues better and it was incredible to see the potential (from a scientific perspective as well as looking at the people) we have in our organization.

To me it is always amazing how much of a difference it makes to have a professional external moderator running such workshops (and we were very lucky with Klaus Schneider Ott from focus-team). Even though many of the psychological games are well known to many of us – they still work well and move discussion forward and help in creating common ground and even increase trust. He lectured us (after much of the discussion has happened) on basics of communication (e.g. transactional analysis) and again not novel they lead to serious reflection and progress.
PS: Do you know how to continue the row in the image below? What is the next sign? …
If you think of M-Omega-8 you are wrong – but still it is really easy 😉

Trip to Dublin, Aaron’s Display Project

Visiting Dublin is always a pleasure – even if the weather is rainy. Most of the day I was at Trinity College reading master theses (which is the second best part of being external examiner, best part is to have lunch at the 1592 😉
In the evening I met with Aaron Quigley and we talked about some ongoing display and advertsing projects in our groups. He told me about one of their recent workshop papers [1] on public displays where they investigated what people take in and what people remember of the content on displays in an academic environment. It is online available in the workshop proceedings of AIS08 [2]. I found it worthwhile to browse the whole workshop proceedings.
[1] Rashid U. and Quigley A., „Ambient Displays in Academic Settings: Avoiding their Underutilization“, Ambient Information Systems Workshop at UbiComp 2008, September 21, Seoul, South Korea (download [2], see page 26 ff)

Richard Atterer defended his PhD-thesis

At LMU in Munich Richard Atterer defended his PhD thesis on Usability Tool Support for Model-Based Web Development. The external examiner was Prof. Martin Gaedke– it was great meeting Martin again, we shared an office some years back at TecO, University of Karlsruhe. While I was with LMU I worked with Richard on a number of interesting topics and I learned to value his technical skills and insightful reflection – hope he stays in academia 😉 
Our experiments with remote usability assessment and effectively fine-grain user tracking on web pages [1] created a number of ideas for follow-up projects. The overall concept is really simple yet powerful: people use a proxy server (willingly or transparent) and by these means Javascript code is included in the html-source of arbitrary web pages to add new functionality [2]. Florian Alt extended this concept to an annotation system and currently we look into more general implication of this approach.
While writing his theses Richard countered our routine question „how is progress?“  with a dynamically generated graph on his webpage. Each time he checked his document in SVN the graph was updated with the current number of words and pages … but we still kept asking 😉 
 [1] Atterer, R. and Schmidt, A. 2007. Tracking the interaction of users with AJAX applications for usability testing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 ? May 03, 2007). CHI ’07. ACM, New York, NY, 1347?1350. DOI= http://doi.acm.org/10.1145/1240624.1240828
[2] Atterer, R., Wnuk, M., and Schmidt, A. 2006. Knowing the user’s every move: user activity tracking for website usability evaluation and implicit interaction. In Proceedings of the 15th international Conference on World Wide Web (Edinburgh, Scotland, May 23 ? 26, 2006). WWW ’06. ACM, New York, NY, 203?212. DOI= http://doi.acm.org/10.1145/1135777.1135811 

Mobile images / video as proof

While waiting for my conneting flight a saw a women posting a letter and filming this as she did it. She created some sort of proof. Thinking a little more and having further information in the background (e.g. a clock, the schedule display, people waiting) this this has some potential to replace registered mail for certain domains? This gives me an idea for a small weekend project…

Closing Panel at Ubicomp 2008

The closing panel at Ubicomp dicussed the last 10 years of ubicomp and potential future direction (with regard to community and technology). On the panel were Gregory Abowd, Hide Tokuda, Lars Eric Holmquist, Eric Paulos, and Albrecht Schmidt. In the following I will just describe some of the points I raised in my short statement. 
For me the first observation is that many ideas that were discussed at the first HUC99 (the first ubicomp conference started by Hans Gellsersen) – and were considered very speculative ideas have become common products and services by now (e.g. pocket bargain finder, predictive text input, mobile collaboration tool, mobile photo-sharing, location aware technologies). Here it is apparent that with regard to envisioning applications the conference has made impact. But there is a curious phenomenon: at the moment a device or service is available in the shop we do not recognize it as ubicomp anymore.
In some areas the complexity of the problems (when moving from the lab to the real world) has been underestimated – here context and context-awareness is a good example. If you realize the full vision it is basically solving AI. But nevertheless we progressed – there are applications for commercial mobile devices that do context and activity recognition – and it is just 10 years that we discussed this in a HUC99 in our paper on advanced interaction in context [1] – which was at that time really innovative! Context-awareness will happen – be patient 🙂 but its has to take into account: humans are adaptive, too. 
The papers which had a large impact (based on citations e.g. check [1] on google scholar) seem more the papers that score high on novelty, even if the may lack scientific rigor.
 
For future directions I hinted some general directions (not necessarily my research directions): 
  • Implanted activity recognition and interaction (put the sensing and actuation into the body solves a lot of the problems … obviously it creates many new ones, too) 
  • Implantable persuasion and amplifying bodily experiences. Here I gave the example that we would be able to create a device to motivate you do sports by making your back hurt. I used this to emphasise that ethics will play a large role in the future…
  • Prediction technologies (e.g. the weather forecast as an inspiration, forecasting traffic conditions, parking situation, restaurant business, costs, …) we will create systems that allo us to look up predictions (cost, quality of the experience, stress, time needed, etc.) for future activities (e.g. when choosing a restaurants, booking a travel, deciding on dating a person, making a business deal, accepting a position, …)
  • And finally I suggested that we will have fun with papers on privacy published now when reading them in 20 years 🙂 because our perception of this topic will change massively.
With regard to the community I made the statement that Ubicomp became the Starbucks of ubiquitous computing research – premium but based on the US idea of quality. Have you ever been in a Vienna coffee house, in an Italian espresso bar, or had tea in the middle east – it is very different. We lost some of the international spirit and we stopped arguing what good research in ubicomp is – this discurse should be started again!. Looking at the countries where ubicomp technologies come from (e.g. a lot from Aisia and Europe) we should again make a effort to more value the international diversity and the different styles and approaches in ubicomp research – scientific rigor is not the only axis to consider. 
[1] Schmidt, A., Aidoo, K. A., Takaluoma, A., Tuomela, U., Laerhoven, K. V., and Velde, W. V. 1999. Advanced Interaction in Context. In Proceedings of the 1st international Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany, September 27 – 29, 1999). H. Gellersen, Ed. Lecture Notes In Computer Science, vol. 1707. Springer-Verlag, London, 89-101. DOI= http://dx.doi.org/10.1007/3-540-48157-5_10