Linking the activities in the physical world to actions in the digital/virtual

Currently we have an assignment in our Pervasive Computing class that asks students to design and develop a system where actions are associated with artifacts. Technically students should develop a web based solution using RFID. Apropos RFID, … if you look for a good introduction on RFID read Roy Want’s IEEE Pervasive Magazin paper [1].

We use the hardware from http://nabaztag.com/ (Ztamp:s and Mir:ror) as the focus is on the concept and application and not on the underlying technology. To ease development Florian and Ali have developed a little system that offers WebCallBacks (students can register a URL and that is called when a tag is read).

Linking by tagging of objects has been well explored, e.g. [2] and [3], and I think it is about time that this technologies will make an impact in the consumer market – the technology gets cheap enough now (and perhaps one of our students has a great idea).

Some years back (in the last millennium) a company tried to push linking of paper adverts and digital content with the CueCat (http://en.wikipedia.org/wiki/CueCat) – I was impressed and inspired at that time but in my view it had two major weaknesses: (1) technically too early and (2) encoding of serial numbers instead of URLs. The RadioShack catalog and the Wired Magazine that included codes showed the potential – but it was too cumbersome as it was restricted to the PC …

We did some work on the topic, too around that time – at RFID reader integrated in a glove – which resulted in a Poster at ISWC [4] and a patent [5].

[1] Want, R. 2006. An Introduction to RFID Technology. IEEE Pervasive Computing 5, 1 (Jan. 2006), 25. DOI= http://dx.doi.org/10.1109/MPRV.2006.2

[2] Harrison, B. L., Fishkin, K. P., Gujar, A., Portnov, D., and Want, R. 1999. Bridging physical and virtual worlds with tagged documents, objects and locations. In CHI ’99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, May 15 – 20, 1999). CHI ’99. ACM, New York, NY, 29-30. DOI= http://doi.acm.org/10.1145/632716.632738

[3] Ljungstrand, P. and Holmquist, L. E. 1999. WebStickers: using physical objects as WWW bookmarks. In CHI ’99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, May 15 – 20, 1999). CHI ’99. ACM, New York, NY, 332-333. DOI= http://doi.acm.org/10.1145/632716.632916

[4] Schmidt, A., Gellersen, H., and Merz, C. 2000. Enabling Implicit Human Computer Interaction: A Wearable RFID-Tag Reader. In Proceedings of the 4th IEEE international Symposium on Wearable Computers (October 18 – 21, 2000). ISWC. IEEE Computer Society, Washington, DC, 193. (Poster as large PNG)

[5] US Patent 6614351 – Computerized system for automatically monitoring processing of objects. September 2, 2003. http://www.patentstorm.us/patents/6614351/description.html

Ubicomp Spring School in Nottingham – Tutorial

The ubicomp spring school in Nottingham had an interesting set of lectures and practical sessions, including a talk by Turing Award winner Robin Milner on a theoretical approach to ubicomp. When I arrived on Tuesday I had the chance to see Chris Baber’s tutorial on wearable computing. He provided really good examples of wearable computing and its distinct qualities (also in relation to wearable use of mobile phones). One example that captures a lot about wearable computing is an adaptive bra. The bra one example of a class of interesting future garments. The basic idea is that these garments detects the activity and changes their properties accordingly. A different example in this class is a shirt/jacket/pullover/trouser that can change its insulation properties (e.g. by storing and releasing air) according to the external temperature and the users body temperature.

My tutorial was on user interface engineering and I discussed: what is different in creating ubicomp UIs compared to traditional user interfaces. I showed some trends (including technologies as well as a new view on privacy) that open the design space for new user interfaces. Furthermore we discussed the idea about creating magical experiences in the world and the dilemma of user creativity and user needs.

There were about 100 people the spring school from around the UK – it is really exciting how much research in ubicomp (and somehow in the tradition of equator) is going on in the UK.

Teaching, Technical Training Day at the EPO

Together with Rene Mayrhofer and Alexander De Luca I organized a technical training at the European Patent Office in Munich. In the lectures we made the attempt to give a broad overview of recent advanced in this domain – and preparing such a day one realizes how much there is to it…. We covered the following topic:
  • Merging the physical and digital (e.g. sentient computing and dual reality [1])
  • Interlinking the real world and the virtual world (e.g. Internet of things)
  • Interacting with your body (e.g. implants for interaction, brain computer interaction, eye gaze interaction)
  • Interaction beyond the desktop, in particular sensor based UIs, touch interaction, haptics, and Interactive surfaces
  • Device authentication with focus on spontaneity and ubicomp environments
  • User authentication focus on authentication in the public 
  • Location-Awareness and Location Privacy
Overall we covered probably more than 100 references – here are just a few nice ones to read: computing tiles as basic building blocks for smart environments [2], a bendable computer interface [3], a touch screen you can also touch on the back side [4], and ideas on phones as basis for people centric censing [5].
[1] Lifton, J., Feldmeier, M., Ono, Y., Lewis, C., and Paradiso, J. A. 2007. A platform for ubiquitous sensor deployment in occupational and domestic environments In Proceedings of the 6th Conference on international information Processing in Sensor Networks (Cambridge, Massachusetts, USA, April 25 – 27, 2007). IPSN ’07. ACM, New York, NY, 119-127. DOI= http://doi.acm.org/10.1145/1236360.1236377
[2] Naohiko Kohtake, et al. u-Texture: Self-organizable Universal Panels for Creating Smart Surroundings. The 7th Int. Conference on Ubiquitous Computing (UbiComp2005), pp.19-38, Tokyo, September, 2005. http://www.ht.sfc.keio.ac.jp/u-texture/paper.html
[3] Schwesig, C., Poupyrev, I., and Mori, E. 2004. Gummi: a bendable computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria, April 24 – 29, 2004). CHI ’04. ACM, New York, NY, 263-270. DOI= http://doi.acm.org/10.1145/985692.985726 
[4] Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., and Shen, C. 2007. Lucid touch: a seethrough mobile device. InProceedings of the 20th Annual ACM Symposium on User interface Software and Technology (Newport, Rhode Island, USA, October 07 – 10, 2007). UIST ’07. ACM, New York, NY, 269-278. DOI= http://doi.acm.org/10.1145/1294211.1294259 
[5] Campbell, A. T., Eisenman, S. B., Lane, N. D., Miluzzo, E., Peterson, R. A., Lu, H., Zheng, X., Musolesi, M., Fodor, K., and Ahn, G. 2008. The Rise of People-Centric Sensing. IEEE Internet Computing 12, 4 (Jul. 2008), 12-21. DOI= http://dx.doi.org/10.1109/MIC.2008.90  

Final Presentation: Advertising 2.0

Last term we ran an interdisciplinary project with our MSc students from computer science and business studies to explore new ways in outdoor advertising. The course was jointly organized by the chairs: Specification of Software Systems, Pervasive Computing and User Interface Engineering, and Marketing and Trade. We were in particular interested what you can do with mobile phones and public displays. It is always surprising how much a group of 10 motivated students can create in 3 months. The group we had this term was extraordinary – over the last weeks they regularly stayed in the evenings longer in the lab than me 😉

The overall task was very open and the students created a concept and than implemented it – as a complete system including backend server, end user client on the mobile phone, and administration interface for advertisers. After the presentation and demos we really started thinking where we can deploy it and who the potential partners would be. The system offers means for implicit and explicit interaction, creates interest profiles, and allows to target adverts to groups with specific interest. Overall such technologies can make advertising more effective for companies (more precisely targeted adverts) and more pleasant for consumers (getting adverts that match personal areas of interest).

There are more photos of the presentation on the server.

PS: one small finding on the side – Bluetooth in its current form is a pain for interaction with public display… but luckily there are other options.

Design Ideas and Demos at FH Potsdam

During the workshop last week in Potsdam we got to see demos from students of Design of Physical and Virtual Interfaces class taught by Reto Wettach and JennyLC Chowdhury. The students had to design a working prototype of an interactive system. As base technology most of them use the Arduino Board with some custom made extensions. For a set of pictures see my photo gallery and the photos on flickr. It would need pages to describe all of the projects so I picked few…

The project “Navel” (by Juan Avellanosa, Florian Schulz and Michael Härtel) is a belt with tactile output, similar to [1], [2] and [3]. The first idea along this lines that I have tried out was Gentle Guide [4] at mobile HCI 2003 – it seemed quite compelling. The student project proposed one novel application idea: to use it in sport. That is quite interesting and could complement ideas proposed in [5].

Vivien’s favorite was the vibrating doormat; a system where a foot mat is constructed of three vibrating tiles that can be controlled and different vibration patters can be presented. It was built by Lionel Michel and he has several ideas what research questions this could address. I found especially the question if and how one can induce feelings and emotions with such a system. In the same application context (doormat) another prototype looked at emotions, too. If you stroke or pat this mat it comes out of its hiding place (Roll-o-mat by Bastian Schulz).

There were several projects on giving everyday objects more personality (e.g. a Talking Trashbin by Gerd-Hinnerk Winck) and making them emotional reactive (e.g. lights that reacted to proximity). Firefly (by Marc Tiedemann) is one example how reactiveness and motion that is hard to predict can lead to an interesting user experience. The movement appears really similar to a real firefly.

Embedding Information has been an important topic in our research over the last years [6] – the demos provided several interesting examples: a cable that visualized energy consumption and keyboard to leave messages. I learned a further example of an idea/patent application where information is included in the object – in this case in a tea bag. This is an extreme case but I think looking into the future (and assuming that we get sustainable and bio-degradable electronics) it indicates an interesting direction and pushing the idea of Information at your fingertip (Bill Gates Keynote in 1994) much further than originally intended.

For more photos see my photo gallery and the photos on flickr.

[1] Tsukada, K. and Yasumrua, M.: ActiveBelt: Belt-type Wearable Tactile Display for Directional Navigation, Proceedings of UbiComp2004, Springer LNCS3205, pp.384-399 (2004).

[2] Alois Ferscha et al. Vibro-Tactile Space-Awareness . Video Paper, adjunct proceedings of Ubicomp2008. Paper. Video.

[3] Heuten, W., Henze, N., Boll, S., and Pielot, M. 2008. Tactile wayfinder: a non-visual support system for wayfinding. In Proceedings of the 5th Nordic Conference on Human-Computer interaction: Building Bridges (Lund, Sweden, October 20 – 22, 2008). NordiCHI ’08, vol. 358. ACM, New York, NY, 172-181. DOI= http://doi.acm.org/10.1145/1463160.1463179

[4] S.Bosman, B.Groenendaal, J.W.Findlater, T.Visser, M.de Graaf & P.Markopoulos . GentleGuide: An exploration of haptic output for indoors pedestrian guidance . Mobile HCI 2003.

[5] Mitchell Page, Andrew Vande Moere: Evaluating a Wearable Display Jersey for Augmenting Team Sports Awareness. Pervasive 2007. 91-108

[6] Albrecht Schmidt, Matthias Kranz, Paul Holleis. Embedded Information. UbiComp 2004, Workshop ‚Ubiquitous Display Environments‘, September 2004

What happens if Design meets Pervasive Computing?

This morning I met with Claudius Lazzeroni, a colleague from Folkwang Hochschule (they were part of our University till two years ago).
 
They have different study programs in design and art related subjects. He showed me some projects (http://www.shapingthings.net/ – in German but lots of pictures that give you the idea). Many of the ideas and prototypes related to our work and I hope we get some joint projects going. I think it could be really exciting to have projects with design and computer science students – looking forward to this!
When I was in the UK we collaborated in the equator project with designers – mainly Bill Gaver and his group – and the results were really exciting [1]. We build a table that reacted to load changes on the surfaces and allowed you to fly virtually over the UK. The paper is worthwhile to read – if you are in a hurry have a look at the movie about it on youtube: http://www.youtube.com/watch?v=uRKOypmDDBM
There was a further project with a table –  a key table – and for this one there more funny (and less serious?) video on youtube: http://www.youtube.com/watch?v=y6e_R5q-Uf4
[1] Gaver, W. W., Bowers, J., Boucher, A., Gellerson, H., Pennington, S., Schmidt, A., Steed, A., Villars, N., and Walker, B. 2004. The drift table: designing for ludic engagement. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 – 29, 2004). CHI ’04. ACM, New York, NY, 885-900. DOI= http://doi.acm.org/10.1145/985921.985947

Mechanical Computing, Beauty of Calculating Machines

Instead of covering the history of calculating machines in the DSD lecture, we took the train and went to the Arithmeum in Bonn to the see the artefacts live and to play with some of them.
We started with early means for counting and record keeping. The tokens and early writings did not use numbers as abstract concepts, rather as representatives of concrete objects – this is very inspiring, especially from a tangible interaction point of view. The knots, as used in south America, show impressively how the tools for calculation have to fit the context people live in. Interestingly all these artefacts highliht how the ability to calculate and store information is related to the ability to do trade – quite a good motivation for the setup we have in Essen business studies and computer science within one faculty.
I was again impressed by the ingenuity by the early inventers of calculating tools and machines. There is an interesting separation between calculating tools and machines – the first ones require the user to take care of the carry and the second do it by themselves. We tried out replicas of Napier’s calculating tool and Schickard’s calculating machine.
The beauty and the mechanical precision required of those early machines is impressive. These prototypes (most of them took years and massive funds to be built complete) can teach us something for research today. These inventors had visions and the will to get it implemented, even without a clear application or business model in mind. They were excited by the creating of systems than can do things, machines could not do before. From the professions of the inventors (e.g. Philipp Matthäus Hahn was a clergyman)  it becomes apparent that at these times some considered religion and calculation as closely related – which to mondern understanding is very very alien.

Seeing the Hollerith machine that was used for the US census more than 100 years ago can teach you a lot about data processing. Punch cards, electrical reading and electrical counters (using mainly relays) were the basis for this technology. Looking at the labels on the counters showed that the US has a long tradition in collecting data that is after some time is not seen as political correct 😉
Having learned binary calculations during the DSD course it was nice to see a machine that did binary additions, using small steel balls and gravity. On each place (1,2,4,8, …) there is space for one ball. If a second one comes to this place one moves up to the next place (carry) and one is discarded. This is implemented with very simple mechanics and the working prototype (recently build) is based on designs of Schickard (but he never built – if I am correct).
Moving on with binary systems and finally to silicon, we got to see the Busicom 141 – a desk calculator that uses the Intel 4004. It is impressive to see that this is not even 40 years ago – starting with 2300 transistors and 180kHz. 
you can find the full set of photos at: http://foto.ubisys.org/dsd0809/

One Ez430-F2013 for each student in DSD

This term we teach digital system design and besides the essential (gates, flip-flops, 2-complements, alu, data path, etc) we decided to include some practical parts. In the first part we introduced Verilog. We did the exercises with Icarus Verilog (free, text-based verilog) and there are more powerful tools available, e.g. ISE WebPACK.

In the second part we have practical exercises in assembly language using the Ez430-F2013 development kit. We borrowed one kit to each student (just in case someone is bored over Christmas) and the current task is to complete and assembly program (incomplete sample) that the LED shows repeatedly „Hello“ in Morse code. This is just the start – perhaps we do some more interesting stuff in January. The development kit is really interesting – especially given the fact that it is only 20€ (including hardware, compiler, and IDE). The MCU is small – but still good enought to generate a video signal (eagle CAD files, some assembly code). 

Exporting your cars information to the mobile phone

In our user interface engineering class one of the tasks in the exercise is to create a concept design for providing information from the car on the mobile phone (e.g. millage, amount a fuel in the car, next service date, alram status, etc). The first part is to assess what information could be made accessible and what value it would create for the user. 
Today I came across a device (Tyredog TD-1000A) that is concerned with a one sub-part of this scenario: checking your pressure in the tires of the car. It is a simple sensor system, screwed on to each of the tires, connected to a wireless receiver. There is also a version that includes features for the car alarm (Tyredog TD-3000A).
Another group is looking yet again into the domain of  restaurant finders or more general night life. Apropos restaurant finders, Saturday night we got out of the subway onto union square and discussed where to go for dinner (an we probably looked disoriented). A local lady stoped and recommended the Union Square Café – and it was just great… sometimes just talking to someone in the street may provide you with an excellent alternative to technologies 😉 Perhaps the students find a solution that can reflect personal recommendations well…

History and Future of Computing and Interaction

Today I was teaching my class on user interface engineering and we covered a selected history of HCI and looked at the same time at a potential future. We discussed how user interface evolved and where UI revolutions have happed. To my question „What is the ultimate user interface?“ I got three very interesting answers (1) a keyboard, (2) mind reading, and (3) a system that anticipates what I want. 
With regard to history in HCI one of my favorite texts is the PhD dissertation of Ivan Sutherland [1]. The work described was done in 1960-1963 when the idea of personal computing was very far from main stream. Even just browsing some of the pages gives an impression of the impact the work had…
For future user interfaces we talked about brain computer interfaces (BCI) and how they very much differ from the idea of mind reading. I came across a game controller – Mindlink – developed by Atari (1984) and that was never released [2]. It was drawing on the notion of linking to the mind but in fact it only measured muscle activity above the eye brows and apparently did not perform very well. However there is a new round coming up for such devices, see [3] for a critical article on consumer BCI.
On the fun side I found a number of older videos that look at future technology predictions- see the videos for yourself:
http://www.paleofuture.com one is a site that has an amazing (and largely funny) selection of predictions. There is a more serious – but nevertheless – very entertaining article on predictions for computing and ICT by Friedemann Mattern: Hundert Jahre Zukunft – Visionen zum Computer- und Informationszeitalter (hundred years future – predictions of the computing and information age) [4].
[1] Sutherland’s Ph.D. Thesis, Sketchpad, A Man-Machine Graphical Communication System. 1963 http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf
[3] Emmet Cole. Direct Brain-to-Game Interface Worries Scientists. Wired. 09.05.07. http://www.wired.com/medtech/health/news/2007/09/bci_games
[4] Friedemann Mattern.Hundert Jahre Zukunft – Visionen zum Computer- und Informationszeitalter. Die Informatisierung des Alltags – Leben in smarten Umgebungen, Springer Verlag 2007. http://www.vs.inf.ethz.ch/publ/papers/mattern2007-zukunft.pdf