"Smart Things," my book on ubiquitous computing user experience design is approaching completion. The final draft is done, we're working on the interior design and cover and it's available for pre-order from Amazon now. If everything goes well, it will ship in August.

The outline of the book remains largely the same as it was last year. Starting in April I plan to serialize one chapter draft per month in blog-sized portions until the book is released.

In preparation for it, I decided to emulate Adam's release of the bibliography for "The City is Here for You to Use," his upcoming book, and post the references section from mine.

The following is not a list of the best, most influential, or all the work in the field of ubicomp UX. It's just the works that I reference in my book, which I chose because of a combination of historical importance, the ideas, the availability of the material to me, and my personal interest. I am posting it here so that the citations in the chapters I post later are easier to follow up on.

Adams, L., Toomey, L., and Churchill, E.F.. "Distributed Research Teams: Meeting Asynchronously in Virtual Space." Journal of Computer Mediated Communication, JCMC 4 (4) June 1999.

Addington, M. D., Schodek, D.L., Smart materials and new technologies. Architectural Press, 2005.

Adobe, "Fun with Friends," "On the go", "Everyday life," Adobe Open Screen project concept videos, 2009 (accessed November 30, 2009)

Agar, M., The Professional Stranger: An Informal Introduction to Ethnography. 2d ed. New York: Academic Press. 1996.

AHAM, Average useful life of major home appliances, report, Association of Home Appliance Manufacturers, 1996

Akrich, M., M. Callon, and Latour, B., 2002. "The key to success in innovation part I: the art of interessement." International Journal of Innovation Management 6: 187-206.  

Albrecht, K., and L. McIntyre. Spychips: how major corporations and government plan to track your every move with RFID. Thomas Nelson Inc., 2005

Alexander, C., The Oregon Experiment. Oxford University Press, USA, 1975

Amazon, Amazon Mechanical Turk Best Practices Guide, Amazon Web Services LLC, 2009, http://mturkpublic.s3.amazonaws.com/docs/MTURK_BP.pdf (Downloaded August 8, 2009).

AMD. "Videsh Sanchar Nigam Limited And AMD Launch The Personal Internet Communicator," press release, October 28, 2004.

Anderson, C. "In the Next Industrial Revolution, Atoms Are the New Bits." Wired Magazine, February 2010

Anderson, K., and J. McGonigal. 2004. "Place storming: performing new technologies in context." In Proceedings of the third Nordic conference on Human-computer interaction, 85-88. ACM New York, NY, USA.

Apple Inc., "Apple Reports First Quarter Results," Apple press release, January 22, 2008

Ariely, D., Predictably Irrational: The Hidden Forces That Shape Our Decisions. 1st ed. HarperCollins, February 19. 2008

Arnall, T., "A graphic language for touch-based interactions." Proc. Mobile Interaction with the Real World, 2006

Autodesk, "Digital Prototyping: Questions and Answers," http://images.autodesk.com/adsk/files/dp10_faq_us_finalr1.pdf (Downloaded August 24, 2009)

Azuma, R. T. 1997. "A survey of augmented reality." Presence-Teleoperators and Virtual Environments 6, no. 4: 355-385.

Bachelard, G. 1957. La poétique de l’espace. PUF, Paris.

Back M., Cohen J., Gold R., Harrison S., Minneman S.,"Listen reader: an electronically augmented paper-based book." Proceedings of the CHI 2001 conference on human factors in computing systems, Seattle, Washington, April 2001, pp 23–29, 2001

Barr, P., R. Biddle, and J. Noble. 2002, A Taxonomy of User-Interface Metaphors. In Proceedings of SIGCHI-NZ Symposium On Computer-Human Interaction (CHINZ 2002).

Barrie, J. M., Peter and Wendy. Hodder & Stoughton, 1911

Barry, B. A., "Story Beads: and mobile storytelling." Masters Thesis, Massachusetts Institute of Technology, 2000

Barry, B. A., Davenport, G. & McGuire, D. "StoryBeads: a wearable for construction and trade." Proceedings of the First IEEE International Workshop on Networked Appliances. New Brunswick, N.J. November 2000.

Baudrillard, J., Simulacra and Simulation (translated by Sheila Faria Glaser). Ann Arbor: University of Michigan. 1994. Originally published in French by Editions Galilee, 1981.

Baxter, A., Marzano, S., and Rotte, A.. Vision of the Future. Philips Design Blaricum, The Netherlands: V+ K Publishing, Eindhoven, The Netherlands, 1998

Beard, D., M. Palaniappan, A. Humm, D. Banks, A. Nair, and Y. P. Shan. 1990. "A visual calendar for scheduling group meetings." In Proceedings of the 1990 ACM conference on Computer-supported cooperative work, 279-290. ACM New York, NY, USA.

Beck, K., Extreme Programming Explained, Addison-Wesley, 2004

Beigl, M., and H. W. Gellersen. 1999. "Ambient telepresence." In Proceedings of the Workshop on Changing Places, 32:92–99. Vol. 32., 1999

Beigl, M., Gellersen, H.W. and Schmidt, A. "Mediacups: experience with design and use of computer- augmented everyday objects." Computer Networks, 35(4): 401–409, 2001

Bell, G., Blythe, M. and Sengers, P. (2005) "Making by Making Strange: Defamiliarization and the Design of Domestic Technology." Transactions on Computer- Human Interaction (TOCHI), Special issue on Social Issues and HCI, vol. 12, no. 2, pp. 149-173.

Bell, G., Brooke, T., Churchill, E., and Paulos, E., Intimate Ubiquitous Computing Proceedings of UbiComp 2003, 2003

Benjamin, W., "The Work of Art in the Age of Mechanical Reproduction" (1936), Illuminations. Trans. Harry Zohn. New York: Schocken: 217–51. 1968

Bergman, E. (editor) Information appliances and beyond: interaction design for consumer products. Morgan Kaufmann. 2000.

Best, J., Flavor of the Month: Why smart people fall for fads, University of California Press, 2006

Beyer, H., & Holtzblatt, K. Contextual design—Defining customer centered systems. San Francisco: Morgan Kaufmann. 1998

Bitner, M. J., A. L. Ostrom, and F. N. Morgan. 2008. Service Blueprinting: A Practical Technique for Service Innovation. CALIFORNIA MANAGEMENT REVIEW 50, no. 3: 66., 2008

Blackwell, A. F. The reification of metaphor as a design tool. ACM Transactions on Computer-Human Interaction (TOCHI) 13, no. 4: 490-530. 2006

Bleecker, J. 2005. "A manifesto for networked objects - cohabiting with pigeons, arphids and aibos in the internet of things." Annenberg Center for Communication. http://research.techkwondo.com/files/WhyThingsMatter.pdf, 2005

Blomberg, J., and S. Evenson. 2006. "Service innovation and design." In Conference on Human Factors in Computing Systems, 28-31. ACM New York, NY, USA., 2006

Bluetooth, Special Interest Group. Specification of the Bluetooth system. Core, version 1, no. 2. 2004

Bly, S. A., S. R. Harrison, and S. Irwin. 1993. "Media spaces: bringing people together in a video, audio, and computing environment." Communications of the ACM 36, no. 1: 28-47., 1993

Bolter, J.D., Gromala, D., "Transparency and Reflectivity: Digital Art and the Aesthetics of Interface Design" in Aesthetic Computing, Fishwick, P. (ed.), MIT Press, 2006

Bonanni, L., Lee, C.H., Selker, T., "CounterIntelligence: Augmented Reality Kitchen," in Proceedings of CHI '05 pp. 2239-45, 2005

Borchers, J. A Pattern Approach to Interaction Design. John Wiley & Sons Ltd, March 2001.

Borchers, J., "The Aachen Media Space: Design Patterns for Augmented Work Environments" in Design user friendly augmented work environments (Lahlou, S. ed) Springer. 2009

Brand, S., How Buildings Learn: What Happens After They're Built. Viking Penguin, 1995

Brandt E., Messeter, J., "Facilitating Collaboration through Design Games," Proceedings of the Participatory Design Conference, Toronto, Canada, 2004

Bravo, C., Redondo, M, Ortega , M. and Verdejo M.F., "Collaborative environments for the learning of design: a model and a case study in Domotics," Computers & Education 46, 2006

Brignull, H., and Y. Rogers. 2003. "Enticing people to interact with large public displays in public spaces." In Human-computer Interaction: INTERACT'03; IFIP TC13 International Conference on Human-Computer Interaction, 1st-5th September 2003, Zurich, Switzerland, 17. Ios Pr Inc.

British Broadcasting Corporation (BBC). Standard Media Exchange Framework (SMEF) Data Model 1.5. http://www.bbc.co.uk/guidelines/smef, 2000

Brock, D. L. "The electronic product code (EPC)-a naming scheme for physical objects." Auto-ID Center White Paper WH-002. Cambridge, USA, Massachusetts Institute of Technology. 2001-1

Brock, D. L., "The Physical Markup Language - A Universal Language for Physical Objects," Auto-ID Center White Paper WH-003. Cambridge, USA, Massachusetts Institute of Technology. 2001-2

Buchenau, M., and J. F. Suri. 2000. "Experience prototyping." In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, 424-433. ACM New York, NY, USA.

Burns, Judith. 2009. "Smart meters 'need live displays'." BBC, October 5, sec. Science & Environment. http://news.bbc.co.uk/2/hi/science/nature/8287130.stm.

Buxton, W., 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann Pub.

Buxton, W., and Moran, T. "EuroPARC's integrated interactive intermedia facility (IIIF): early experiences." In Proceedings of the IFIP WG 8.4 confernece on Multi-user interfaces and applications, 34. Elsevier North-Holland, Inc., 1990

Cairncross, F., The Death of Distance: How the Communications Revolution Is Changing Our Lives. Harvard Business School Press, 2001.

Card, S.K., "Foreword", Information Visualization: Perception for Design, Second Edition, (Ware, C. editor) Morgan Kaufmann, 2004

Card, S.K., Moran, T.P., and Newell, A. The psychology of human-computer interaction. Routledge. 1983

Chaffers, William, and Christopher Alexander Markham. Hall Marks on Gold and Silver Plate. 9th ed. London: Reeves and Turner, 1905

Chalmers, M. "Seamful Designs and Ubicomp Infrastructure" Proceedings of Ubicomp 2003 Workshop At the Crossroads: The Interaction of HCI and Systems Issues in UbiComp

Chalmers, M., & Galani, M. (2004). "Seamful interweaving: Heterogenity in the theory and design of interactive systems." In Proceedings of the ACM Designing Interactive Systems DIS’04.

Chang, A., B. Resner, B. Koerner, and H. Ishii. "LumiTouch: an emotional communication device." In Conference on Human Factors in Computing Systems, 313-314. ACM New York, NY, USA, 2001

Chang, Y.L. and R.A. Grot, Field performance of residential refrigerators and combined refrigerator-freezers, NBSIR 79-1781, 1979, quoted in Kay, J.Y. and Kelly, G. E. "Factors Affecting the Energy Consumption of Two Refrigerator-Freezers," SA-96-7-1;ASHRAE Transactions, Vol. 102, No. 2, 1-11, 1996.

Churchill E.F., Bly, S. "It's all in the words: Supporting work activities with lightweight tools." In Proceedings of GROUP '99 (Phoenix, AZ), ACM Press, 1999.

Churchill, E.F. and Nelson, L.D. From Media Spaces to Emplaced Media: Digital Poster Boards and Community Connectedness. CSCW 2006 Workshop on Media Spaces, Banff, November 2006.

Churchill, E.F., Nelson, L, Hsieh, G. "Cafe life in the digital age: augmenting information flow in a cafe work-entertainment space" Experience reports / Proceedings of ACM CHI 2006 Conference on Human Factors in Computing Systems 2006 v.2 p.123-128

Churchill, E.F., Nelson, L., Denoue, L., Murphy, P. and Helfman, J.I. "The Plasma Poster Network: Social Hypermedia on Public Display. In Public and Situated Displays." Social and Interactional Aspects of Shared Display Technologies. K. O’Hara, M.Perry, E. Churchill and D. Russell (Eds) London: Kluwer Academic Publishers, December 31, 2003

Claypool, M., and K. Claypool. 2006. "Latency and player actions in online games." Communications of the ACM 49, no. 11: 45.  

CNN.com, "Japan’s Lonely Hearts Find Each Other with 'lovegety'." CNN.com, 7 June 1998, http://www.cnn.com/WORLD/asiapcf/9806/07/fringe/japan.lovegety/

Coates, T., "The Age of Point-at-Things," http://www.plasticbag.org/archives/2005/04/the_age_of_pointatthings/, retrieved November 19, 2008

Coelho, M., S. Sadi, P. Maes, J. Berzowska, and N. Oxman. 2007. "Transitive Materials: Towards an Integrated Approach to Material Technology." In the International Conference on Ubiquitous Computing (Ubicomp 2007).

Cohn, M., User Stories Applied for Agile Software Development. Boston: Addison-Wesley, 2004

Coiera, E., "Interaction design theory." International journal of medical informatics 69, no. 2-3: 205-222, 2003

Colbert, M., "Problems Rendezvousing: A Diary Study," in Handbook of Research on User Interface Design and Evaluation for Mobile Technology (Lumsden, J. ed), Information Science Reference, 2008

Consolvo S., and Walker, M., "Using the Experience Sample Method to Evaluate Ubicomp Applications." IEEE Pervasive Computing Mobile and Ubiquitous Systems: The Human Experience, 2(2):24–31, Apr-June 2003.

Cooper, A. The Inmates are Running the Asylum. Sams Press. 1999

Cottam, M., "Wooden logic: In search of heirloom electronics," Master's Thesis, Umeå Institute of Design, Umeå University, 2009

Cowan, R.S. (1983). More Work for Mother: The Ironies of Household Technology from the Open Hearth to the Microwave. New York, NY: Basic Books, Inc.

Crabtree, A. and Rodden, T.: "Domestic Routines and Design for the Home." JCSCW 13, 2004

Crowley, D., Rainert, A., Dodgeball, http://www.dodgeball.com, 2004, accessed September 8, 2006

Csikszentmihalyi, M. and Larson, R.,"Validity and reliability of the Experience Sampling Method," Journal of Nervous and Mental Disease, 175, 526-36, 1987

Cummings, M., Bruni, S., "Collaborative Human–Automation Decision Making" in Nof (ed), 2009

Cute Circuit. Hug Shirt. 2004.

Dahlbäck, N., Jönsson, A. and Ahrenberg, L. "Wizard of Oz studies—Why and how." In Intelligent User Interfaces ’93 (pp. 193-200), 1993

Davis, F. D. "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology." MIS Quarterly, 13(3), 319 ‐340. 1989

Davis, M, and Quincy T. Miles, the Autobiography. Simon and Schuster. 1990.

de Bono, E. 1999. Six Thinking Hats. 2nd ed. Back Bay Books, August 18.

DeLanda, M., 1991, War in the Age of Intelligent Machines, Swerve Editions, Zone Press, New York, pp. 12–14.

Diakonoff, I.M., The Paths of History, Cambridge University Press, 1999

Dodge, M., and R. Kitchin. 2007. Outlines of a world coming into existence': pervasive computing and the ethics of forgetting. Environment and Planning B: Planning and Design 34, no. 3: 431.  

Dourish, P. "What we talk about when we talk about context." Personal and Ubiquitous Computing 8, no. 1:19-30, 2004

Dourish, P. 2001. Where the action is: the foundations of embodied interaction. MIT Press, 2001

Dourish, P., "Implications for Design", CHI 2006 Proceedings, April 22-27, 2006

Dourish, P., and S. Bly. "Portholes: supporting awareness in a distributed work group." In Conference on Human Factors in Computing Systems: Proceedings of the SIGCHI conference on Human factors in computing systems. Association for Computing Machinery, Inc, One Astor Plaza, 1515 Broadway, New York, NY, 10036-5701, USA., 1992

Droms, R., "Dynamic Host Configuration Protocol", RFC 2131, March 1997.

Druin, A. (editor), Mobile Technology for Children: Designing for Interaction and Learning, Morgan Kaufmann, 2009

Dubberly, H., "The Making of Knowledge Navigator," March 30, 2007 blog post: http://www.dubberly.com/articles/the-making-of-knowledge-navigator.html, accessed August 30, 2009

Dunne, A., and F. Raby. Design noir: The secret life of electronic objects. Birkhäuser. 2001.

Eagle, N., and Pentland, A. (2005), "Reality Mining: Sensing Complex Social Systems", Personal and Ubiquitous Computing 10, 255-268, September 2005

Eames C., and Eames, R., Powers of Ten, Eames Office, 9 min, 1968

Economist, "Mobile marvels," September 24, 2009

Edwards W. K., Grinter R. E., "At home with ubiquitous computing: seven challenges" Proceedings of the Conference on Ubiquitous Computing (Ubicomp 2001) Atlanta, GA., 2001

Encarnação, J. L., and T. Kirste. 2005. "Ambient intelligence: Towards smart appliance ensembles." From Integrated Publication and Information Systems to Virtual Information and Knowledge Environments: 261–270.  

Eno, B., and P. Schmidt. Oblique Strategies: Over 100 worthwhile dilemmas by Brian Eno and Peter Schmidt. London: Apollo, 1975

Erixon, F., "Globalization, earnings and consumer prices: taking stock of the benefits from global economic integration," European Centre for International Political Economy, Policy Brief No. 05/2008, 2008

Eustice, K. F., Lehman, T. J., Morales, A., Munson, M. C., Edlund, S. & Guillen, M., "A universal information appliance," IBM Systems Journal, VOL 38, NO 4, 1999

Fass, A., Forlizzi, J., and R. Pausch. "Messydesk and messyboard: two designs inspired by the goal of improving human memory." In DIS ’02: Proceedings of the conference on Designing interactive systems, pages 303–311, New York, NY, USA, 2002. ACM Press.

Ferscha, A., M. Hechinger, R. Mayrhofer, M. dos Santos Rocha, M. Franz, and R. Oberhauser. 2004. "Digital aura." Advances in Pervasive Computing 176.

Fishwick, P., ed., Aesthetic Computing, MIT Press, 2006

Florins, M. "Graceful Degradation: a Method for Designing Multiplatform Graphical User Interfaces." PhD Dissertation, Université catholique de Louvain, Faculté des sciences economiques, sociales et politiques. 2006

Florins, M., and J. Vanderdonckt. 2004. Graceful degradation of user interfaces as a design method for multiplatform systems. In Proceedings of the 9th international conference on Intelligent user interfaces, 140-147. ACM New York, NY, USA.

Fogarty, J., Hudson, S.E, Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C., and Yang, J., "Predicting human interruptibility with sensors", ACM Transactions on Computer-Human Interaction (TOCHI)  12(1), 119–146, March 2005.

Footen, J. and Faust, J., The Service-Oriented Media Enterprise, Elsevier, 2008

Foth, M. Handbook of Research on Urban Informatics: The Practice and Promise of the Real-Time City. Information Science Reference, 2008

Freeman, D. 2004. Creating emotion in games: the craft and art of Emotioneering. Computers in Entertainment (CIE) 2, no. 3:15-15.

Fried, L. and Torrone, P., "Tweet-a-watt," http://www.ladyada.net/make/tweetawatt/

Friedman, B., Kahn, P.H. and Borning, A., "Value Sensitive Design and Information Systems." In P. Zhang & D. Galletta (eds.), Human-Computer Interaction and Management Information Systems: Foundations, New York, 2006, 348-372.

Fujimuri, N., Remote Furniture, http://www.andrew.cmu.edu/user/noriyuki/artworks/remotefurniture/index.html, accessed November 2, 2006, 1999

Gajos, K. Z., M. Czerwinski, D. S. Tan, and D. S. Weld. "Exploring the design space for adaptive graphical user interfaces." In Proceedings of the working conference on advanced visual interfaces, 201-208. ACM New York, NY, USA. 2006.

Galloway, A., A Brief History of the Future of Urban Computing and Locative Media, PhD Dissertation, Carleton University, 2008

Galloway, K., and Rabinowitz, S., Hole in Space. Mobile image videotape. 1980

Ganz, "Webkinz Provides Safe Online ‘Chatting’ in A World Where Kids Are Still Kids!" official press release, http:// www.ganz.com/corporate/media/press_release/WebkinzInternetSafety.pdf, 2006

Garfinkel, S., and Abelson, H. 1999. Architects of the information society. MIT Press.  

Garrett, J.J., "Elements of User Experience" (diagram), 2000 http://www.jjg.net/elements/ (Downloaded October 10, 2005)

Garrett, J.J., Elements of User Experience, New Riders Press, 2002

Gaver, W. "Provocative Awareness." CSCW, 11, 475-493, 2002

Gaver, W.W., Dunne, A., & Pacenti, E., "Cultural probes." Interactions, 6(1), 21–29., 1999

Gellersen, H., Beigl, M. & Krull, H. "The MediaCup: Awareness Technology embedded in an Everyday Object." In H.-W. Gellersen (Ed.), Proceedings of the Handheld and ubiquitous computing first international symposium, (HUC’99) (pp. 308-310). Heidelberg: Springer., 1999

Ghazala, R. Circuit-bending: build your own alien instruments. Wiley. 2005

Gilder, G., "Metcalfe's law and legacy". Forbes ASAP, 152: Supplement, 158-166, September 13, 1993

Gold, R., The Plenitude, unpublished manuscript, 2002

Goodell, J. "Steve Jobs: The Rolling Stone Interview." Rolling Stone, December 3, 2003 http://www.rollingstone.com/news/story/5939600/steve_jobs_the_rolling_stone_interview/ (Downloaded January 11, 2010)

Goodman, E., and M. Misilim. "The sensing beds." In UbiComp 2003 Workshop. 2003.

Gould, J. D., Conti, J., & Hovanyecz, T. (1983). "Composing letters with a simulated listening typewriter." Communications of the ACM, 26(4), 295–308.

Greenberg, S., Rounding, M. The Notification Collage. In Proceedings of CHI 2001, ACM Press, 514-521.

Greenfield, A., Everyware: The Dawning Age of Ubiquitous Computing, New Riders, 2006

Greenfield, A., Shepard, M., Urban Computing and its Discontents, Architectural League of New York, 2007

Gutwin, C., and Greenberg, S., "Workspace awareness for groupware." In Conference on Human Factors in Computing Systems, 208-209. ACM New York, NY, USA, 1996

Haladjian, R., "Let All Things Be Connected," presentation to the PICNIC 08 conference, October 22, 2008

Hall, E.T., The Hidden Dimension: Man’s use of Space in Public and Private. Doubleday, New York, 1966

Hall, T., "Poor Little Lisa," American Heritage Magazine, Volume 15, Issue 1, 1999

Hamilton, Anita, and et al. "Coolest Inventions 2004." Time Magazine, November 29, 2004, http://www.time.com/time/magazine/article/0,9171,995741,00.html (Accessed February 5, 2010)

Hartmann, B., Doorley, S., Klemmer, S. R. Hacking, Mashing, Gluing: Understanding Opportunistic Design. Pervasive Computing, IEEE 7, no. 3:46-54. 2008

Headrick, D.R., Technology: a world history, Oxford University Press, 2009

Heckman, D.. A small world: Smart Houses and the Dream of a Perfect Day. Duke University Press, March 2008

Hendler, D. Design and Magic Cap, General Magic, Inc., developer document, version 4/4/00, 2000

Highsmith, J., Agile Software Development Ecosystems, Addison-Wesley, 2001

Hill, D., "The Street as Platform," http://www.cityofsound.com/blog/2008/02/the-street-as-p.html (2008), retrieved May 3, 2009

Hillier, B. "The art of place and the science of space." World Architecture 185: 96-102., 2005.

Hillier, B., and J. Hanson. The social logic of space. 1984.

Hillsdale, NJ: Lawrence Erlbaum, 1995

Hiltzik, Michael A. 2000. Dealers of Lightning. HarperBusiness, March 23.

Holmquist L.E. and Skog, T. "Informative Art - Information Visualization in Everyday Environments," in Proceedings of GRAPHITE 2003 International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, ACM Press / ACM SIGGRAPH, 2003

Holmquist, L.E. "Bootlegging: Multidisciplinary Brainstorming with Cut-Ups." Proceedings of Participatory Design Conference 2008, Bloomington, IN, ACM Press, 2008

Holmquist, L.E., "From 'Human-Computer Interaction' to 'The Use and Design of Digital Artifacts,'" submission to alt.chi, CHI'08: ACM Conference on Human Factors in Computing Systems, Florence, Italy, 2008

Holtzblatt, K., Wendell, J. B., Wood, S., Rapid Contextual Design, Morgan Kaufmann, 2005

Hommels, A. "Studying obduracy in the city: Toward a productive fusion between technology studies and urban studies." Science, Technology & Human Values 30, no. 3: 323. 2005

Hong, Y, "A Dynamic Constructivist Approach to Culture: Moving from Describing Culture to Explaining Culture" in Understanding Culture: Theory, Research and Application (Wyer, R.S., Chiu, C., and Hong, Y., editors), Psychology Press, 2009

Howe, J., "The Rise of Crowdsourcing," Wired Magazine. 2006.

Hughes, T. P. American Genesis: A Century of Invention and Technological Enthusiasm, 1870-1970. University Of Chicago Press., 2004

Hulkko, S., Mattelmäki, T., Virtanen, K., & Keinonen, T. (2004). "Mobile probes." In Proceedings of the third nordic conference on human-computer interaction (pp. 43–51). (Tampere, Finland, October 23–27, 2004). NordiCHI ’04, Vol. 82. ACM, 2004

Hutchins, E. Cognition in the Wild. Cambridge, MA: MIT Press. 1996

Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B., Druin, A., Plaisant, C., Beaudouin-Lafon, M., Conversey, S., Evans, H., Hansen, H., Roussel, N., & Eiderbäck, B. "Technology probes: Inspiring design for and with families." In Proceedings of the CHI’03 (pp. 17-24). ACM Press. 2003

IGDA, "2006 Casual Games White Paper," International Game Developers Association, http://www.igda.org/casual/IGDA_CasualGames_Whitepaper_2006.pdf (Downloaded January 3, 2009)

Igoe, T., Making Things Talk. O'Reilly, 2007

InfoTrends, The North American Commercial Digital Display Market, white paper, 2008

Innis, H. A., Empire and communications. Oxford University Press., 1950

Ishii, H., M. Kobayashi, and J. Grudin. Integration of interpersonal space and shared workspace: ClearBoard design and experiments. ACM Transactions on Information Systems (TOIS) 11, no. 4: 349-375. 1993

Iwatani, Y., "Love: Japanese Style". In Wired News, 11 June 1998. Available at: http://www.wired.com/news/culture/ story/12899.html., 1998

Jobs, S., "Macworld 2001 Keynote," presentation, January 9, 2001

Johnson-Lenz, P., and Johnson-Lenz, T. "Groupware: The emerging art of orchestrating collective intelligence." in First Global Conference on the Future. 1980. Toronto, Canada.

Johnson, S., Emergence: The Connected Lives of Ants, Brains, Cities, and Software. New York: Scribner, 2002

Jones, M., and Marsden, G. Mobile interaction design. John Wiley & Sons, 2006

Jordan, J. M. "Beyond Servicization: The Rise of Digital Hybrids," white paper, Smeal College of Business Pennsylvania State University, http://www.smeal.psu.edu/cdt/pubs/white-papers/Beyond%20Servicization%209.07.pdf., 2007 (Downloaded February 8, 2009)

Journal of the Electronics Industry, "MiniDisc players spur Japan's audio sales, expand user base," Dempa Publications, Inc, ISSN: 0385-4515, 1998, http://www.faqs.org/abstracts/Electronics-and-electrical-industries/MiniDisc-shipments-roll-along-mainly-in-Japan-Europe-MiniDisc-players-spur-Japans-audio-sales-expand.html (Downloaded January 11, 2010)

Jyske Bank, "Jyske Bank - re-launch in domestic market," press release and promotional video, December 8, 2006

Kalanithi, J. J., Connectibles: Tangible Social Networking, Master's Dissertation, Massachusetts Institute of Technology, 2007

Karahalios, K., Donath, J. "Telemurals: Linking Remote Spaces with Social Catalysts." Proceedings of the ACM Conference on Human Factors in Computing Systems 2004, 2004

Kaye, J. and Goulding, L. "Intimate Objects." In DIS '04: Proceedings of the 2004 Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques. Cambridge, MA, USA: 1-4 August 2004. ACM Press, New York, NY, 2004

Kidd, C. D., R. Orr, G. D. Abowd, C. G. Atkeson, I. A. Essa, B. MacIntyre, E. Mynatt, T. E. Starner, and W. Newstetter. 1999. "The aware home: A living laboratory for ubiquitous computing research." Lecture Notes in Computer Science: 191-198.

Kikin-Gil, R., "BuddyBeads: techno-jewelry for non verbal communication within groups of teenage girls." Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, pages 375-376, 2005.

Kindberg, T., M. Chalmers, and E. Paulos. "Guest Editors' Introduction: Urban Computing." IEEE Pervasive Computing 6, no. 3: 18-20. 2008

Kirste, T., "Ambient Intelligence: Towards Smart Appliance Ensembles," unpublished manuscript, 2004

Kline, D. "The embedded internet," Wired Magazine 4: 98-109. 1996

Koolhaas, R. and Mau, B, S, M, L, XL New York: Monacelli Press, 1995

Krieger, M., "Improving Collaboration for Collective Action," Masters Thesis, Stanford University, 2009

Kunert, T., User-Centered Interaction Design Patterns for Interactive Digital Television Applications, Springer, 2009

Kuniavsky, M., "The Smart Furniture Manifesto," Metropolis, June 2004

Kuniavsky, M., "User Expectations in a World of Smart Devices," essay on adaptivepath.com. Initially published October 17, 2003. Retrieved May 31, 2009 from http://www.adaptivepath.com/ideas/essays/archives/000272.php

Kuniavsky, M., Projections into the world: service avatars as ambient intelligence objects. Interactions, 12(4): 55-57 (2005)

Kurt, T. E. 2006. Hacking Roomba. Wiley. 2006

Labrune, J. B., and Mackay, W. "Telebeads: social network mnemonics for teenagers." In Proceedings of the 2006 conference on Interaction design and children, 57-64. ACM New York, NY, USA, 2006

Lakoff, G., and Johnson, M.. Philosophy in the flesh: The embodied mind and its challenge to Western thought. Basic Books. 1999

Lamming, M., and M. Flynn. 1994. Forget-me-not: Intimate computing in support of human memory. In Proceedings of FRIEND21, 94:2-4. Vol. 94.

Latour, Bruno. Reassembling the Social: An Introduction to Actor-Network-Theory (Clarendon Lectures in Management Studies).  Oxford University. Oxford, UK. 2007

Laurel, B.: Computers as Theater. Addison-Wesley, Reading (1993)

Lawler, J. M., "Metaphors we compute by." Lecture notes available at http://www-personal.umich.edu/~jlawler/meta4compute.htm, 1987

Lenhart, A., "Teens and Mobile Phones over the Past Five Years: Pew Internet Looks Back," white paper, Pew Internet and American Life Project, August 2009

Levin, G., Shakar, G., Gibbons, S., Sohrawardy, Y, Gruber, J., Semlak, E., and Schmidl, G., Dialtones: A Telesymphony, performance, 2001-2002

Levy, S., Insanely Great - The Life and Times of the Macintosh, the Computer That Changed Everything, 1994

Lewis, S.C.R., "It must be a sign, " white paper, Lucid Signs, 2005

LG, "GRD-267DTU Digital Multimedia Side-By-Side Fridge Freezer with LCD Display," http://www.lginternetfamily.co.uk/fridge.asp (retrieved from Internet Archive, December 1, 2008)

Liebowitz, J., J. Liebowitz, J. Liebowitz, and K. Liebowitz. 1994. Intelligent systems at home in 2000. AI & Society 8, no. 4: 359-363.

Loguidice, B., and M. Barton. Vintage Games: An Insider Look at the History of Grand Theft Auto, Super Mario, and the Most Influential Games of All Time. Focal Press. 2009

Long, J., Magnolfi, J., Maassen, L., Always Building: The Programmable Environment, Herman Miller Creative Office, Zeeland, MI, 2008

Löwgren, J., and E. Stolterman. Thoughtful interaction design: A design perspective on information technology. The MIT Press, 2004.

Mackay, W.E., Ratzer, A.V., Janacek, P.: "Video artifacts for design: Bridging the gap between abstraction and detail." In: Proceedings of DIS 2000, pp. 72–82 (2000)

Maglio, P. P., S. Srinivasan, J. T. Kreulen, and J. Spohrer. Service systems, service scientists, SSME, and innovation. Communications of the ACM 49, no. 7: 81-85., 2006

Mankoff, J., A. Dey, G. Hsieh, J. Kientz, S. Lederer, M. Ames. "Heuristic Evaluation of Ambient Displays." In Proceedings of the CHI 2003, 2003

Mann, S., Nolan, J., Wellman, B. "Sousveillance: Inventing and Using Wearable Computing Devices for Data Collection in Surveillance Environments," Surveillance Society. 1.3 (2003):331-355, 2003

Marcus, A. 1998. Metaphor design in user interfaces. ACM SIGDOC Asterisk Journal of Computer Documentation 22, no. 2: 43-57.

Marx, L. The Machine in the Garden: Technology and the Pastoral Ideal in America. Oxford University Press. 1964.

Masten, D.L. and Plowman, T.M.P. "Digital Ethnography: The next wave in understanding the consumer experience." Design Management Journal. Vol 14 no 2. 2003. (2003), 75-81.

McCormick, G., "Q&A: Designing Virtual World Toys with Ginny McCormick, Techno Source" interview on virtualworldsnews.com, December 23, 2008, http://www.virtualworldsnews.com/2008/12/vwn-so-what-i-like-about-clickables-is-that-while-weve-seen-lots-of-toys-that-enable-content-in-virtual-worlds-these-reall.html (retrieved December 31, 2008)

McDonough, W. and Braungart, M. Cradle to Cradle: Remaking the Way We Make Things. New York: North Point Press, 2002.

McGonigal, J. E. 2006. This might be a game: ubiquitous play and performance at the turn of the twenty-first century. PhD Dissertation, University of California.

McLuhan, M, Fiore, Q. and Agel, J. War and peace in the global village. McGraw-Hill. 1968.

Mediamatic, Social RFID Games, http://www.mediamatic.net/socialRFID, 2008

Meloan, S. 2003. Toward a Global “Internet of Things”. Sun Developer Network.

Merholz, P., "Stop Designing Products," http://www.adaptivepath.com/blog/wp-content/uploads/2006/09/stop_designing_products.pdf, 2006

Michalko, M. 2006. Thinkertoys: A Handbook of Creative-Thinking Techniques. 2nd ed. Ten Speed Press, January 1.  

Microsoft Corporation, "Microsoft Office Labs: Envisioning," concept videos, 2009

Microsoft Corporation, "Microsoft Presents Smart Personal Objects Technology (SPOT)-Based Wristwatches at CES," press release, January 9, 2003

Mignonneau, L., and C. Sommerer. "Media Facades as Architectural Interfaces." The Art and Science of Interface and Interaction Design, 2008

Milgram, P., and F. Kishino. "A taxonomy of mixed reality visual displays." IEICE TRANSACTIONS on Information and Systems 77, no. 12: 1321-1329. 1994

Moggridge, B. Designing Interactions, The MIT Press, Cambridge, 2007

Mont, O. Product-service systems: Panacea or myth. The International Institute for Industrial Environmental Economics (IIIEE), Lund University: Lund, Sweden: 233, 2004

Moore, G.E., "Cramming more components onto integrated circuits," Electronics Volume 38, Number 8, April 19, 1965.

Mutanen, U., Biddulph, M., Thinglink White Paper, 2006

Myerson, J.M. RFID in the supply chain. Auerbach, 2006.  

Nebeker, F., Dawn of the Electronic Age: Electrical Technologies in the Shaping of the Modern World, 1914 to 1945, John Wiley & Sons, Inc., Hoboken, New Jersey, 2009

Nemeth, C. P., R. I. Cook, M. O’Connor, and P. A. Klock. 2004. "Using Cognitive Artifacts to Understand Distributed Cognition." IEEE Transactions on Systems, Man and Cybernetics—Part A: Systems and Humans 34, no. 6.  

Newman, M.W., Izadi, S., Edwards, W.K., Sedivy, J.Z., and Smith, T.F.. "User Interfaces When and Where They are Needed: An Infrastructure for Recombinant Computing," UIST'02, Paris, France, October 27-30, 2002. pp. 171-180.

Nichols, J., "Automatically Generating High-Quality User Interfaces for Appliances", PhD dissertation, Carnegie Mellon University, 2006

Nof, S. Y. (ed), Springer Handbook of Automation, Springer -Verlag Berlin Heidelberg, 2009

Nokia, "Nokia Mixed Reality," concept video, 2009 (retrieved September 15, 2009)

Nokia. "Mobile Mixed Reality." June 2009, http://research.nokia.com/files/insight/NTI_MARA_-_June_2009.pdf (Retrieved September 15, 2009).

Nokia. "Nokia - The New Nokia 252 Analog Cellular Phone Takes Color and Design to New Heights - Press Releases - Press - About Nokia." September 4, 1997. http://press.nokia.com/PR/199709/778036_5.html (Retrieved August 6, 2009).

Norman, D.A, Ortony, A., Revelle, W., "Effective Functioning: A Three Level Model of Affect, Behavior, and Cognition," in Who Needs Emotions? The Brain Meets the Machine, J. M. Fellous and M. A. Arbib, Editors, New York : Oxford University Press, 2005

Norman, Donald A., The Invisible Computer. The MIT Press. 1999

Norretranders, T. (1998) The User Illusion: Cutting Consciousness Down to Size, tr. J. Sydenham, Viking Press, Denmark, 1991

Nova, N., "Physical space and collaboration: How the environment supports social and collaborative processes," Course slides from course given at CSCW '05, 2005

O'Connell, M. A., and J. C. Cherry. The Health Hero (R) Online Service: A New Internet-Based Communications Platform for Disease Management, Case Management and Performance Measurement. Disease Management and Health Outcomes 7, no. 3: 149., 2000

O'Hara, K., Perry, M. and Lewis, S. "Situated Web Signs For Socially Mediated Space Management: Understanding social coordination around a room reservation display appliance." in O’Hara, K., Perry, M., Churchill, E. and Russell, D. (Eds) Public and Situated Displays. Social and Interactional aspects of shared display technologies. Kluwer Academic Publishers, 2003

O'Hara, K., Perry, M. and Lewis, S., Social Coordination Around a Situated Display Appliance.  In Proceedings of CHI ’03 Conference on Human Factors in Computing Systems, Ft Lauderdale, Florida., 2003

O'Malley, Michael. Keeping watch. Viking. 1990.

O'Reilly, Tim. "What is Web 2.0: Design Patterns and Business Models for the Next Generation of Software." Communications & Strategies, no. 65, 2007

O’Sullivan, D., Igoe, T.: Physical Computing: Sensing and Controlling the Physical World with Computers. Thomson, Boston, 2004

Ogasawara, Y. "Revamped 'Power Generating Floor' to Be Tested at Tokyo Station," Nikkei Electronics, December 4, 2008 http://techon.nikkeibp.co.jp/english/NEWS_EN/20081204/162357/.

Olsen Jr, D. R., S. T. Nielsen, and D. Parslow. "Join and capture: a model for nomadic interaction. "In Proceedings of the 14th annual ACM symposium on User interface software and technology, 131-140. ACM New York, NY, USA. 2001

Open Mobile Alliance, XHTML MP specifications, http://member.openmobilealliance.org/ftp/public_documents/bac/MAE/Permanent_documents/OMA-XHTMLMP-V1_2-20041222-D.zip, 2004

Oster, D. P. 2009. Flying Through Google Earth at Macworld. January 8. http://google-latlong.blogspot.com/2009/01/flying-through-google-earth-at-macworld.html.

Ottawa, O. A Brief History of the Future of Urban Computing and Locative Media. Carleton University PhD Dissertation. 2008

Oulasvirta, A., E. Kurvinen, and T. Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and Ubiquitous Computing 7, no. 2: 125-134.

Pahlavan, K., T. H. Probert, and M. E. Chase. 1995. Trends in local wireless networks. IEEE Communications Magazine 33, no. 3: 88-95.  

Parasuraman, R., Sheridan, T.B., Wickens, C.D. "A model for types and levels of human interaction with automation," IEEE Transactions on Systems, Man, and Cybernetics: Part A, Systems and Humans. 30(3), 286–297 (2000)

Parker, D.S and Stedman, T.C., "Measured Electricity Savings of Refrigerator Replacement: Case Study and Analysis," ACEEE 1992 Summer Study on Energy Efficiency in Buildings, Proceedings - Commercial Performance: Analysis and Measurement Panel 3, American Council for an Energy-Efficient Economy. 1992

Paulos, E. and Goodman, E. The Familiar Stranger: Anxiety, Comfort and Play in Public Places. Proc. SIGCHI’04 Conference on Human Factors in Computing Systems, 2004.

Paulos, E., and T. Jenkins. 2005. "Urban Probes: Encountering Our Emerging Urban Atmospheres." In CHI 2005: Technology, Safety, Community: Conference Proceedings: Conference on Human Factors in Computing Systems: Portland, Oregon, USA, April 2-7, 341. Association for Computing Machinery (ACM). 2005

PC World Editors. "The iPhone: Lots to Love, but Flaws Too." PC World, June 30, 2007.

Pering, C. "Pet pals: a game for social mediation." In CHI ’02 extended abstracts on human factors in computing systems (pp. 778–779). (Minneapolis, Minnesota, USA, April 20–25, 2002). CHI ’02. ACM. 2002

Perryman, N., "Doctor Who and the Convergence of Media," Convergence, Sage Publications, 2008

Persson, P., Jung, Y. (2005) "Nokia Sensor: From research to Product," Conference on Designing for User eXperience (DUX) Proceedings, San Francisco, CA, Nov 3-5, 2005

Petroski, H. 1994. The evolution of useful things. Vintage Books.

Philips, FW-M55 datasheet, 2000

Pierce, J. S., H. Mahaney, and G. Abowd. 2004. Opportunistic annexing for handheld devices: Opportunities and challenges. Proceedings of HCIC 2004.

Pillkahn, Ulf. 2008. Using Trends and Scenarios as Tools for Strategy Development: Shaping the Future of Your Enterprise. Wiley, August 4.

Pinch, G., Magic in Ancient Egypt, British Museum Press, 1994

Pogue, David. "Some E-Books Are More Equal Than Others." Pogue's Posts (The New York Times). http://pogue.blogs.nytimes.com/2009/07/17/some-e-books-are-more-equal-than-others/, July 17, 2009.

Postrel, V. The Substance of Style: How the Rise of Aesthetic Value Is Remaking Commerce, Culture, and Consciousness. Harper Perennial, 2004

Pousman, Z., Stasko, J., "A taxonomy of ambient information systems: four patterns of design," Proceedings of AVI'06: the working conference on Advanced visual interfaces, Venice, Italy, 2006

Povall, R. "Realtime control of audio and video through physical motion: STEIM's BigEye" In Proc. In Journées d'Informatique Musicale, 1996

Prante, T., C. Röcker, N. Streitz, R. Stenzel, C. Magerkurth, D. van Alphen, and D. A. Plewe. 2003. "Hello.Wall–Beyond Ambient Displays". In Video and Adjunct Proceedings of UBICOMP Conference. 2003

Prensky, M. 2001. Digital natives, digital immigrants. On the horizon 9, no. 5: 1-6.  

PriceGrabber.com, Consumer Behavior Report on Mobile Internet Shopping Trends, April 2009, https://mr.pricegrabber.com/Mobile_Internet_Shopping_Trends_April_2009_CBR_1.pdf (accessed August 5, 2009)

Pyla, P. S., M. Tungare, and M. Perez-Quinones. "Multiple User Interfaces: Why Consistency is Not Everything, and Seamless Task Migration is Key." In Proceedings of the CHI 2006 Workshop on The Many Faces of Consistency in Cross-Platform Design, 2006

Raff, D. M. G. "Superstores and the evolution of firm capabilities in American bookselling." Strategic Management Journal: 1043-1059, 2000

Ravasi, D., and G. Lojacono. 2005. Managing design and designers for strategic renewal. Long Range Planning 38, no. 1: 51-77.  

Redström, J. Designing Everyday Computational Things. rapport nr.: Gothenburg studies in Informatics, 2001

Reeves, R. and Nass, C. The media equation: How people treat computers, televi- sion, and new media like real people and places, CSLI Publications and Cambridge university press, 1996.

Rekimoto, J. "From Stone to Skin: Organic Interaction Technologies." Communications of the ACM 51, no. 6. 2008

Rekimoto, J. 1997. Pick-and-drop: a direct manipulation technique for multiple computer environments. In Proceedings of the 10th annual ACM symposium on User interface software and technology, 31-39. ACM New York, NY, USA.

Richter, K., Nichils, J., Gajos, K., Seffah, A., Proceedings of the CHI 2006 Workshop on The Many Faces of Consistency in Cross-Platform Design, 2006

Röcker, C., "Requirements and Guidelines for the Design of Team Awareness Systems", Proceedings of the World Academy of Science, Engineering and Technology, p. 164, March, 2009.

Rogers, E. M., Diffusion of innovations, New York: Free Press, 1962

Rogers, Y. 2006. Moving on from Weiser's vision of calm computing: Engaging UbiComp experiences. Lecture Notes in Computer Science 4206: 404.

Rogers, Y., S. Price, G. Fitzpatrick, R. Fleck, E. Harris, H. Smith, C. Randell, H. Muller, C. O'Malley, and D. Stanton. "Ambient wood: designing new forms of digital augmentation for learning outdoors." In Proceedings of the 2004 conference on Interaction design and children: building a community, 3-10. ACM New York, NY, USA. 2004

Rohrer, T. "Metaphors we compute by: bringing magic into interface design." Center for the Cognitive Science of Metaphor, Philosophy Department, University of Oregon., accessed at http://www.uoregon.edu/~uophil/metaphor/gui4web.htm, 1985

Roibás, A. C., Johnson, S., "Use of Experimental Ethno-Methods to Evaluate the User Experience with Mobile Interactive Multimedia Systems" in Handbook of Research on User Interface Design and Evaluation for Mobile Technology (Lumsden, J. ed), Information Science Reference, 2008

Rollings, A. and Adams, E, Andrew Rollings and Ernest Adams on Game Design, New Riders Publishing, 2003

Rosella, F. and Sakai, R. F+R HUGS, http://projects.interaction-ivrea.it/2002/desire/f+rhugs/F+R HUGS.pdf, 2002 (link broken as of May 15, 2009)

Rosen, M., Lublinsky, B., Smith, K.T., Balcer, M.J., Applied SOA: Service-Oriented Architecture and Design Strategies, Wiley Publishing, Indianapolis, IN, 2008

Roussel, N., H. Evans, and H. Hansen. 2004. Proximity as an interface for video communication. IEEE Multimedia 11, no. 3: 12-16.

Rudström, S, Svensson, M, Cöster, R, and Höök, K, "MobiTip: Using Bluetooth as a Mediator of Social Context," UbiComp 2004: Ubiquitous Computing: 6th International Conference, Adjunct Proceeding, 2004s

Saffer, D., "Learning Interaction Design from Las Vega$," presentation at SXSW 2007, http://www.odannyboy.com/portfolio/presentations/learningfromvega$.pdf (Downloaded October 5, 2009)

Saffer, D., Designing for Interaction, Peachpit Press, 2006

Saffer, D., Designing Gestural Interfaces, O'Reilly and Associates, 2008

Saffer, D., The Role of Metaphor in Interaction Design, Master's Thesis, Carnegie Mellon University, 2005

Salvador, T., Bell, G., and Anderson, K. (1999), "Design Ethnography," Design Management Journal, 33. 1999

Schank, R. C., and R. P. Abelson. 1977. Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Lawrence Erlbaum Associates Hillsdale, NJ.

Schenker, J. L. "MPedigree's Rx for Counterfeit Drugs." BusinessWeek: Europe, December 3, 2008 http://www.businessweek.com/globalbiz/content/dec2008/gb2008123_027994.htm?chan=globalbiz_special+report+--+tech+pioneers+of+2009_special+report+-+tech+pioneers+of+2009 (Retrieved December 20, 2009)

Schmitt, A., http://www.gratin.org/as/design.html (retrieved June 10, 2009)

Scholl, G., "Taking a Functional and a Symbolic Perspective on Usership," Perspectives on Radical Changes to Sustainable Consumption and Production (SCP), Proceedings of the Workshop of the Sustainable Consumption Research Exchange (SCORE!) Network, Thursday 20 and Friday 21 April 2006, Copenhagen, Denmark, 2006

Schultz, D. E., and P. J. Kitchen. 1997. Integrated Marketing Communications in US Advertising Agencies: An Exploratory Study. Journal of Advertising Research 37, no. 5: 7-18., 1997

Schwartz, Peter. The Art of the Long View: Planning for the Future in an Uncertain World. Currency Doubleday, April 15, 1996.

Scott, B. and Neil, T., Designing Web Interfaces, O'Reilly, 2009

Sevtsuk, A.: "Real-Time Space Management," in Pae, K., Ahas, R., Mark U. (eds.), Joint space : open source on mobile positioning and urban studies, Positium OÜ 2006, Tallin,p. 31ff

Shardanand, U., and P. Maes. 1995. Social information filtering: algorithms for automating “word of mouth”. In Proceedings of the SIGCHI conference on Human factors in computing systems, 210-217. ACM Press/Addison-Wesley Publishing Co. New York, NY, USA.

Sharp, K. and Earle, S. "Cyberpunters and Cyberwhores: Prostitution on the Internet", in Y. Jewkes (ed.) Dot.cons: Crime, Deviance and Identity on the Internet, pp. 36–52. Cullompton, Devon: Willan Publishing, 2003

Sharpe, W., "Information Appliances, An Introduction," white paper, Appliance Studio, 2001, http://www.appliancestudio.com/publications/whitepapers/ApplianceIntro.pdf, accessed March 16, 2006.

Sharpe, W.P., and Stenton, S.P. 2003. "Information Appliances." The Human-computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications: 731.

Shwe, H.. "Smarter play for smart toys," Zowie Intertainment White Paper No. 3208, 1999.

Siio, I., N. Mima, I. Frank, T. Ono, and H. Weintraub. 2004. "Making Recipes in the Kitchen of the Future." In Conference on Human Factors in Computing Systems, 1554-1554. ACM New York, NY, USA.

Silverberg, C., "Real Touch: the Finger Review," about.com, http://sexuality.about.com/b/09/07/01/real-touch-the-finger-review.html, July 1 2009, (Retrieved August 30, 2009)

Skog, T., Ljungblad, S. & Holmquist, L.E.. "Between Aesthetics and Utility: Designing Ambient Information Visualizations." In Proceedings of InfoVis, 2003

Slater, M., "Why Pricey Gadgets Remain a Tough Sell," Appliance Design magazine, August 13, 2001, http://www.appliancedesign.com/CDA/Archives/250950f96b938010VgnVCM100000f932a8c0____, accessed March 16, 2006.

Smart, E. J., J. Cascio, and J. Paffendorf. "Metaverse roadmap overview" (white paper) http://www.metaverseroadmap.org/MetaverseRoadmapOverview.pdf  (Downloaded September 15, 2009). 2007

Smith, R. B. 1987. Experiences with the alternate reality kit: an example of the tension between literalism and magic. ACM SIGCHI Bulletin 18, no. 4: 61-67.

Snopes, "Webkinz Murderer," http://www.snopes.com/horrors/madmen/webkinz.asp, January 3, 2008 (Downloaded March 1, 2010)

Snowdon, D., Grasso, A., Diffusing information in organizational settings: learning from experience. Proceedings of CHI 2002, Minneapolis, Minnesota, USA, ACM Press, 2002

Snyder, C., Paper Prototyping, Morgan Kaufmann, 2003

Spiekermann, S.S., User Control in Ubiquitous Computing: Design Alternatives and User Acceptance, Shaker Verlag, Aachen, 2008

Spigel, L., "Designing the Smart House; Posthuman Domesticity and Conspicuous Production," European Journal of Cultural Studies, 8(4): 403-26, 2005

Starner, T., S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R. W. Picard, and A. Pentland. 1997. "Augmented reality through wearable computing." Presence: Teleoperators and Virtual Environments 6, no. 4: 386-398.

Steelcase, RoomWizard Reporting Tool User Guide, 2002

Steelcase, RoomWizard Specifications, 2002

Steele, V. (ed), Encyclopedia of clothing and fashion. Thomson. 2005

Stephenson, N. Snow Crash. New York: Random House, Inc., 1992

Stephenson, N., In the Beginning Was the Command Line, Harper Perennial, 1999

Sterling, B. "When Blobjects Rule the Earth." In SIGGRAPH 2004.

Sterling, B., "Viridian Note 00241: Furniture Fair 02001," Contemporary Furniture speech ICFF, New York City, May 19, 2001, http://www.viridiandesign.org/notes/226-250/00241_furniture_fair.html (Retrieved December 20, 2009).

Sterling, B., "Viridian Note 00113: Pervasive Computing," IDSA/IBM Designabout on Pervasive Computing, December 3, 1999, http://www.viridiandesign.org/notes/101-125/00113.html (Retrieved December 20, 2009).

Sterling, B., Shaping Things, Cambridge, Massachusetts: MIT Press, 2005

Steuer, J. (1995) ‘Defining Virtual Reality: Dimensions Determining Telepresence’, in F. Biocca and M. Levy (eds) Communication in the Age of Virtual Reality, pp. 33–56.

Streefkerk, J.W, van Esch-Bussemakers, M. P., Neerincx, M.A., Looije, R., "Evaluating Context-Aware Mobile Interfaces for Professionals," Handbook of Research on User Interface Design and Evaluation for Mobile Technology (Lumsden, J. ed), Information Science Reference, 2008

Streitz, N. A., J. Geißler, T. Holmer, S. Konomi, C. Müller-Tomfelde, W. Reischl, P. Rexroth, P. Seitz, and R. Steinmetz. "i-LAND: an interactive landscape for creativity and innovation." In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, 120-127. ACM New York, NY, USA. 1999

Streitz, N., and P. Nixon. "The disappearing computer." Communications of the ACM 48, no. 3. 2005

Strommen, E., "When the interface is a talking dinosaur: Learning across media with Actimates Barney." Proceedings of CHI ’98. ACM Press, pp. 288-295, 1998

Suchman L.A. Plans and Situated Actions: The Problem of Human Machine Communication. Cambridge: Cambridge University Press, 1987

SusProNet, Sustainable Product-Service Systems, pamphlet, 2002

Tardo, J., Valente, L., "Mobile agent security and Telescript." Compcon'96.'Technologies for the Information Superhighway' Digest of Papers: 58-63, 1996

Thackara, J. 2000. Edge effects: the design challenge of the pervasive interface. In Conference on Human Factors in Computing Systems, 199-200. ACM New York, NY, USA.

Thackara, J. In the bubble. Designing in a complex world. Cambridge, MA: MIT Press., 2005

Thevenin, D., and J. Coutaz. 1999. Plasticity of user interfaces: Framework and research agenda. In Human-computer Interaction, INTERACT'99: IFIP TC. 13 International Conference on Human-Computer Interaction, 30th August-3rd September 1999, Edinburgh, UK, 110. IOS Press.

Tognazzini, B. The “Starfire” video prototype project: a case history. In Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, 99-105. ACM New York, NY, USA. 1994

Tufte, E. R. The Visual Display ofQuantitative Information. Cheshire, CT: Graphics Press.(Original work published 1983), 2001

Turkle, S. 1984. The second self. Simon and Schuster New York.  

Ullmer, B. & H. Ishii. "Emerging Frameworks for Tangible User Interfaces." IBM Systems Journal 39 (3 & 4), 915–931, 2000

US Bureau of Mines, Aluminum Statistics Worksheet, 2007

V Sync Technology, "Internet Refrigerator" press release, May 1, 1998

Vainio A.-M., Valtonen M., and Vanhala J. "Learning and adaptive fuzzy control system for smart home" in Developing Ambient Intelligence, Proceedings of the First International Conference on Ambient Intelligence Developments (AmID'06), 2006

Vallgårda, A., and J. Redström. Computational composites. In Proceedings of the SIGCHI conference on Human factors in computing systems, 513-522,  ACM Press New York, NY, USA., 2007

Vander Wal, T. 2005. "Explaining and showing broad and narrow folksonomies: Personal infocloud." Blog posting. February 21, 2005. http://personalinfocloud.com/2005/02/explaining_and_.html (Retrieved May 31, 2009)

Venturi, R., D. S. Brown, and S. Izenour. 1972. Learning from Las Vegas. MIT Press Cambridge, Mass.

Vetere, V., Gibbs, M.R., Kjeldskov, J., Howard, H., Mueller, F., Pedell, S., Mecoles, K. and Bunyan, M., "Mediating Intimacy: Designing Technologies to Support Strong-Tie Relationships," CHI 2005: Technology, Safety, Community: Conference Proceedings: Conference on Human Factors in Computing Systems: Portland, Oregon, USA, April 2-7, 341. Association for Computing Machinery (ACM). 2005

Villarroel J.A. and Tucci C.L., "Motivating firm-sponsored e-collective work," unpublished, 2009

Violet, Nabaztag API (V2), specification, published March 24, 2006

Vogel, D., and R. Balakrishnan. 2004. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In Proceedings of the 17th annual ACM symposium on User interface software and technology, 137-146. ACM New York, NY, USA.

von Hippel, E., "Lead users: a source of novel product concepts," Journal of Management Science, Vol. 32, No. 7. July 1986

Wan, D. "Magic Medicine Cabinet: A Situated Portal for Consumer Healthcare." In Handheld and Ubiquitous Computing: First International Symposium, Huc'99, Karlsruhe, Germany, September 27-29, 1999, Proceedings, 352. Springer.

Wang, J., M. Lewis, and J. Gennari. 2003. "USAR: A game based simulation for teleoperation." In Proc. 47th Ann. Meeting Human Factors and Ergonomics Soc, 2003.

Ware, C., editor, Information Visualization: Perception for Design, Second Edition. Morgan Kaufmann. 2004. 

Wasson, C. "Ethnography in the field of design," Human Organization; Winter 59, 4. 2000

Weiser, M,. Gold, R., Brown, J.S., "The origins of ubiquitous computing research at PARC in the late 1980s," IBM Systems Journal, Vol. 38, No. 4, 1999

Weiser, M. "Creating the invisible interface: (invited talk)." UIST '94: Proceedings of the 7th annual ACM symposium on User interface software and technology. ACM, 1994

Weiser, M. "Some computer science issues in ubiquitous computing." Communications of the ACM, 36(7):75–83, July 1993.

Weiser, M. and Brown, J.S. "Designing Calm Technology." PowerGrid Journal, 1:1, 1996.

Weiser, M., "The Computer for the Twenty-First Century." Scientific American, September 1991.

Weiser, M., "The World Is Not a Desktop," ACM Interactions, vol. 1, no. 1, Jan. 1994, pp. 7–8.

Westin, A., Privacy and Freedom. New York: Atheneum, 1967.

Whirlpool, in,kitchen: design landscapes for a new built-in experience, published by Whirlpool Corporation, 2004

Wickens, C. D. 1991. Processing resources and attention. Multiple-task performance: 3–34.

Wiener, N., Cybernetics. Wiley. 1948

Wikipedia, "Nike+iPod," http://en.wikipedia.org/wiki/Nike%2BiPod (Accessed November 23, 2008).

Wilson, Ian, and Bill Ralston. Scenario Planning Handbook: Developing Strategies in Uncertain Times. 1st ed. South-Western Educational Pub, June 9., 2006

Wilson, J.S. (editor), Sensor Technology Handbook, Newnes: Oxford, 2005

Wilson, R. A. and Keil, F.C. (eds), The MIT Encyclopedia of the Cognitive Sciences, 1999

Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P. (1998). Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information. in Proceedings of International Workshop on Cooperative Buildings (CoBuild '98), (Darmstadt, Germany, February 1998). Lecture Notes in Computer Science. Springer Publishing. 22-32.

Wong, Y. Y. (1992) "Rough and ready prototypes: lessons from graphic design" 1992 SIGCHI conference on Human factors in computing systems. Monterey, California ACM Press.

Wyche, S. P. Designing speculative cleaning products: using elders' past and present cleaning experience to inform design. Masters Thesis. Cornell University. 2004

Yakal, K., "Smart Products," Compute! Magazine, September 1983, http://www.atarimagazines.com/compute/issue40/smart_products.php (Retrieved December 20, 2009)

Yamada, T., Shingu, J., Churchill, E.F., Nelson, L., Helfman, J., and Murphy, P. "Who cares? Reflecting who is reading what on distributed community bulletin boards." Proceedings of UIST 2004, October 24–27, 2004, Santa Fe, New Mexico, USA. ACM Press.

Young, I., Mental Models, New York: Rosenfeld Media, 2006

Posted by at 12:39 PM | Permalink

(Photo CC Attribution-Share Alike 2.0 by Marshall Astor)

Once again I had the honor of being invited to speak to Kimiko Ryokai's Theory and Practice of Tangible User Interfaces class at UC Berkeley today. I used the opportunity to prepare a talk on a topic I've been thinking about for a while: the relationship between products and services in a ubicomp environment. For this presentation, I set out some of my ideas and presented a set of examples of device/service relationships that show some of the variation in the possibilities.

The talk is available as an 844K PDF.

Posted by at 1:51 PM | Permalink

In writing my book, I've been trying to keep track of companies that are creating consumer-facing information shadows for various kinds of products (as opposed to the other kind of item-level identification technologies that are primarily for use by businesses in their logistics operations).

Here are several of them, and the products they're tracking:

- High Fashion, CertiLogo (a client of mine)
- High Design, ThingLink
- Food, TraceTracker (and a BusinessWeek story about them)
- Food and high technology, YottaMark
- Food, FoodLogiQ
- Pharmaceuticals, mPedigree (another BusinessWeek story about them)
- Goods in general, Sproxil, which appears to be a for-profit venture by the founders of mPedigree, a nonprofit.

There are many other companies that are doing other kinds of identification (for example, tesa scribos, but

Posted by at 5:36 PM | Permalink

Thanks to Tim O'Reilly's generous promotion of my Etech talk, the concept of information shadows has been mentioned a bit in the last couple of weeks.

I'm very flattered, and although the coinage of the specific term is roughly mine, I did not originate the concept. The idea of data entities associated with objects, but having lives of their own, probably goes back as far as the oldest identification technologies, but there are several precedents that deserve mention (especially Adam Greenfield's). Here's a tiny bibliography of two of them (and thank you Google Scholar for helping me find them).

Data shadow

More modern references can be found at Word Spy

Informational Shadow

Davidson, Paul, and Louise Davidson. The Collected Writings of Paul Davidson. 1990. (this is a collection of macroeconomic essays, one of which says that we can't make judgments about the future because it does not cast an informational shadow back toward us--this is a very different use of the term, but still obliquely related)

Greenfield, Adam. Everyware, 2006 (My information shadow is nearly identical to Adam's informational shadow. In Everyware, he writes: "The significance of technologies like RFID and 2D barcoding is that they offer a low-impact way to "import" physical objects into the datasphere, to endow them with an informational shadow.")

Posted by at 2:56 PM | Permalink

This is an outline of a project that I've had on the drawing board for years, and it looks like I'm not going to actually instantiate it, so I decided today (after being prompted by a foo camp mailing list thread) to say screw it and give the idea out to the world, for better or for worse.

The core of my idea is this: that where ethernet-over-power (also known as Powerline, or HomePlug) is useful is for communication with and control of household devices. I've ranted for a long time that there isn't a good appliance communication protocol, but what I've come to realize is that it's not that there isn't a good protocol, but that all the so-called standards that try to solve smart device communications try to reinvent every layer at once. That's shortsighted, because it ends up with mass incompatibilities at all levels, so there is no agreement between device manufacturers at any level, and all of the consortia are just mini-trusts trying to get vendor lock-in so that they can be the sole suppliers of the technology. It's big companies trying to get vertically-integrated vendor lock-in and failing.

Look, folks, we have all of the pieces and we don't have to create any new standards. Here's how I see it:

1. Wifi is great for moderately high-speed general-purpose communication to easily-movable end-user devices (and I don't mean just "portable," since this includes things like printers and set-top boxes).
2. Cat6 is good for very high-speed communications between devices that don't move.
3. Bluetooth, zigbee, z-wave, and all of the other short-haul, low-power, low-bandwidth wireless standards are good for movable devices that need highly near-range communication.
4. Ethernet-over-power is good for low-speed communication to static devices.

Anyway, I digress. My point here is to discuss a specific application for Ethernet-over-power. I've been enamored with this technology for a while, but it's struggled in the market by trying to compete with Wifi and failing. The lack of a wire, even if it's a power cable, will always beat out the wire. This competition has lead e-o-p's developers to continue to pour money into making it faster, rather than making the technology cheaper. This has limited its use to a small niche of people for whom neither Wifi nor Cat6 Ethernet works. That's essentially like saying "We're going to make cars for people who like cars that are neither fast, nor capacious, nor cheap." Sure, you'll find some niche, but it's not going to be big.

I feel that the big niche in smart household device communication. Essentially, optional low-bandwidth communication between devices that are already going to be plugged in that helps them work together, but doesn't form the core of their functionality.

Let me give you an example:

1. You subscribe your e-o-p-enabled DSL modem to an electricity price service. It gets spot prices every 15 minutes or so from one of the realtime electricity price services.
   
2. It then broadcasts that information as TCP/IP broadcast packets over the local e-o-p network.
   
3. Lights throughout the house/workplace are equipped with a digital dimmer that is listening to power price packets.
   
4. When the price goes over some value (which could be set once a day through a slightly different kind of broadcast packet) the lights go into power-saving mode and dim.
   

The technology all exists. All of it. And I'm sure it's already possible to make it cheap enough that it adds $1 or so to the price of devices at the low end. These devices do not have to be sold in special configurations that only work if you buy a single company's (or consortium's) products, they can just be sold as what they are: lamps, microwaves, picture frames, clocks, etc. The functionality only needs to come into play if you want it, and it device works as advertised whether there are any other devices on the house network or not.

The core value is that this solution creates a market justification for developing inexpensive devices that have the capability of augmented functionality, without requiring that functionality to take center stage in terms of what the devices do. This, I believe, makes the adoption of these devices by consumers more likely, and therefore the further development of such technology, and therefore the network effects that everyone wants. Until people start using the open standards that are already available, they will forever be stuck on lonely, unprofitable islands of proprietary standards, even ones that are touted as open.

[12/31/08 update: I just learned that this is called demand response in the energy business. So I guess what I'm advocating for is technologies for the development of small-scale demand response systems using ethernet-over-power broadcasts of energy pricing information.]

Posted by at 2:57 PM | Permalink

I was one of the international keynote presenters at this year's Dansk IT Usability and Design conference. I would first like to thank them for the invitation: it was a pleasure to spend a couple of days in Copenhagen and an honor to present to such a distinguished organization (they're an IT organization that just turned FIFTY!).

In my presentation I rolled up a bunch of my ideas from the last six months and added some examples of some new projects (such as Disney/TechnoSource's Clickables-PixieHollow product line) and I talked about the iPhone's applianceness.

You can download the presentation (792K PDF) with extensive notes.

The gist of this keynote, as with many of the presentations I've been giving over the last six months, is that a combination of ubiquitous computing, wireless networking and item-level identification is changing the nature of people's relationship to everyday objects. This change, in turn, creates a number of deep user experience design challenges as objects become intertwined with services and as computation becomes a more ingrained part of how the object is designed. In other words, objects that we find familiar now dematerialize into services, while abstract ideas that had been services before materialize as new, and unfamiliar, appliances. This crossover is pretty alien and implies a rethinking of relationships and design processes.

I'm still working on the practical implications that these big ideas boil down to, but I'm beginning to see the outline of what it implies for the world in which design is going to happen for the next 5-10 years.

Posted by at 7:10 PM | Permalink

I wrote an article on ubiquitous computing user experience design for ACM's interactions magazine. The final article is only available to subscribers, but here's a preprint version of it:

Ubiquitous Computing User Experience Design

I think 2005 was the year we began living in the world of commonplace ubiquitous computing devices. That year Apple put out the screenless iPod Shuffle, Adidas launched the adidas_1 shoe, and iRobot launched the Discovery—its second-generation vacuum robot.

Sadly, even though we live in that world, the user experience design of most everyday ubiquitous computing devices—things you see in gadget blogs—is typically terrible. That’s because we do not address ubicomp user experience design as a distinct branch of interaction design, much as we did not treat interaction design as separate from visual design in the early days of the Web.

In the last couple of years, I have conducted research for and designed a number of ubicomp user experiences. In the process, I've seen some of the seams between industrial design, interaction design, architecture, and ubiquitous computing user experience design. In this article, I have tried to pull together some approaches that seem particularly valuable in the ubiquitous computing user experience world. None is unique to it: They’re all general design guidelines, but they seem to apply particularly well to the particular design challenges of this field.

Make Tools, Not Platforms

It is because CPU power is so cheap that ubicomp UX design should concentrate all design and processing on a narrowly focused set of functionalities. Yes, a single device can be a dictionary, a calendar, a notebook, an alarm clock, a TV, an audio recorder, play every media format, and work as an 8-bit game machine, but doesn’t that just sound like an underpowered laptop?

Define Services Before Designing Devices

ThingM, my company, developed WineM, our prototype smart wine rack, as an avatar of a service. The rack uses RFIDs on each bottle to track where every bottle is and then displays information using glowing LEDs behind the bottles. When we designed it, we treated the rack as one way to provide access to a service that associated a specific bottle with metadata about it, which was in turn part of a system that linked wine producers, distributors, retailers, and consumers together in such a way that everyone in the chain benefited from adopting the technology. The rack is a particularly visual manifestation of the service, but the service would be available through an API that could be accessed through many avenues.

Don't Overload Affordances

Mixing the two philosophies can create confusion. Your doorknob doesn’t double as a volume control for your stereo, though in today’s fly-by-wire world, it can. For example, when BMW developed its iDrive system, which mapped a large number of different functions to a single input device, the mismatch in expectations created interface havoc that took the company many revisions to correct.

(image copyright Nick Humphries, CC Licensed)

Don’t Reinvent the Wheel

Respect the Society of Devices

(image copyright Joichi Ito, CC Licensed)

Riffing off of Marvin Minsky’s Society of Mind, let’s call this technology cloud the society of devices. Each device does something specific, and some are more powerful than others. How do they all work together? How do they integrate into the larger set of devices and services out in the world?

On the interaction-design level, this means understanding users and their needs in light of the all of the devices that they may have. For example, while it’s possible to get email on many different devices, presenting it in a way that respects the unique constraints of a device and stays consistent with other devices becomes key when helping people transition between them. Text email accomplishes this using a universal format (text) with a well-defined structure (To:, From:, etc.). The minute that an attachment is included or there is HTML in the message, that consistency vanishes.

Create Physical Behaviors, Not Visual Representations

Industrial design incorporates the physical senses of temperature, texture, and vibration into devices. Ubicomp UX is essentially the coupling of these two sets of ideas to create behaviors that match information priority with available sensory bandwidth and less cognitive load.

For example, say I’m looking for a new apartment in the town where I already live. I don’t need to move, but I’d like to. I set my (hypothetical) GPS unit to download a data stream of apartments that match my criteria of price, size, neighborhood, and proximity to at least three cafes with free Wi-Fi. As I drive/ride/walk around the city when I approach one of these locations, the GPS vibrates in proportion to how well it matches my criteria. I don’t need to look at it; I just need to feel it to get the crucial piece of information.

(photo by Timo Arnall)

Use Information Processing As a Material

I believe that ubiquitous computing technologies are incredibly powerful. However, ubicomp user experience design is still a very young discipline, without a track record of obvious best practices. In its failures, we see the inadequacy of applying older design paradigms to the capabilities of new technologies. If design people first encounter new technologies through design, then careful reflection on our design processes early on is essential for increasing the chances of technology’s positive impact. That time is now.

Posted by at 12:07 PM | Permalink

PICNIC

I spent last week in Amsterdam at the PICNIC conference. Vlad Trifa of SAP/ETH invited me to present at an Internet of Things special session he organized, and it was one of the highlights of the conference. His timing was impeccable, with the session arriving just days after Cisco's Internet of Things consortium (IPSO Alliance) announcement (which, tangentially, now canonizes that term as yet another name for ubiquitous computing, though, as a term, you could certainly do worse and you could argue that it's the sub-1m granularity of ubicomp). It was great to share the tiny cafe stage with folks representing a wide range of organizations from giant conglomerates and emerging players to other fledgling startups.

I gave a talk entitled Shadows and Manifestations (440K PDF) (Matt Jones has posted a video of the talk on Vimeo--thanks Matt!) that focused on several of the ways I've been thinking about ubicomp UX design (and, by extension, Internet of Things UX design). If you've collected the whole set, there's little that's totally new. I have expanded my thinking on information processing is a material with some historical parallels to other materials and I have included some newer thinking about the implications of digital identification technology and ubicomp.

The most interesting result of the session for me was the high degree of similarity between the various ideas. This could just be a product of Vlad's curatorial process, but there were uncanny resonances between a number of the ideas in the presentation, and many ideas came up repeatedly. My presentation was roughly in the middle of the session, and I spent the whole first half frantically updating my slides in response to what others were saying. It was clear that in this group it wasn't necessary to talk in detail about ubicomp as an emergent property of the economics of CPU prices, that devices become intimately coupled with services, or that networks of smart things generate whole new universes of services.

The Internet of Things and Directories

One of the ideas that emerged in multiple presentations in conversations is for a device information brokerage and translation service. The idea is that a central service brings together information generated by all of these smart devices in a standard way and in a predictable location to facilitate mashups between various devices. Violet, tikitag, OpenSpime and Pachube, all of whom were represented, all essentially share this idea.

This got me thinking about other such systems I've seen, and I realized that I've seen this pattern before at least twice before: in Internet hostname resolution and P2P file sharing. For host resolution, before DNS there was HOSTS.TXT, a canonical file that stored all of the addresses of all the computers on the Internet. Eventually, this became untenable and the distributed system we know today was devised. In DNS there is no single central authority, but distributed authorities and protocols that extracts the canonical answer from a web of connections. Similarly, the P2P file sharing world started with Napster, which had a single server (or service) that knew where all the files were and redirected various queries in a top-down way. However, the deficiencies in that approach gave way to the distributed indexes of Gnutella. We're now seeing the same thing with BitTorrent, which relies on trackers to connect someone who has data and someone who wants it, but is moving a distributed trackerless model.

The unifying pattern here is:

1. Create a service that runs on a number of devices
   
2. Create a central phonebook so that those devices can find each other
   
3. Create a distributed phonebook that is as distributed as the devices it indexes
   

1. A crisis happens if the service is successful. The centralized server model becomes too resource-constrained (i.e. it's overloaded beyond a "reasonable" cost of upgrading).
   
2. This is a necessary evolution. Systems that start out with distributed indexes are significantly more complex than centralized ones. This requires a lot of implementation overhead on the part of the server and device designers and is brittle because it assumes feature priorities that may not match actual needs. In other words, if there isn't a perceived need, people don't want to write a bunch of code for scaling a service they aren't sure about. Moreover, it's not clear what the system should do, and initial assumptions are notoriously error-prone. I've seen a number of protocols that attempt to abstract a problem before it's clear what the problem is.
   

Google Protocol Buffers as a device communication standard

Finally, this made me think back to a discussion I started having with Bjoern, Tod and some of the other Sketching folks, which is the use of Google's new Protocol Buffers as a meta-protocol for devices to speak to each other. The point of Protocol Buffers is that they are simultaneously flexible and lightweight, which is valuable both when you're moving huge amounts of data around AND when you have very little processing power.

Google lists as advantages that:

• [Protocol buffers] are simpler
  
• are 3 to 10 times smaller
  
• are 20 to 100 times faster
  
• are less ambiguous
  
• generate data access classes that are easier to use programmatically
  

[Tangentially, "Protocol Buffers" is a terrible name; it is simultaneously generic and overly specific and I think that the name will significantly hurt its adoption.]

Posted by at 2:07 PM | Permalink

I have been busy for months on many projects, thus the infrequent updates. Not all of the things that kept me busy were as pleasant as presenting at Adaptive Path's UX Week. In the reception area, Tod and I showed WineM (which Bruce Sterling blogged about!), and I was invited to talk to the crowd.

It was a pleasure to be invited and to present to a group of old and new friends. I felt so comfortable that I decided to abandon my usual presentation format and present without any slides. I brought a suitcase of stuff and a stapled pile of notes, and pulled things out of the suitcase to illustrate my point. Kind of like a prop comedy version of a presentation, except not as funny. Or not as intentionally funny, anyway. However, as usual, I had written the presentation (108K PDF) in PowerPoint, so you can still read what I said, even though the slides are blank and you don't get to see me pulling a Motorola Dynatac phone out of a fake Prada purse. AP says they'll have video of it up soon.

My goal for the talk was to link many of the ideas I've been talking about lately (information shadows, objects as avatars of services, the shift to devices as subscriptions, etc.) into a single narrative, explaining why user experience design for ubiquitious computing is important, and why it's important to engage in it now. I spend a fair amount of time telling the audience to cast off the shackles of screen-based interaction design. I conclude with:

I believe that ubiquitous computing holds amazing promise for making the world a better, happier and more interesting place. One that’s a little more magical than what we have now. We, the interaction designers, have the responsibility to make it that better place. Manufacturers are going to make ubicomp devices with us or without us, and this technology is going to be the next big thing, simply because it’s so cheap. It is our job to make technology work for people, regardless of whether there’s a screen involved or not. I think that far too much attention has been paid to screen real estate speculation in the last 10 years, and it’s time to burst that bubble.

Also, Gregg Zachary of the New York Times wrote a very interesting story about the importance of hands-on engagement with technology. He quotes me in relation to the Sketching in Hardware conference.

Posted by at 3:26 PM | Permalink

Luxist reports on a new service to help track the provenance of wine. When Tod and I were at NextFest we spoke to some folks at Hitachi who had contemplated using their RFID technology to do the same thing, but just recently two new services have come online that are designed to track wine by the bottle (the other is Great Wall of Wine, a Chinese merchant, who are using RFIDs to stop counterfeit wine).

This seems to be an unnecessarily narrow use of the technology. Counterfeit wine is a problem, but (in my understanding) it's primarily it's a problem with a few very old, very expensive bottles. Old wine is not going to have RFID stickers on it, and if one is applied in such a was as to not damage the expensive bottle, it'll probably be easily moved to another bottle. Moreover, the business plan (as I understand it) is dependent on Metcalfe's Law network effects to work. In other words, it becomes useful if lots of wine has the stickers (the old "if only you have a fax machine, then faxing isn't a useful technology" argument), but initially very little wine is going to have stickers on it. For systems like that to be successful, the technology needs to be useful to just a single user. That's exactly the situation where a rich information shadow becomes valuable, and if you excuse the self-promotion, that's why we made WineM focused on the richness of wine's information shadow rather than just a single application. But maybe I'm misunderstanding the business model. Maybe the people who really need these systems are not end consumers, but dealers, distributors and regulators. Regardless, between these projects, Smartcorq and the Queen's ISETAN experiment, it looks like wine and RFID are going to be mated at some point in the very near future. The question is whether the power of all of the information about the wine will be exposed to end users (to everyone's great benefit, in my opinion) or whether the technology will remain stuck in the realms of logistics and security.

Posted by at 9:19 PM | Permalink

Over the years there have been many projects that use mobile phones to associate physical objects with their information shadows (YellowArrow, QR Code, ScanLife, etc.). There have also been many projects that use phones in social setting as a way for people to find out about the people around them (Nokia Sensor, Dodgeball, etc.), usually with the promise of dating. None have been particularly successful, in either category (though I gather that QR Coded stuff is pretty common in Japan).

Now comes a company that's trying to combine the two unsuccessful ideas to make one successful one. Wickd is betting that there will be enough people who want to snap shotcode barcodes on t-shirts to find out about someone, and that there will be enough people who wants to wear those (kinda expensive, kinda plain) t-shirts to create a critical mass of urban singles willing to pay for the privilege to make their business model work. I'm pretty dubious of the underlying interaction model (it's more related to first person shooters--where you run up to a target, fire your weapon while they're not looking, and run away--than dating, where you get close, interact face-to-face and ideally stay close), but for me it's another example that people are the most popular object to unify with its information shadow. Having information about the people with you at brunch or at a conference (which is where an adult version of Disney Clickables could be very valuable), and having your intersection in physical space mapped to social networking space opens up huge possibilities for maintaining your social network.

All of these projects point to the inevitability of that unification, it's just that no one has found the right vehicle to move the penetration of the technology to the appropriate place on the Metcalfe's Law curve.

Oh, and two final words of advice to the Wickd people: Hanky Code. Think about it. ;-)

[UPDATE: Several people suggested I should be more clear about why I think that this is a questionable idea. OK, here it is: I think that no woman in her right mind wants to wear a shirt that gives random people behind her back personal information. It's not the quality of the information or the content, it's the coupling of that with anonymity and immediate physical presence.]

Posted by at 11:00 AM | Permalink

There are two projects I've become aware of recently that represent the explicit linking of physical objects to their information shadows, both in children's products. This kind of thing has existed before, but its prevalence seems to be on the rise.

The first (found by Liz) is WebKinz which are plush toys that each have a unique code on their tag that brings up a unique play space that's just for that specific toy (randomly generated of course). The idea is "Beanie Babies meets NeoPets meets Cabbage Patch Kids" and although I think the execution of the concept leaves much to be desired (why buy furniture for your toy basset's online "room"?), it's an interesting example of how toy companies are merging offline and online conceptual play spaces in a very direct way.

The second is Disney's Clickables, which I learned about from CNET's Matter/Antimatter blog.

Clickables that we are launching in connection to our new Disney Fairies virtual world. It's a way for kids to take their online world experience into the real world. The core of it is a magical bracelet. By simply clicking their bracelets together, girls become friends in the online environment.

From the press release:

“The future of toys is about connecting online and offline play,” said Chris Heatherly, vice president of technology and innovation, Disney Consumer Products. “Kids and tweens are quickly embracing virtual worlds and, while there are several Internet-related toys in the market today, the play ends when the computer gets shut down. With our new line of Disney Fairies toys featuring Clickables technology, we're bringing the fun of social networking, collecting, and trading into the real world so that girls can extend the fun of the enchanting online world of Pixie Hollow to school, the park, or wherever they may be.”

This system of course owes a lot to Ruth Kikin-Gil's Buddy Beads project in terms of its use of jewelry to communicate social relationship between BFFs, but it also explicitly links the online world to the physical world using magic as a metaphor. It's not surprising that it's coming from Disney thematically, but what's interesting to me is how much Disney is investing in it. This is a sizable product rollout, which typically means that they have done enough research to believe that it'll be successful on their terms, which typically means hundreds of thousands, if not millions of customers. It's a project, and a genre, to watch.

Posted by at 6:19 PM | Permalink

Here here's your latest computer fridge news: Whirlpool has partnered with a domestic groupware software company called Cozi. Right now, it's just a branding partnership with Cozi's calendar/to-do list/grocery list etc. software for families, but it's clear where this is going: WP is going to create an embedded version of Cozi's software for their centralpark fridge line and then create other ways to connect to the same service. First it's the fridge, then it'll be an iPhone widget, and if it's a hit, a "household activity dashboard" on Mom's desk at the office, like what Ambient devices has done with some data feeds. Or at least that's the hope.

Electronic household organization tools has been around a long time (I took a half-hearted stab at it a couple of years ago). Not counting pre-Cambrian kitchen computer technology, getting into kitchens was an early goal of the first wave of Internet appliances in the late 90s. 3COM's Audrey, one of the classic failures of this first wave, advertises that it "can be the family's nerve-center in no time, handling schedules, phone books, and notes." Cozi's pitch is similar: "Cozi helps busy families manage schedules, appointments, shopping and communications from wherever you are — the kitchen, car, office or even the grocery store."

Timing is critical in technology adoption so there's no reason why these technologies can't work now when they failed 8 years ago. Many people who in WP/Cozi's likely core audience of affluent 30-something new home buyers are probably thinking much more about their families now than they were 8 years ago, because they probably did have them then. However, the repeated failure of the idea is something to learn from and I hope that Cozi has been studying people's habits and the pattern of earlier similar technologies to see why they didn't work out. Is it purely because the value of the service versus the cost isn't great enough (i.e. dry erase boards are cheaper and more flexible, but don't allow you to check your kids' schedule from the road, but that's OK with most people) or is there something deeper? I'll be interested to see where this goes.

Posted by at 9:40 AM | Permalink

Last night I presented a version of the Sketching Smart Things talk I gave last month at CHIFOO to BayCHI. It was an honor to be invited to speak there because BayCHI is such an institution in the HCI world and because the talk was in the PARC auditorium, feet from where the core concepts of ubiquitous computing were first formulated. Thank you, BayCHI and Rashmi!

The presentation is available on Slideshare:

| View | Upload your own

And as a 710K PDF where you can see a complete transcript of my talk in the notes.

Posted by at 9:00 PM | Permalink

The title implies more mean than this blog post will have in it, but in my research on digital rings, I discovered an amusing factoid: the first people to have worn digital jewelry on a regular basis are the children of the public schools in Celebration, Florida, the New Urbanist Disney planned community.

In this article, from 1999. Here are some interesting pieces:

Residents like to call Celebration a "perfect town," but even perfection needs security. That need has led to use of access control at Celebration School, a K-12 county-run institution.

[...]

All 930 students move freely around the campus through locked doors using a new system of Java-enabled Schlage Primus industrial door locks, modified by Lares Technology Inc., San Antonio, Texas, to work with the Java computer application. Each student is issued a Java Ring, built by Dallas Semiconductor, Dallas.

[...]

Celebration conducted the initial trial for the system when they equipped 100 middle school students with the iButton in a Java Ring, said Mori. "We wanted to see if the rings were of benefit to students. Now, it's a wearable accessory for the kids. We found that for some kids the rings work very well, but some wanted a key fob or a watch. In the beginning everyone was given a ring, but now they have the option of purchasing a watch or a key fob.

Without reading too much into suburban perfection requiring an incredibly technologically sophisticated security and monitoring system, it's interesting that the first sizable deployment of smart jewelry will have turned out to be for a bunch of 13 year-olds in a Disney planned community.

Posted by at 11:11 AM | Permalink

Hideaki Matsui's ring-based concept made the blog rounds this week, and it's only the latest of a trend of ring-shaped ubicomp devices (as helpfully cataloged by Yanko Design):

Right now they replicate simple functions that may be done better by other technologies (Matsui's design, for example, is very close to IBM's Personal Area Network from 1996, but embodied in a ring). However, that's not the point. What's interesting to me is that people have started to think about the capabilities of rings as a form factor for the development of devices. It no longer seems far fetched to design functionality in this way. As with many objects that are a product of their time, the idea is reinvented seemingly simultaneously in multiple locations as culture churns through the possibilities. That process generally predicts the actual development of an actually useful product (followed by a lot of people grousing that htye thought of it first). That's exciting because it means that an actually useful enchanted ring device may be forthcoming.

This also tangentially reminds me of NTTDoCoMo's FingerWhisper bone conduction phone speaker project (which was supplanted in the market by Bluetooth headsets, but had lots of potential), and I predict it's only a matter of time before someone comes out with a cell phone concept that's a ring or a bracelet, or both.

Posted by at 2:50 PM | Permalink

Doing some research for my upcoming CHIFOO presentation, I realized that there have been a number of attempts at merging computers and refrigerators. Here's a timeline:

1998: The V-sync "Internet Refrigerator"

"With a speedy Pentium II microprocessor and huge hard drive, it packs more computing power than most home PCs, and has separate compartments for fruit and vegetables."

1999: The Electrolux Screenfridge

"Electrolux earlier this year unveiled the Screenfridge, a connected refrigerator designed to allow users to order groceries over the Internet, but the product has yet to ship."

2000: Whirlpool/Cisco fridge

"While the Whirlpool refrigerator won't cook an omelet, it does have an integrated Web-browser to search for recipes that match the food items people have on hand. In case you have no idea how to make an omelet, you can prepare the meal by watching a celebrity chef on the Web pad."

2002: Whirlpool's Connected refrigerator

"Whirlpool's refrigerator transforms into a multimedia communications centre. The owner can surf the Internet, receive e-mails, listen to the radio, watch TV, videos and DVDs and even talk on the phone."

2003: LG's Digital Multimedia Side-By-Side Fridge Freezer with LCD Display

"It's the ultimate in kitchen technology with a built-in MP3 player for downloading and playing music from the internet, e-mail and video mail using a built-in camera and microphone. It even has full internet access so you can re-stock the refrigerator on-line or check on the latest news and weather - all without leaving the kitchen. And it's great for storing food too."

2006: Electrolux Screenfridge (again)

"The Screen Fridge is connected to broadband and TV via wireless connection. 15" touch screen and pop-up keyboard. As if Internet, email, phone, radio and MP3 player are not enough, Electrolux adds highly advanced calendar and video messaging system so the kitchen truly becomes the center hub in your house."

2007: Whirlpool centralpark

"Custom choices will include satellite radio, a Web tablet with interactive message board and family calendar, a digital picture frame, a DVD/CD player and more."

I like to watch how companies try the same idea over and over and how the ideas evolve. Initially, the computer fridges were just tablet PCs stuck into the door of a conventional refrigerator. Why were they there? Who was going to be using it? How were they going to be used? No clue. And sure you could do all the stuff they advertised (listen to music, make a phone call) because you could do anything you could do on a laptop. You could compose a Powerpoint or write software, too, but you wouldn't do that, or anything else, because there was no clear reason for it. The products quickly disappeared with the end of the first dotcom boom.

Then a resurgence happened. What was different? I think that companies, helped by the staff user researchers they hired in the interim, started to realize that it wasn't the computer that was important, but what people did with it. Not until the most recent Whirlpool offering does the idea of a computer fridge disappear entirely to be replaced by a series of functions that various modules can do. Each module is, of course, a computer, and every module can probably do the same functions as the other modules from a computational perspective. But that's not the point. The point is that the modules have different interfaces. They're different tools. Focused tools. Tools where the design uses a computer to help the user accomplish a task, just as they use waterproof plastic for the buttons and stainless steel for the shell.

[4/20/08 UPDATE: I noticed that in the proceedings to the 2008 Internet of Things conference there's a paper by Matthias Rothensee called "User Acceptance of the Intelligent Fridge: Empirical Results from a Simulation." His conclusions, from the abstract, are "It was found that generally a smart fridge is evaluated as moderately useful, easy to use and people would tend to buy it, if it was already available. Emotional responses differed between the assistance functions. Displaying information on durability of products, as well as giving feedback on nutrition health and economics are the most appreciated applications." That sounds like, not surprisingly based on the market response, faint praise at best, but I haven't read the paper yet.]

Posted by at 7:24 PM | Permalink

Yesterday I had the pleasure and honor of speaking at the Information Architecture Institute's IDEA Conference in New York. I got to share the stage with David Rose, CEO of Ambient Devices, probably the most pioneering consumer ubicomp company. The title of our session (as chosen by Peter Merholz, who curated the conference) was "Digital IA in a Physical World." David spoke about Ambient Devices' history from the Ambient Orb (well, actually, from his childhood home) to their current products, and how they've changed their vision since they began. I spoke about how ThingM developed our smart wine rack, and the larger context in which we do our work.

Speaking to an audience of information architects, I really wanted to emphasize what I feel is the fundamental change that information processing goes through when it becomes ubiquitous. One of the ways I've been discussing this transformation in the last couple of years is by talking about information as a material. I reiterated that argument for the IDEA audience:

[...] embedded information processing and networking starts behaving like a material.

Let me explain. When a product designer can include information processing in a product for very little cost, the calculation becomes not one of engineering complexity, that’s relatively cheap, but one of competitive advantage. Including a CPU becomes a line item in the competitive analysis of making an object, just like the calculation about what to make it out of. Of course, as any brand new material, adoption doesn’t come all at once, it trickles in first in one industry, then another. Think of nylon going from being a molding material in 1941, to panty hose, to cogs in sewing machines by the 1960s. You can already see it happening with information processing. All kinds of toys now depend not just on their physical appearance, but behavior created by electronics, for their competitive advantage.

I also described why we chose to augment existing objects, rather than creating wholly new devices. For me, this is because existing objects already have highly-developed information shadows that we aren't fully taking advantage of and that there's still much potential in exploring augmented everyday objects:

Every material object casts an information shadow. It exists simultaneously in the physical world and in the world of data. That information shadow has a life of its own. Sometimes that life is pretty simple. But it can also be complex, maybe as rich as the physical object’s life. Like Frank [Lanz of area/codesaid yesterday, the border between the real and the virtual is becoming more porous. This is also where IA comes in: the information architecture of these shadows IS the information architecture of objects. With this project we wanted to unify the informational and physical aspects of a common object. We looked around for objects with rich informational lives, and after discussion about books, clothes, and cars, we settled on wine.

In the rest of the talk, I describe some of the design challenges we faced in creating the WineM and how we solved them. You can get the whole presentation(684K PDF) with my notes.

Posted by at 8:55 PM | Permalink

[I wrote this for the ThingM newsletter that went out yesterday, but thought it may be of broader interest]

Wine keeps reappearing at the intersection of the digital world and the physical one. Bruce Sterling's pioneering book on the implications of ubiquitous computing, Shaping Things, uses it extensively as an example, but he wasn't the first to discuss it. Wine is the textbook example (literally) in the Information Architecture world, where the problem of organizing is often used to explain an approach known as faceted classification. Virtual Vineyard (arguably the first successful ecommerce website) launched before Amazon did.

Why? Our theory is that wine exists in two worlds: as a physical object and as an informational one. The informational object doesn't just exist as a way to help people select wine to drink, but the information about the wine becomes an important part of the process of collecting wine. Moreover, unlike other collectibles that exist as physical and informational objects (think Magic the Gathering cards), wine is a consumable. You can never get a complete set and what you have is always shrinking, so there's a perpetual pressure to gather new information to gather new wine.

The problem is that wine bottles are terribly difficult to track. As collectibles, there are market pressures to create scarcity, which leads many producers (especially of high-end wines) to avoid using the most common object tracking mechanism, the UPC barcode. Barcodes symbolize mass production to wine producers struggling to create scarcity, so they don't use them, or use them haphazardly. We feel this ends up backfiring on wine producers, creating obscurity instead. Wine is a classic Long Tail product: in other words, there's a huge volume of potential in the obscure end of the market, but despite wine's early entry as objects of cutting-edge technological consideration, it hasn't achieved nearly its potential.

We believe the core problem is that most wine is virtually untrackable in the information space. It's a physical object that has no anchor to which to attach data. There is huge potential in creating such anchors. Ulla-Maaria Mutanen created the Thinglink project to create "social objects" that "make it possible to 1) aggregate online discussion around particular items, 2) track their history, and 3) develop new ways of connecting through particular objects on the web." She's talking about handicrafts, but the same thing can--and should--apply to wine.

However, when I went to the Wine Industry Technology Symposium a couple of weeks ago, there was virtually no discussion of ideas like this, even as the group discussed the power of "Web 2.0" and social networks.

Since we're currently working on an RFID wine rack, we're thinking a lot about these issues. We would like the answer to be RFIDs embedded in wine labels (invisibly) coupled with open, shared communication standards for exchanging wine information. These should look forward toward the capabilities of the technology and the "social life" of objects that bridge the information and physical worlds, rather than trying to copy UPCs or ISBNs, as valuable as those have been. Until then, wine, that most textbook example of hybrid objects, will be frustratingly out of reach for consumers, who then will be themselves frustratingly unavailable to producers. It's a situation that could be much better (i.e. profitable and enjoyable) for everyone involved.

Posted by at 10:18 AM | Permalink

A couple of months back, receiver magazine, Vodaphone's magazine about art, society and technology, asked me to write a short piece for them. I decided to write about the evolution of appliances. As with many of my recent articles, it starts with the history of post-WWII American society and how the values, technologies and social structures of that time created the basic framework that allows for ubiquitous computing to appear. Here's an excerpt:

1947 was a big year. That year, Bell Labs invented the transistor and Levittown, New York, the first modern American suburb and the model for most others to come, opened for business. 1947 was not only the beginning of the Baby Boom, but of a whole new lifestyle of electronic home appliances.

[...]

With [semiconductor] prices so low, including information processing in a product becomes less an exotic research project and more a competitive calculation akin to selecting plastic over rubber or aluminum over steel. Manufacturers will likely soon begin to use information from the domestic environment in an effort to make appliances more effective and more attractive to buyers. Some will even succeed.

[...]

This means that our everyday domestic devices will soon change. Hybrid devices, "smart things", have already begun to appear and will continue to do so, blurring the lines between furniture, tool, computer and robot.

[etc.]

You can find the whole article on the receiver site. I also highly recommend the rest of the magazine, which features articles by danah boyd, Louise Barkhuus and John Seely Brown talking about technology and the home.

Also, Tamara Adlin has posted an interview with me on her UX Pioneers site. I'm flattered and grateful to be in such distinguished company. It's probably the most coherent explanation of my career and motivations, ever. THANK YOU, Tamara!

Posted by at 4:40 PM | Permalink

I had the great privilege of speaking at the Taste3 conference on wine, food and art in Napa today. This is a terrific conference that's run as a kind of "TED for food" by many of the folks who organized TED years ago, and with the same ultra-high quality of experience design and attendees. I've had a number of great conversations with really smart, successful and fun people.

My talk today was a somewhat speculative presentation tracing some of the history of appliance design and how ubiquitous computing may change that. In the talk, I present the history of blender controls as an example of the encapsulation of knowledge into our tools. I then show several examples of how networked kitchen devices may (or may not) present a fundamental shift in the nature of how we relate to our kitchen tools:

Imagine that every time you used this [networked, barcode-reading microwave], it quietly told a database somewhere--say, in your iPod--how many calories you just ate. Then your iPod could query your shoes about how much you had run the previous day. The next time you went for a run, your iPod would pick songs with a different tempo to encourage you burn off that Mac and Cheese. Now that’s starting to get interesting. It is now possible for our tools to automatically encapsulate knowledge and share it with each other.

The full presentation is available here (360K PPT).

Posted by at 6:14 PM | Permalink

Steve sent me a link to the Interactivos? workshop at Media Lab Madrid. I'm sad I wasn't able to propose for it and won't be able to attend. The theme is "Magic and Technology" (which everyone knows I'm a big fan of ;-). Their introduction reads:

Magic and illusion have always gone hand in hand with technology; from mechanical illusions, optical and mirror tricks, through the incorporation of electricity and the filmed image, to digital technology: augmented reality, reactive objects, reality hacking and immersive spaces.

This new edition of Interactivos? in Medialab Madrid is inspired by the strategies of magic and illusion, in order to harness some of the old and new technological resources to collectively build software pieces and interactive installations which can propose a rethinking of the usual scenario in magic tricks, marked by a very clear separation between the wizard and the spectators.

[...]

The call is focused on projects of digital and sound art, critical design, educational applications, etc., which, inspired in magic and illusionism techniques, propose experiments on perception and attention, behaviour and interaction generated by social relations. The call is also focused on projects inscribed within the open hardware and software philosophy.

I'm happy to see that they're taking the social perspective and talking about breaking down the barriers between spectator and performer, owners and observers, adapts and novices.

I also really like their focus on "artists, wizards, engineers, musicians, programmers, designers, architects, and hackers." Explicitly crossing the barriers of designers, programmers and artists is critical right now as the field is maturing. In the beginning of film theater magicians were major drivers of innovation. This was not because they were magicians, but because they were applied creative professionals who had an immediate financial stake in the innovations--this differentiated them from scientists or artists, and makes them closer in terms of motivation to today's designers.

Posted by at 4:29 PM | Permalink

This morning I gave a keynote at O'Reilly's Etech. It was an elaboration on the theme of magic in the design of ubiquitous computing user experience that I've been developing for a while now.

The core of the piece were three linked arguments about emergence:

• The emergence of ubiquitous computing from market forces acting on, and in concert with, CPU prices
  
• The emergence of animist reactions to devices that have behaviors that go beyond action-reaction physics
  
• The emergence of magic as a metaphor for the design of ubicomp devices
  

I've made the presentation (710K PDF) with all of my speaking notes available.

Posted by at 11:46 AM | Permalink

(photos (cc) by eecue and decade_null, found on Flickr)

A couple of years ago I wrote about an idea I had for visualizing the implicit heat maps in Wifi signal strength using actual heat.

I never made the device, but I thought about what the ideas it was pointing at and generalized this as an observation I called "new data for old senses" and wrote some notes about it that I never shared here. Today PT at Makezine posted a link to a project along the lines that I was thinking about. It's a Wifi sensor that uses vibration to give you a sense of the Wifi landscape around you without having to look at anything, which was the crux of my idea in 2005. So, since the idea is now out there, here are my notes:

-----
I'm interested in the idea of using senses that don't normally get used for device communication as secondary display channels. This is a way to allow access to what John Udell calls the vast middle ground between devices that either demand our full attention or none.

We have more senses than sight and sound, which are channels already full with important information, so how do we use our "secondary" senses to communicate "secondary" information?

What are other kinds of senses and other kinds of data we can use?

Here are the somatic senses (thanks, Google!):

• touch
  
• pressure
  
• vibration
  
• heat
  
• cold
  
• pain
  
• proprioception (the feeling of joint movement)
  

What to visualize? Liz has been doing a bunch of stuff about visualizing people's relationship with the RF spectrum and geography, but I've been thinking that there are several granularities that would change in perceptible and interesting ways. At human scale in a city there's Wifi strength; at car scale there are things like crime maps, and at airplane scale there are political boundaries (voting records, natural phenomena).

-----

The bottom line is:

How we can introduce secondary information into people's awareness in a secondary way, using their less-used senses and without adding additional cognitive noise to the primary channels of sight and sound?

Posted by at 7:33 PM | Permalink | Comments (1)

I don't advocate the implicit weirdness of such a device, since I can't think of any case where using it doesn't imply some seriously screwed-up priorities, but this lamp that hides a secret GSM camera is an interesting example of fusion between technology and furniture. It uses a mobile phone and camera embedded, and concealed, in a lamp.

Room monitoring has gone to a new level. Simply dial into this unit and the sensitive microphone will secretly be activated so that you can hear everything that is going on within 15 to 20 metres of the lamp. Unlimited range.

As an example of technology integrated into everyday life, it's an interesting data point (if only for the surprisingly high price). At least for me, however, the ethical considerations greatly outweigh its utility.

(Originally seen on BornRich)

Posted by at 6:32 PM | Permalink

Although the event at which the technology is described appears to be have happened some months ago, I just saw this video on YouTube about a Philips project called "Drag & Draw." It appears to be a projector and a motion-tracking camera that's tuned to specific colors in an RGB LED pointer. The colors are selected using a "paint bucket" of glowing LEDs. It seems to be a small-scale version of the Graffiti Research Lab's LASER Tag project (though predating it, the GRL project got more media coverage recently), and not unlike Golan Levin's many projects.

Philips describes that with Drag & Draw, "the entire home becomes a virtual canvas for expression and play for young children, thanks to a magic brush, a magic eraser, a magic wand, and a laser projection bucket." That's a lot of magic.

In the YouTube video they go further, "You can cast magic onto the drawing and it will come to life." Their video scenario is fanciful with how this animation works, but as Levin shows, there are many precedents for this kind of static-to-dynamic drawing.

Of course I find it interesting that they're using magic as a metaphor to describe functionality, but I suspect that this is primarily to explain it to themselves and to other adults. If their audience is genuinely small children (and why not people who need a white board replacement?), the kids probably don't need the explanation that it's magic. It just is, it works the way it works. How does glue work? Does it matter when you're making a collage? Same here.

Also, speaking of wands, in the same show Philips showed the "uWand" that they describe as:

a Philips vision of a revolutionary way of accessing and managing content, enabling the user to interact with their digital environment in a natural and self-expressive way. A simple stroke of the uWand allows users to intuitively point at a device and to scroll, select, play and move elements.

This is the first pairing of the "u" prefix (which typically implies "ubiquitous" in the same way that "i" originally implied "Internet" and now implies something like "interactive" or "me") and "Wand." Magic and ubiquitous computing. From Philips, no less.

Posted by at 6:42 PM | Permalink | Comments (1)

Last week, in honor of Valentine's Day, Tod and I put together another ThingM Technology Sketch, LoveM. It was a result of a process we regularly use: we look at a technology and apply "what if" metrics to it. "What if this technology cost 1/10 as much?" "What if it was 10 times more powerful?" "What if we could make one of these in 1/10 the time?" That sort of thing.

One of these discussions--"what if LCD screens continued to drop in price?"--led to LoveM. It's a chocolate box with an LCD display. When we came up with the idea, it started as kind of a joke, but then we went to a grocery store display to look at chocolate boxes and there was a $4 box of Valentine's Day candy with an LED blinker on the front. Sure, one LED does not make an LCD video display, but LED technology has existed for a long time and at current prices a chocolate box with an LED could have been done five years ago. It wasn't. I suspect because people weren't ready to have that level of technology in their food. That box and the relationship it implied made it clear to us that it was a lot more acceptable to introduce technology to food, and led us to pursue the idea and make a concept video (and give the box a cameo in it).

There's a more elaborate description on the ThingM site, but here's the video:

Posted by at 11:17 AM | Permalink

I started keeping track of interesting developments in furniture design recently. I'm primarily focused on the integration of furniture and technology (call it "smart furniture" ;-). Most of these have been collected from the gadget blogs over the last couple of months, so you may see familiar things if you follow those.

Massage Chairs

I had just noticed that massage chairs are domestic robots when Matsushita had to recall 68K of them. Now what this means is that there were at 68K massage chairs in the world. That's fewer than the number of Roombas or iPods, but that's a lot of chair-shaped domestic robots, and Panasonic is only one of dozens of brands producing the things.

EL Couch

Designer Danielle Sobik has made a prototype reactive electroluminescent couch. I think that her narrative that it's designed to bring couples apart by providing glowing color feedback that they're sitting too far apart is a bit simplistic (OK, so it works once, what's the daily utility) and, frankly, unnecessary: why not just have a couch that glows when people sit on it, and the glow changes depending on the orientation. I think it's also interesting that her goal seems to make the modernist design esthetic of the couch form she's using "more personal." This is a theme that seems to be reappearing recently, and it makes sense to me (but, then again, I'm no fan of Modernist minimalism).

Herman Miller includes charging into desktops

Induction based battery charging (most frequently seen in cordless electric toothbrushes these days) has been one of those technologies that has been possible for a long time, but practical maybe only recently (and maybe not now). Herman Miller's licensing of the technology, and including it into their portfolio of products makes it distinctly more possible. It also creates an interesting technology lock-in problem. This may be the first technology H-M has installed that's tuned to a specific set of devices that the company doesn't make. As those devices evolve, how will H-M update this technology? More importantly, how will they reassure office managers that the desks they're buying today--the very, very expensive Herman Miller desks--won't be "obsolete" 2 years from? This may be the thin edge of the smart furniture wedge, since the drastically different lifecycles of furniture and technology are what have prevented deep inclusion of tech into furniture.

Whirlpool Centralpark

A similar idea is Whirlpool's new fridge/accessory charger. They're being explicit about the pieces being swappable, so therefore trying to make it "futureproof." I'd be interested to see if that'll work, or if the whole assumption that things need to be recharged is going to disappear in a couple of years with new battery technology and induction charging. But that's of course not how Whirlpool's customers purchase: they have iPods today and when comparing Whirlpool's fridge against the Subzero, any slight advantage helps. It'll be interesting to see how well this sells. It's also interesting, because it provides electronics-level power input into the face of a refrigerator, which not only opens up all kinds of hacking possibilities, but also expansion possibilities. The classic "stick an LCD display into the front of your fridge" idea can't be far behind (though probably no one will buy it, as they didn't before).

Finally, a couple of design-oriented technology notes. Exploring the integration of technology into everyday life is the leitmotif of this blog, and the other side of installing technology into everyday objects is making technological objects that already exist better integrated. To that note, here are two fascinating recent examples:

Suck UK's fabric clock

An excellent melding of DIY, ironic anti-Modern design, historical reference (think 1960s tabletop radios) and technological camouflage, a simple idea for a 30-year-old technology and a great use of LEDs. Go Suck UK, go.

Samsung's blinged-out washer

Again, I'm no fan of cold Modernist minimalism. My reasons are many, but--among other things--it no longer make sense to treat the products of technology as if they were from a scientific lab, even when a lot of science goes into making them. That level of communication is no longer necessary, and we've grown used to tech. So much so, that casemodders, the hotrodders of our age, are the vanguard of technology personalization and expression. They show us that bling is a core part of our lives, or it should be. In our hearts, we all bling. Rimless architect glasses and Eames furniture are still bling, they just come from a different social background than gilt and Baroque curlicues. I think it's about time that appliance makers recognized and embraced the latter as a legitimate design style, which is why I'm so happy to see this washing machine. More bling!

Posted by at 5:52 PM | Permalink

I've talked about video scenarios and sketching before. Well, now that ThingM is up and rolling, i figured it was time to practice what I was preaching. Today we put the first of our Technology Sketches, which are video scenarios we use in the rapid prototyping of interaction ideas. We did this one in a little more than 3 days, including all technology, design and video work, and learned a ton from the process. Here's the abstract:

WineM is a Technology Sketch of a smart wine rack. It's designed to locate wines in a wine rack using RFIDs attached to bottles and to display which wines have been located using LED backlights behind the bottles. Collectors (or anyone with a large wine cellar) can use it to search through collections, track the location of specific bottles and manage inventory with a minimum of data entry. Linking bottles to networked databases can provide information that would otherwise be too time consuming or difficult to obtain (for example, the total value of a collection, or all the wine that is ready to drink).

A full description of the ideas and technologies behind this sketch are available on the ThingM Site.

Posted by at 2:10 PM | Permalink | Comments (2)

A group in Switzerland has been doing some interesting experiments with technology embedded in everyday objects that helps people use those objects. Two of their papers were mentioned on Engadget and I enjoyed what they had to say. One paper, Instructions immersed into the real world–How your Furniture can teach you (160K PDF) was presented at Ubicomp 2003; the other, Towards Situation-Aware Affordances: An Experimental Study (240K PDF) was presented at Pervasive 2004. Though their field is giving instructions in the everyday environment, their platform is flatpack furniture (aka IKEA):

If the user takes a wrong action, a red light pattern appears reporting a mistake. Additionally, a green flash pattern shows the right alternative. After boards have been aligned together in the right way, individual green lights direct user’s attention to the holes where the screws have to be inserted and tightened. Once the final assembly state is reached, synchronous flash patterns on all LED’s indicate that the task is finished.

Their results present some comparative experiments with the LED-augmented shelves versus the standard instructions. Their results pretty uniformly favor the augmented version:

[...] there is a measurable time gain when using LED based instructions. [...] errors during assembly can be reduced using instructions in the right place. [...] determining which part fits where is one of the main problems using today’s instructions. [...] 75% of the participants found that the LED based instructions help with exactly this problem.

They also provide a good comparison (in the 2003 paper) of embedded technology versus other kinds of technological approaches:

[Augmented Reality] is cumbersome and typically computationally expensive. Audible instructions offer a cheaper way of immersion but have to tackle with the problem of addressing the appropriate parts by a vocabulary the user is familiar with or has to learn before. There is the possibility of presenting information on a screen [...]. However, the integration of instructions with the task remains unsolved.

Of course it's unlikely IKEA's margins will allow it include this kind of technology in their furniture in the future, but it's an interesting example of how cheap (relatively, by hardware standards) can be used to augment everyday tasks and how furniture can be the conduit of that. I'd also be interested in some exploration of what the embedded technology can do in the furniture after it's been assembled. I've posted about smart bookshelves before and ThingM will be posting about them again in the near future, so this is particularly timely.

Posted by at 4:06 PM | Permalink

Researchers and designers at Aachen University and the Swiss Federal Institute of Technology have a magical user experience design project called REXplorer. As described in their paper entitled REXplorer: A Pervasive Spell-Casting Game for Tourists as Social Software,

REXplorer is a pervasive and mobile spell-casting game designed for tourists of Regensburg, Germany. The game platform blends location services, mobile photo and video blogging, and phonecam-based interactions to create a fantasy world that brings the history and culture of Regensburg to light. REXplorer applies mobile social software concepts to enhance the game and tourist experience.

They go on to describe the game:

The basic premise of the game is that the historic buildings of Regensburg have magical spirits, secrets, and treasures locked inside of them that can be unleashed with the proper magical spell.

Magic spells are cast with a wand that consists of a mobile phone in a plastic shell, which is running custom software that uses gestural recognition of the motion of the video from the phone's camera to identify certain basic movement shapes, which then invoke "ghosts" that talk about the city's history. The game aspect takes the magical metaphor to a greater extent than I've seen before:

[...] players hear voices from magical spirits trapped inside the buildings through their magic wand (the loudspeaker on the smartphone). If they cast the spell incorrectly, the spirits will be disgruntled and uncooperative. If they cast the spell correctly, the spirits will open up to the participants and divulge stories from the past that contain elements to help lead them on their path to master wizard. Participants may also need to duel against other participants in a spell-casting battle to achieve their goals.

It's a great, thought-through and exciting project. The use of magic as a interaction metaphor in a Medieval town as the backdrop is perfect. Judging from the video, the technical execution is also impressive. Great stuff.

I'll be updating my Magic in UX bibliography momentarily.

Technorati Tags: magic ubicomp ubiquitous+computing museum location augmented+reality smart+objects

Posted by at 2:04 PM | Permalink

I'm pleased to announce two exciting events, in the form of two websites.

First is the appearance of Tod Kurt's Hacking Roomba book, and its companion site, roombahacking.com. Tod is my friend and business partner, but (that notwithstanding) I still feel the book is still an exciting statement about technology and people's understanding and access to it (and an a fun way to get close to an otherwise difficult subject, robotics). Of the many books about hacking software, and homebrew hardware, this is one of the only ones that focuses on a single, cutting-edge appliance.
For me, this is an important milestone. For many years the trend in device design has been toward less end-user access to core functionality because the incentives for creating clarity about device functionality decreased in proportion to the increase in complexity and price competition. Cheap component prices mean there's little reason for developers to let people other than the manufacturers access the core of everyday technological objects. Even appliances, which designers once labored to make easily fixable, have become nearly disposable. I was recently surprised when I opened a 1970s blender and discovered that it was made to be easily repaired: the individual parts were clearly marked with part numbers and positioned to be easily replaced.
iRobot, whose company culture grew out of the MIT hacking tradition, designed the Roomba otherwise. They included a serial port on nearly every Roomba, and then they revealed the protocol for talking to that interface. Tod was one of the first to jump on this opportunity and wrote an excellent, broad ranging book about how to use this port (and the Roomba as a whole) to explore the possibility of personal robotics. He especially focused on uses outside the typical "robot fighting" genre that the majority of hobby robotics energy goes to. He feels that robots can do other things than push each other out of circles, flip each other over, or be the first to score a goal. His Roombas dance, draw and make music. As Tod puts it, the hacked Roomba can be a lover, not just a fighter.

The second announcement ( which I alluded to above) is that Tod and I have started a company, ThingM (pronounced "thingum" as opposed to "thing em"). We started this company close to a year ago, but we decided to put up a site to coincide with the launch of his Roomba book.
ThingM is a design and development studio focused exclusively on ubiquitous computing. We have many hopes for the company, but my dream is to rethink objects in the age of ubiquitous information processing. I believe that information processing can be considered a new kind of material in design (this is the basis of my Smart Furniture Manifesto, and furniture is one of the "object genres" that we have been studying), and that tangible networked objects can be considered a kind of projection of services, rather than mere standalone entities. At ThingM we aim to create a new class of smart everyday objects that abandon the idea of computers as general-purpose devices with a screen, a keyboards and a mouse. Our goal is to change the fundamental nature of all designed objects using pervasive networking and computing. In this, ThingM can be considered a combination of an interaction design studio, an industrial design studio, an engineering consultancy and a software development house, but really, we're a ubiquitous computing studio. Expect to hear more from us in the upcoming months.

Thanks to Judith Zissman, who helped us envision and produce these two sites (in addition to doing much of the site design and production, editing and basically everything else). Her contribution was instrumental, as it was with the Sketching in Hardware conference over the summer (the first ThingM product, if you look at the bottom ;-). Sonia Harris talked us (really me) through a bunch of overly-nebulous ideas and overly-specific requests to design the ThingM logo. Thank you!

[and thanks to Phil Torrone for blogging Tod's book on MAKE!]

Posted by at 8:25 PM | Permalink | Comments (2)

It looks like National (aka Matsushita/Panasonic) is launching a smart bed. It's a combination of a bed with a pressure-sensitive pad (roughly serving a similar duty to the sensors in Stanford's Sleepsmart project [120K PDF]) and an ambient environment that's designed for optimal sleep:

The Kaimin System offers an integrated control of lighting fixtures, reclining bed, air mattress, air conditioning and audio-visual equipment through a ‘pleasant sleep environment controller’ which is built in with programs to adjust the room environment for a quality sleep.

I'm really pleased to see this coming to pass, and I'm glad they're actually testing it with people before launching it to the world at large, though I wonder how much testing went into the effectiveness of the environment. Their description of the testing says it was "used to improve the system and key devices in various manners to make them more practical" which implies usability testing of the interfaces, but doesn't get at whether it actually helps people sleep. The outline of their "Standard Course" sleep program has a pretty rigid definition of sleep, and implies that behavior that deviates from any of their "Standard Course" will need to be hand selected or programmed, which relies on people to know more about their own sleep patterns and to do more work than I think most people are going to be willing to do. Still, it's an interesting development in the world world of smart furniture and I wish 'em luck (and I'd like to know what their 60 applied-for patents are and how they differ from Stanford's work).

Posted by at 3:42 PM | Permalink

Tod's latest hack allows you to control a Roomba using the Bluetooth radio in your mobile phone. The key is Mobile Processing:

Trying to develop J2ME (aka “JavaME”) applications for cell phones has been a mess, especially for non-Windows users. Thankfully, Mobile Processing wraps up the ugly details, like Processing does for normal Java. It makes writing little programs for your phone pretty easy, and makes whipping up a program to control a Roomba possible.

Processing continues to be a great project. Thank you Processing team and go Tod.

Also, Tod's definitive Roomba hacking book is now available for pre-order on Amazon:

It lands on November 20.

Posted by at 3:49 PM | Permalink

Ubicomp got a publicity boost this week courtesy of Vernor Vinge's commen ts on ubicomp, which have been linked widely since they were posted a couple of days. Much of what he says is right on (RFIDs, wireless networks, etc.) and has been discussed in the ubicomp world for a while, but he makes several interesting points and identifies (and names, in that race to name concepts) a couple of things I find new and interesting:

• Localizers are "a feature that is on networked embedded processors, whereby the processor knows where it is in 3D space." He then couples this idea with wireless networks, "If you know exactly where things are, not only can you make use of the ultra-wideband that we already are moving into, but you could even imagine using very good localizer technology to set up extremely high bit-rate lengths that were highly directional." This is in order to keep all data from moving over all networks, a kind of spatially-informed multicasting. I think his vision of things knowing their position in 3D is insightful, but I wonder if his idea of the elimination of whole industries coming because people no longer have to arrange stuff only scratches the surface. I also wonder whether the bandwidth requirements necessary for whole environments of objects to communicate and coordinate will overwhelm the system. It becomes the kind of problem that's likely to be solved only if there's a compelling commercial reason, and self-locating objects don't exactly have one yet. That said, the accelerometers in MacBooks and the mini gyroscopes in Wii mice represent a start toward object orientation as an efficient interaction technique, which definitely has a business case. Whether objects end up using that when talking to each other remains to be seen. I've noticed that the possibilities of object-to-object communication don't get nearly as exploited as object-to-human in personal technologies. Which is probably as it should be.
• Node guano. "If you have lot of ad hoc nodes, in a situation where nodes don’t last forever, ultimately we could be hip deep in dead nodes." I'm not sure I agree with this. There's certainly a lot of link rot on the Web, but ubicomp devices will probably require a lot more energy to maintain dead links (and on that note let me mix a couple terms from different discplines: what is the carbon footprint of all of those dead links on the Web?).
• High-resolution heads-up displays. Continual data overlays have been the stuff of science and science fiction even before the Terminator figured out to say "Fuck you, asshole" in 1984, but it's been really hard for that technology to take hold socially. Vinge's assertion that a high resolution heads-up display "destroys all other display technology" understates the value of shared viewing. TV glasses have existed for a good decade, but the display technology that's really taken off in the home is the giant TV. In pair programming, even the space where people have traditionally been most likely to have one (or more) monitor per person becomes a shared display experience That objection notwithstanding, I think that his idea of consensual imaging among belief circles is interesting. I consider it a kind of physical manifestation of software skinning, mixed with ideas shared among members of a social-network (as a blogroll is, for example). The implications of this both excite and scare me: it would be totally cool to overlay a trusted source's view of a given scene on mine, but I feel people already ignore the complexity of reality too much and tend to live on parallel planes that exclude ideas that challenge theirs. I don't want Orrin Hatch's world skin (though I'd try it on to see what it looks like), and I don't think he wants mine.
• Vinge starts to touch on my current favorite topic, magic as metaphor in interaction design for ubiquitous computing. "[...] Virtually every aspect of purpose, faith and fantasy could have a constituency in such a world, and that really raises a lot of possiblities for products, and the products actually go beyond games. [...] It’s not so much a question of the place of games in the future world, but a question of whether there’s anything going on besides games." He's talking about games, and I feel the kinds of games he's talking about are augmented reality fantasy games. Fantasy is explicitly not an explanatory framework for understanding the world, as genuine animism or belief in magic are, but it's a close neighbor. It's a way of trying to see how things could be if some fundamental rules were different.
• His conclusions on how these technologies shape his version of his technological singularity I find particularly interesting. His implication is that when there's all of the bandwidth available and all of these instrumented objects, it will change society so that people will join groups (he calls them lifestyle cults) that will allow for mass leverage of their ideas economically and socially. This implies a kind of mass-mind that one either joins or is helpless to defend against as a single person. Not to reduce his argument but: 1. hasn't that already happened on the level of media consumption, which in turn drives mass behavior? People no longer have to get bad news if they don't want to (well, not exactly). and 2. Doesn't that overestimate the homogeneity of people's desires? As a social researcher, I am continuously surprised at both the similarity of people on certain levels (demographically-determined audiences are sometimes frighteningly similar) while being perenially surprisingly at others (people always surprise me, even when I think I know "their group" very well). In other words, I think his conclusions have already come to pass and we're all OK: the Singularity happened and we feel fine. Maybe I'm too optimistic.

Finally, Brenda Laurel is talking about animism at the closing keynote of Ubicomp 2006 next week, bookending Bruce Sterling's opening. I'm glad that animism is finally making it to mainstream thought about people's attitudes toward user experience design for ubicomp. I feel vindicated enough to include a gratuitous reference to the piece I wrote about it three years ago. ;-) AND I'm pretty excited to see what Brenda has to say.

Posted by at 4:04 PM | Permalink

Video scenarios present people interacting with fictional technology by faking the actual functionality through the use of film techniques. I'm a big fan of the technique, since they can free designers from obsessing about the how of technology design and focus on the what, who and why. People's needs form the core of good video scenario, rather than technological capabilities. Once people's needs have been identified and interactions explored, the more rigorous work of technology development can start, but the idea of making little movies that demonstrate interaction ideas is really liberating. Plus, they're entertaining to make and watch, and the process of making can identify problems with interaction concepts that mere descriptions don't.

At the 2006 Milan Furniture Fair, back in April, I was particularly pleased to watch a set of video scenarios for the design of technology products created by students of the Kingston University product and furniture design program. They took the idea of detaching technology from interaction and ran with it, clearly stating that their concern was not with how their products did what they did, but that they did them "as if by magic".

The ideas that came out of this exercise are whimsical, interesting and highly creative: a rolling suitcase that follows you, a mug vacuums up spilled coffee, dust turns into bubbles that float to a specific person and pass on a message (and 28 others). Great stuff.

Posted by at 1:38 PM | Permalink

ISEA/ZeroOne San Jose ended yesterday. The artists, Anu (the producer), Liz and I took most of the C4F3 projects down last night. It's difficult to step back from a project that you've been involved in and evaluate it objectively, so I won't try. I had a great time and was proud and honored to be involved in the festival. I thought that the works in the C4F3 were great and I was amazed at how much effort the artists, jurors, production people, designers (and everyone else) put into the project. Thank you.

I didn't see everything in the festival. Of the things I was able to see, these five projects I saw and liked a lot. These are in no particular order and there are probably 20 others that I thought were great.

• Mission Eternity, etoy

  

  Etoy's Mission Eternity is one of the most coherent, ambitious and wholly-realized conceptual projects I've seen in a long time. The basic notion is to use the power of networked digital technology and inexpensive storage to keep aspects of us alive after we're dead. On one conceptual level, it externalizes the network of memories and documents we leave behind, and places them into a digital world, which is projected into the physical one as a shipping container sarcophagus filled. The sarcophagus is simultaneously a display, an environment and metaphor, and as it ages, etoy will replace the LEDs with the ashes of the people whose digital selves they manage. I think it's brilliant, deep and ambitious. Go etoy.
  

• Altitude Zero, Hu Jie Ming

  

  Tucked in one of the Container Culture shipping containers, Hu's piece is an excellent, simple and subtle interactive experience. As you approach the simulated portholes, the view changes from a boat's view of one port city to another. The initial similarity of the views underscores the differences in the views and gives Hu an opportunity to quietly comment on each city and on the nature of living at the edge of the ocean, in general. The flotsam for each city is different (I think) and it's probably meaningful. However, I was mostly entertained by the pure joy of watching the portholes teleport me from one place to another.
  

• Nocturne, Colin Ives

  

  Again, simple interaction coupled with a clever visual trick makes for an elegant piece. In the fox piece, my favorite of the three that make up this piece, a video projector acts as a spotlight. A slide projector provides the "normal" view. The two are showing images that line up. We interrupt the slide projector beam as we approach the piece and cast a shadow, except the part with the spotlight. This moment creates some interesting surprise and draws attention to the spotlight. Ives wants us to look closer at our relationship with the things around us at night, and this projector relationship (for me) really forces that, simply and directly.
  

• Bioteknica Lab Remix, Shawn Bailey, Oron Catts, Jennifer Willet and Ionat Zurr

  

  I didn't understand this piece at first. I saw that it was a simulation of a biotech lab, but there's a lot of fetishism of the esthetics of science in the technology art world and that's not exciting to me. What's interesting is that this is a "remix" of Bioteknica's actual lab, which means that this is a simulation of their actual working environment (down to the water cooler), where they actually use the technologies of bioscience to create visual art (put simply, they grow cells to make sculptures). Their art practice is a commentary on and exploration of the practice of biotechnology in a similar way that Richard Serra explores shipbuilding with his monumental metal pieces. However, without the permanent product, all they can show is a simulation of how they create their work. It's an unassuming hut that has a lot of innovative ideas behind it.
  

• Karaoke Ice, Nancy Nowacek, Katie Salen, and Marina Zurkow

  

  A simple idea--popular songs played back in the style of an ice cream truck, with karaoke where the cooler is supposed to be--taken to an obsessive level of finish. An exuberantly absurd statement.
  

Also, Steve Dietz' contextualization of the festival, in the form of his Edge Conditions show at the San Jose Museum of Art, is excellent. Possibly the best show of digital art I've ever seen in a museum (and I'm not just logrolling because he was the festival director and co-chair on the C4F3, it's really a great show).

Posted by at 7:03 PM | Permalink

Come to the ISEA/ZeroOne electronic art festival in San Jose next week! The festival features The C4F3, a cafe of augmented objects at the San Jose Museum of Art for which I'm a primary curatorial lead (aka "co-chair" in juried festival speak). If you've wondered where a significant chunk of my last two years has gone, this is it. There's a (partial) list of artworks that are going to be in the C4F3.

There are all kinds of shows (from SRL to Bill Viola to Peter Greenaway working as a VJ and remixing his own films in realtime) and great art pieces. etoy is installing their latest shipping container experience in front of the museum as I type this.

C4F3 details:

Open from Monday, August 7 until Sunday August 13.
Opening hours:
8 a.m.-5 p.m. Monday
8 a.m.-10 p.m. Tuesday-Saturday
noon-5 p.m. Sunday

San Jose Museum of Art
110 S. Market St.
San Jose
(408) 294-2787.

Meet the C4F3 Artists 2-2:30 p.m Tuesday

The San Jose Mercury News has a Good section on navigating the festival as part of their extensive coverage of the festival as a whole.

The C4F3 is designed by my good friends at Syneo in Milan.

Posted by at 10:20 AM | Permalink

More evidence that magic as an interaction metaphor may be viable for some applications. Themeaddicts, a company with some former Disney people involved, is making a kind of home talking mirror, a la "Snow White." They're claiming it as potentially useful for home security stuff.

When your home automation / security system senses a change in your environment, the character within the mirror theatrically appears and notifies you of the change. Once notified you may choose to directly view a camera feed on the stealthy monitor.

From the description it sounds pretty limited. Unless it has a lot of different ways to do that one function, I can imagine it'll get old awfully fast, like the novelty talking GPS voices. They're planning to charge a lot for it (see below), so if it's to be broadly accepted, its novelty and/or utility will have to stay high for a long time (much longer than, say, for a singing bass from Walgreen's).

It is important to understand that these devices are not inexpensive due to the limited production numbers and direct production and equipment costs.

That said, technology transfer from Magic Kingdom expats is of course welcome and I'll be interested to see how it actually works. For now, it seems like it's little more than some software running a simple animation and video switch connected to a security system. If the experience is really compelling, maybe that's enough, but I'm dubious. Originally seen on Engadget.

Posted by at 11:22 PM | Permalink | Comments (1)

Every generation expresses their need to exchange ideas in person differently. For the Victorians, it was the parlor, for the Beats, coffeehouse culture, for Hippies, it was the Happening. Now, it's the mini-conference. I could go on about the origins of the emergent phenomenon that's given us Foo Camp, Design Engaged, IDEA, GEL, etc. (the short version: ravers reinventing TED), but it doesn't really matter: I think face to face exchanges of ideas in interesting settings is a terrific way to enjoy life and spark great ideas.

Background and History

I did some research, and sure enough, lots of people were building toolkits to address the frustration of entering the physical computing world, but the development of the toolkits seemed to be happening in parallel and all similar solutions were being re-engineered. Around the same time, I met Matt Cottam of Tellart at the ICSID Industrial Design forum at CHI in April 2004 in Vienna. We had a good talk about smart furniture and design for physical computing. By October 2004, I was thinking about how to frame the problem conceptually. About a year later, while talking about the InstantSOUP project, I realized it was time to get everyone together.

Which, last week, is what we did.

Observations

1. The Henry Ford is an amazing place to hold an intimate technology conference. I had suspected that talking about the development of new technology in the context of the development of important familiar technology could be instructive, but I had no idea how strong the resonances would be. It was incredibly interesting and valuable to see how consistent the process of design and invention is. Seeing the actual products of that process in person was very powerful. It's one thing to make analogies about the creation of networking systems in the home as analogous to the introduction of electric power, and it's another to stand in a perfect recreation of Edison's lab and see all the pieces of that system, and the parts which led to the system, laid out around you. Looking at the pieces while a curator describes Edison's venture-capital funded development process, his management style and his handling of his media image, gave me about 20 deja vu moments. The museum staff are outstandingly helpful, knowledgeable, professional and flexible. Their house chef even made us a vegan dinner better than any I've had in San Francisco. And they're a 20 minute drive from a major airport, with a historic hotel within walking distance.
2. Hardware platforms are maturing. We saw a new version of the Arduino board, a new design for the EZIO board, and were loaned brand new interface hardware from ICubeX. With the upcoming Making Things MAKE Controller board, this represents a new generation of lightweight prototyping hardware. The benefit of this new generation is that they can control many recently developed sensors and actuators, which are cheaper and easier to use than previous generations.
3. Software is taking some of the burden. NADA (Tellart's product that Matt and I showed at Etech in March), Stanford's d.tools, and extensions to MaxMSP (which were all demoed at Sketching) shift some of the complexity of connecting physical computing components from hardware development to software. This shift has the benefits of enabling many of the prototyping platforms to interoperate--so you can mix and match functionality--leveraging people's experience with software development and making the process of sketching in hardware more approachable. To paraphrase one presenter, wires are scary. They just are. Especially if you forget ground.
4. Hardware needs to get cheaper. Sketching, in my original definition, meant making something quickly, but it also means making something cheaply. If you want make 100 hardware sketches in a day (as art students are told in traditional drawing classes or as designers may want to do when brainstorming), then it should be cheap to make each one. Sparkfun is doing a lot make hardware pieces cheaper and points the way to making things cost one or two orders of magnitude less, while retaining the same functionality, but that's definitely an important consideration that hadn't occurred to me before.
5. We need to work on solving common problems simply. Simplicity is hard and hardware was the province of professional electrical engineers and robotics enthusiasts, and neither group had great incentives to solve the design problems problems simply: one wanted to ship faster and cheaper, the other wanted more powerful, interesting robots. This idea has begun to appear, however: Sparkfun boards don't expose all of the functionality, just the stuff they think is interesting. At least one person is starting to rethink spec sheets. USB on Rails, means using a USB mode that doesn't require writing device drivers. Good libraries do just this, which is why d.tools is focusing on creating a product with libraries and open sourcing their work (because open source creates more libraries and enables faster sketching, aka agile development in the software vernacular).

There were probably 10 other major themes that came up during the discussion, but I believe that these, together, will drive a revolution in first the development of hardware prototyping toolkits, then the development of prototypes (and art, which is a kind of unconstrained prototype) and finally the development of ubiquitous computer products. I believe that the capabilities of our tools shape the final products of those tools. This is where I see our tools going, which means that we can now start to see (or at least hypothesize about) where the field as a whole is going.

Sketching in Hardware 2

• Packaging. Some schools, like RISD, teach students to make two prototypes of any idea: one that looks like what they envision and one that works like it. Merging the two is difficult, and even many of the prototyping toolkits (ICubeX excepted) don't try to cover the basic circuit board with a case. End users experience the interaction through the arrangement of user-facing interface elements and this has not been addressed by the toolkits yet. This is the industrial design component of interaction design, and I think it's important to couple it closely with the actual electronics and functionality, since it's the last step in the bridge to the end user.
• External data sources. If we're sketching to develop ubiquitous computing products, which I expect we are, then the objects we sketch for need to be able to manage (create and process) external data sources. As a first cut, objects need to be able to import information feeds from the Net in order to display them, but if they start producing such feeds (like Julian Bleeker's blogjets), that creates a whole new environment. A conversation cross-pollinating the ideas from the microformat folks to the ubicomp folks may be valuable, especially when the microformats are designed not for the world at large, but for the room you're in.

I look forward to next year. This year, we met each other and began the creation of a community. Next year, we get down to business.

Resources

• Links to presentations. As we get more, we'll link them here. You can see who everyone is on the participants page.
• Liz's notes. Elizabeth Goodman has posted her notes of all of the presentations. I took photos of many of the slides as people presented them (especially on June 24), and you may be able to reconstruct the gist of the talks by using these together (especially if you work backwards through the set).

Pictures

Thanks

For your incredible assistance: Judith Zissman in communicating, organizing, focusing and structuring the conference, Matt Cottam for discussing the ideas and providing design support, Liz Goodman for her last-minute late-night design and assembly magic, Mom (yes, my mother) for picking up the t-shirts and loan of her and Dad's inkjet printer at 3AM, Andrew Otwell whose format for Design Engaged I appropriated for Sketching and especially the whole Henry Ford staff for your tremendous hospitality.

Posted by at 6:01 PM | Permalink | Comments (1)

The recent interest (thank you Engadget, BoingBoing, Digg, and the rest!) in my recent magic wand prototype led me to think it may be good idea to start making a bibliography of magic as user interface. Adrian McEwen articulates the broader implication of my thoughts well in this blog entry inspired by my wand piece:

[The desktop metaphor] is becoming increasingly stretched, as we need more powerful ways to visualize and manipulate data, and particularly as computing power bleeds out of the PC and into the world around us. With ubiquitous computing making our "intelligent devices" more specialised, the requirement to have their interface conform to a general-purpose workspace such as the desktop becomes less and less desirable.

I agree, obviously, and he said it better (and more succinctly ;-) than I would have. Magic as an alternative UI metaphor has appeared a number of times in HCI writing in the last 20 years, talked-about by many of the greats in the field. Now we can actually implement some of it, I figured it may be useful to go back and see what has been written about it in the past. Here is a list of publications that have talked about magic or enchantment in HCI contexts. It's by no means exhaustive, but it's what I could find in an evening back in April using Google and chatting with Ben and Liz.

• Ballagas, R., Walz, S., and Borchers, J. REXplorer: A Pervasive Spell-Casting Game for Tourists as Social Software. In CHI 2006 Workshop on Mobile Social Software (MoSoSo), April 2006.
  
• Barton, J., Pierce, J., Quantifying Magic in Ubicomp System Scenarios, Postion Paper, Ubisys 2006 at Ubicomp 2006, September 17-21, 2006
  
• Bier, E. A., Stone, M. C., Pier, K., Buxton, W., DeRose, T. D., Toolglass and Magic Lenses: The See-Through Interface, SIGGRAPH 93 proceedings, 1993 (the rest of Xerox PARC's Magic Lens papers)
  
• Binsted, Kim (2000) Sufficiently Advanced Technology: Using Magic to Control the World. Plenary at CHI 2000.
  
• Center for Tactical Magic, "Magic(k) Calls", Arthur Magazine, October 2006 (8.6MB PDF)
  
• Cheok, A.D., Nguyen, T.H.D, et al, Magic Land - A Mixed Reality Interactive System Integrating 3D Live Persons, Singapore Mixed Reality Lab, 2004
  
• Ciger, J., Guiterrez, M, Vexo, F., Thalmann, D., The Magic Wand, Proceedings of the 19th spring conference on Computer graphics, 2003
  
• Crone, M., Mattsson, J., Eliasson, E., Wiatr, P., Magic Bowl: a Tangible User Interface for Configuration of Interactive Environments, Proceedings of COOP04
  
• Davis, E., Techgnosis: Myth, Magic + Mysticism in the Age of Information, New York, Harmony Books, 1998
  
• Doherty, J., Keeling, K., Newholme, T., Fowler, D., McGoldrick, P., Mcaulay, L., Stories, myths and metaphors: Explaining self-exclusion and Internet use in the home, Proceedings of HOIT 2003
  
• Gross, M.D. and Eisenberg, M. Why Toys Shouldn't Work "Like Magic": Children's Technology and the Values of Construction and Control DIGITEL 2007: The First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning, Jhongli, Taiwan, 2007.
  
• Hinske, S., Langheinrich, M.: Managing Augmented Toy Environments - A New Perspective for Smart Space Management, Proceedings of 4th International Workshop on Managing Ubiquitous Communications and Services (MUCS), held in association with IM 2007, Munich, Germany, May 2007
  
• Hulterström, K., Witches, Warlocks and Traffic Encounters: Designing the interaction for an ad hoc multiplayer gaming experience, Master's Thesis, 2003
  
• Lacucci, G., Kuutti, K. and Ranta, M., On the Move with a Magic Thing: Role Playing in Concept Design of Mobile Services and Devices. In proceedings of DIS 2000, ACM (2000)
  
• Laurel, B., Computers as Theater, Addison-Wesley, 1993
  
• Lawler, J., Metaphors we compute by, 1987/1999 (this was also the title of a class I took from Lawler in 1987 at the University of Michigan's Residential College. That class was incredibly influential on my thinking about language, interface and computers. THANK YOU,Prof. Lawler!)
  
• Levy, S., Interaction through Spells - Establishing traces of Invisible Connections, master's thesis, Stockholm University and Royal Institute of Technology, 2005
  
• McCarthy, J., Wright, P., Wallace, J., The Experience of Enchantment in Human-Computer Interaction, Proceedings of CHI04, 2004
  
• Nolan, S., Building magical realms: responses to pervasive and locative media technology, Digital Creativity Journal, pp. 185-192, Volume 17, Number 3, 2006
  
• Raskin, J., Silicon Superstitions, ACM Queue vol. 1, no. 9 - December/January 2003-2004, 2003
  
• Rawat, T., Wonder objects: magic and interactive storytelling, Proceedings ISIMD 2005
  
• Redström, J., Designing Everyday Computational Things, PhD Dissertation, 2001
  
• Rohrer, T., Metaphors we compute by: bringing magic into interface design, 1995
  
• Scott, J., UbiComp: Becoming Superhuman, Proceedings of Ubicomp 2005
  
• Schmitz, M., Baus, J., Schmidt, S., Towards Anthropomorphized Objects: A Novel Interaction Metaphor for Instrumented Spaces, 3rd International Workshop on the Tangible Space Initiative, 2006
  
• Smith, R., B., Experiences with the alternate reality kit: an example of the tension between literalism and magic, CHI 86 Proceedings
  
• Smith, W., Lewi, H., Magical Beginnings of the Mundane, International Workshop on Simtech, Melbourne, November 26-27, 2007
  
• Stern, E., A Touch of Medieval: Narrative, Magic and Computer Technology in Massively Multiplayer Computer Role-Playing Games, Computer Games and Digital Cultures Conference Proceedings, 2002
  
• Svanaes, D., Verplank, W., In Search of Metaphors for Tangible User Interfaces, Proceedings of DARE2000
  
• Tabor, P, Crampton-Smith, G., Steiner, Y., Usable Witchery workshop, January 2008
  
• Tiplady, R., Harry Potter' technology
  could be coming your way, BusinessWeek Online, June 9, 2004
  
• Twist, J., Potter magic charms Nokia chief, BBC, October 28, 2003
  
• Tognazzini, B. Principles, Techniques and Ethics of Stage Magic and Their Application to Human Interface Design, Proceedings of INTERCHI'93, 24-29 April 1993. ACM: New York. pp355-362.
  
• van Loenen, E, On the role of Graspable Objects in the Ambient Intelligence Paradigm, Smart Objects Conference SoC 2003, Grenoble, 15-17 May, 2003
  
• Vinge, V., "True Names" (1981), in True Names: And the Opening of the Cyberspace Frontier, Tor Books, 2001
  
• Waern, A., Söderberg, J., Akesson, K., Björk, S., Enhanced Reality Live Role Playing, Proceedings of Pervasive 2004
  
• Walz, S., Ballagas, R., Borchers, J., Mendoza, J., Kratz, S., Wartmann, C., Fuhr, C., Tann, M. J., Shin, D. Y., Hameed, B., Bardos, L., and Hovestadt. L., Cell Spell-Casting: Designing a Locative and Gesture Recognition Multiplayer Smartphone Game for Tourists In Proc. PERGAMES, Third International Workshop on Pervasive Gaming Applications at PERVASIVE 2006, Dublin, Ireland, May 2006.
  
• West, D., Magic, Proceedings of OOPSLA 2002
  
• Weston, D., Barney, J.A., US Patent 6761637: Method of game play using RFID tracking device
  
• Wilson, A., Wireless User Interface Devices for
  Connected Intelligent Environments, Ubicomp 2003 Workshop on Multi-Device Interfaces for Ubiquitous Peripheral Interaction, 2003
  

[UPDATE: Thanks to Eoin Brazil for the David West reference, which in turn references Vernor Vinge's True Names, which I've also included here.]
[6/8/06: I wrote a small addendum to this list]
[10/10/06: Added Levy and Raskin references; thanks Tod and Cassidy!]
[10/19/06: Added Schmitz, et al and Mattsson]
[11/20/06: Added Tiplady and Twist]
[11/22/06: Corrected Tog reference; thanks Chris!]
[11/29/06: Added Balagas, et al, and Walz, et al.]
[12/1/06: Added Davis, though I feel it's not as design focused as I really want to go, but I do want to acknowledge that it was an early hypothesis of role technology plays in a changing society. There are many history and philosophy of science documents that I could cite if I continued down this path, but I'll include them sparingly.]
[12/22/06: I turned off comments because this post is getting too much spam. If you want comment on this, send me email at blog c/o this blog. Thanks.]
[1/3/07: Added Nolan]
[2/9/07: Added Center for Tactical Magic, thanks Jordan!]
[1/8/08: Added Barton, Pierce; Hinske, Langheinrich; Gross,Eisenberg; Smith, Lewi]
[1/12/08: Added Weston, Barney]
[3/30/08: Added Wilson--thanks Jeff!]
[5/9/08: Added Cheok, Nguyen, et al]
[11/16/08: Added Tabor, Crampton-Smith, Steiner]

Posted by at 6:52 PM | Permalink | Comments (4)

It's not surprising that The Grand Rapids Press is covering an inventor's creation of embedded a/v furniture. After all, Grand Rapids is the home of the steel furniture industry in the US (both Steelcase and Herman Miller are in the area and have been for 100 years). What's interesting is that they're covering a local inventor/designer's technology furniture. Now, it's not that surprising that someone near the heart of the steel furniture business mates it with technology, but it's still surprising because Allendale is so far from the other centers of technology and design.

A hidden projector plays video on a recessed, ceiling-mounted projection screen.

[...]

Sit down next to a stylish end table. Then open the top drawer and flip up the flat monitor tucked neatly inside.

[...]

Kick back and watch TV on your large plasma screen. Also use it to browse the Web, edit documents and play video games, because the entertainment center has an integrated computer.

Pete Freeman, the designer, is experiencing problems marketing his designs (there are photographs in an extended interview with him). In some ways, I'm not surprised. Freeman's designs are a kind of a wooden casemod, but he wants to sell the computer and the furniture together as a unit: that makes for either a very expensive piece of furniture or a computer that feels like it's going to be difficult to upgrade. The vastly different replacement cycles of furniture and computers thus becomes a big stumbling block, and an interesting one from the perspective of the potential customers of ubicomp: how do you sell something that's simultantously "built in" and replaceable? To some extent, I think that he's doing the thing that early radio cabinet designers did, and then again television cabinet designers: they made the designs look like furniture, in the hope that it would be more acceptable in the living room; eventually (and probably quickly), as people became comfortable with the new devices, their designs stopped trying to hide their functionality. The issue is that living room computers jumped to the "looking different" phase (by not being designed for the living room at all, or being designed to look like stereo equipment) and perhaps there's no clear need to disguise the functionality, to just to focus it better. Then, if it's clear that a computer is not just a general purpose computer, but a task-specific device, it no longer has to be hidden; or if it is, that hiding doesn't have to bring the baggage of having to hide a full-fledged PC (with cooling vents, power, CD tray, keyboard, monitor and mouse). If the function is focused, it'll be clearer where it should go and how it should go there.

But, you know, that's all theory. Freeman's actually made furniture that has computers hidden in it. It's not calm computing and it's not ambient display, but it's on the continuum. Having recently seen most of the Milan Furniture Fair, I know there's a market for everything, but I hope he doesn't have his hopes set on protecting his intellectual property rights: the furniture design industry is all about idea appropriation and recombination (that's what's called "fashion") and the best you can do is to get to market first and hope that you can get a season ahead of the copies, or use techniques that no one can easily duplicate.

Posted by at 7:35 PM | Permalink | Comments (1)

Nike and Apple have announced the Nike Plus line of shoes that integrate with an iPod Nano through an antenna that attaches to the iPod. As far as I can tell, the shoes broadcast telemetry and the Nano collects it and then downloads the data to software running on your computer as it syncs the music (which is grouped into playlist-based workouts). This may be the biggest commercial wearable computing project since the Adidas 1 robotic adaptive shoe (and, tangentially, an interesting departure from Nike's previous technology venture with Philips, which produced some MP3 players in the 90s--Nike + Apple clearly makes more sense brand-wise in today's market), and there are many more models than that product--which went from running shoe to basketball shoe in its latest incarnation.

The possibilities are, again, pretty interesting: because they broadcast (even if near-field) and probably broadcast with a unique ID (so that two people running together don't get each others' telemetry) these shoes could be used to track people in the panoptic scenario, but they could also produce a much more sophisticated pedometer. For example, why just sports shoes? Why not make a deal with Campers and give urban hipster iPod users an idea of whether they should have just walked home instead of taking the subway 10 blocks? Or Bruno Magli? Or why not do a version of Dance Dance Revolution without the pad? (well, OK, maybe just a simple two-leg hokey pokey version) I'm waiting for someone to hack it.

Posted by at 2:39 PM | Permalink | Comments (1)

MIT's Technology Review did a nice short article about Roombas and hacking.

One Bluetooth-enabled Roomba was given a green fabric pelt and was used in a real-life Frogger game at the SXSW Interactive conference in Austin, TX, this year, and ThingM's Tod E. Kurt is collaborating on a live-action Pac-Man re-creation, with reprogrammed Roombas dressed up as Pac-Man and ghosts, navigating a maze and vacuuming up tissue paper dots.

The key part of the article is the popup "Roomba tour" (click on the "click here" link in the article). Thank you, MIT review. Go Tod!

Posted by at 11:29 PM | Permalink

Continuing on the magic thread Sony demoed an augmented reality card game for the PS3 video game console at E3 called "The Eye of Judgement". Cards are, of course, an everyday magical element in myth, up there with wands, and this is the first instance that I'm aware of (though folks will tell me otherwise, I'm sure ;-) of treating them as literally magical interface elements. This iteration, and people still know little about it, but the demo appears to be a combination of some vision processing using the PS3 and the EyeToy camera. Gamespot describes it this way:

In The Eye of Judgement, it appears that you'll buy packs of cards, just like you would for Magic or Pokemon, with the idea of being able to create a deck capable of beating someone else's deck. The difference is that when you put the cards down on a table, the PlayStation 3 is able to recognize the card, via the EyeToy. Each card is associated with a different monster, and on the television screen, a virtual monster erupts out of the card.

Engadget took a photo off the E3 presentation screen:

It'll be interesting to see how this kind of UI moves into the daily world and how vision recognition or embedded RFIDs in otherwise everyday objects make them behave in magically when viewed through the enchanted mirror of technology.

Posted by at 8:58 AM | Permalink | Comments (1)

It appears that Sony was granted a patent in August 2005 on an optical magic wand idea. The description is for "an input device for interfacing with a computer" which "includes a body configured to be held with a human hand" that "includes a light emitting diode" that changes color in a way that "is capable of being detected" by "an image capture device." In other words, it blinks in a specific pattern that's detectable to a camera connected to a computer. I've heard of other systems that encode data into rapidly blinking LEDs (the most obvious is the infrared LEDs that power remote controls), but I think they're thinking of visible light. Maybe not, maybe it's all still IR LEDs, but the key is that it's detected by a general purpose camera, rather than a specialized IR receiver.

My prediction: soon, wands for everyone!

(The link to the patent is here and I was alerted of this by this New Scientist article.)

Posted by at 11:46 AM | Permalink

Why Magic Matters

As a designer, I want to know how to design for these expectations. Why not start by borrowing from a familiar and popular set of myths that offer a system describing the operation of the word which is somewhat animist, occasionally capricious, often controllable and tolerant of the occasionally unexplainable? Namely, the myth of magic. Magic can be an useful framework for describing many of the kinds of functionality that contemporary technology enables. Things such as: action at a distance, objects with memory, nonobvious behavior, etc. "Are you saying we should not tell people how things actually work and tell them it's magic?" That may seem like a pretty irresponsible move, but if the options are telling people nothing (as it is currently) with the hope they create the correct mental models for themselves, or giving them a framework that's known to be a fiction but serves as a conceptual umbrella for explaining the interactions between objects whose technology is (ahem ;-) indistinguishable from magic, maybe magical explanations will produce better experiences?

Others have talked about enchantment, especially John McCarthy of University College Cork, Ireland, who's currently editing an issue of Personal and Ubiquitous Computing on enchantment and published what's probably the earliest paper (1MB PDF) on enchantment in HCI.

[11/19/06 UPDATE: I wrote a more thorough discussion and compiled a bibliography of magic and user experience design.]

One of the most common everyday magical items in myth is the wand. Last year, based on some work I saw at IBM's NPUC conference, I realized that wands were possible using a combination of wireless tracking and gesture recognition. This weekend, I decided to make one, just to see what would happened. It kinda worked. Here, written on an ancient scroll I just wrote, is how you can reproduce my eerie results.

Thee yngredyentse

1. A Mac running OSX
2. One Gyration "air mouse". This rare beast (found mostly in Froogle) is the key ingredient. It's a mouse that has a gyroscopic internal organ that senses the mouse's orientation. There are several other accelerometer-based mice, but this one seems to perform better, especially with broad arm movements (although it doesn't work well as an actual computer mouse in that situation: to use it as a pointer mouse you need to keep your arm still and just move your wrist).
   
   (an Amazon link, though you can get them cheaper refurbished)
3. USB Overdrive, a universal OSX USB human-interface device driver
4. Quicksilver, the universal launcher utility
5. The Abracadabra plugin for Quicksilver. There is a good set of instructions for how to use the thing, but for some reason I couldn't download it from within Quicksilver. If you have that problem download it here and then drag it onto Quicksilver to install it.

Thee yncantationne

1. Install the mouse. As a USB human interface device, this means "unpack and plug it in." OSX may ask you to verify a phantom keyboard because the mouse base station (which, in the case of the mouse I bought, is huge--Gyration, wassup with that?) announces itself as both a mouse and a keyboard. I got the mouse without the keyboard, so I just closed that window.
2. Download and install USB Overdrive. Restart. Open your System Preferences and set it up so that the right mouse button on the Gyration mouse does nothing but hold the Option key down. Your USB Overdrive control panel should look like this:
   
3. Set the Quicksilver Preferences and modifier to match the one you set for the righthand button in USB Overdrive. Your Quicksilver Preferences should look like this:
   You may want to set sounds at this point, too, since apparently Abracadabra has problems without sounds.
4. Set the mouse tracking to minimal. Go to the "Keyboard&Mouse" control panel and move the mouse tracking slider all the way left. This is so that the gestures you make in the air do not get distorted by the operating system, which normally makes tracking proportional to movement speed (normally, as you move the mouse faster, the cursor moves proportionally farther). It should look like this:
   
5. Set a gesture-based trigger in Quicksilver as per these instructions. Keep the gestures simple. I'm still experimenting, but basic back and forth lines seem best. Circles are doable, but hard to master. Maybe this is where the fabled practice all young wizards have to do comes in: you need to create gestures that are useful AND you need to learn how to perform them successfully. THAT, I leave up to you, but here's a basic one:
   

Now you have the basic ingredients for a functional wand-like device that runs under OSX using mostly free software. Grab the index finger trigger and push the right mouse button on the Gyration mouse, and wave it in the the air (you don't have to wave it at the computer screen, that's part of the fun). Enjoy the magic.

Commentarye

[Update: it appears that my use of Abracadabra for this project is not an accident. I tried a number of different systems before settling on it and it just seemed right. According to this blog post, that's because a CMU student named Jason Cornwell developed the software for a class, in order to do just this, i.e. to make a magic wand using a gyroscopic mouse. I haven't found any documentation of the project online, but Blacktree acknowledges that he had a hand in making Abracadabra. Thank you Jason!]

[Update 2: Tod Kurt, my business partner in ThingM, took the initiative and blogged what he saw when he took the Gyration mouse apart.]

[Update 3: I expanded my thoughts extensively on the "why magic matters" section of this post in a talk I gave on 10/11/06.]

Posted by at 9:48 PM | Permalink | Comments (10)

This has been making the rounds today: Max Dean, Raffaello D'Andrea and Matt Donovan's self-assembling chair. It's a really entertaining project. Why a chair?

First of all Dean and D'Andrea (I believe) did "The Table" five years ago, which is a great early piece of robotic furniture that followed people around in a room. That piece, I think, got at a lot of the ambiguous feelings people feel when encountering an animist relationship between themselves and something that looks familiarly inanimate (the artists carefully chose to conceal all the functionality in a traditional table design) but has behavior. There's a classic quote in the video for "The Table" that illustrates this: "I don't think it likes to be touched...Oh, I think it's just me."

Second, I think, chairs have a uniquely anthopomorphic relationship to people, anyway. They have four legs, like animals, and they look (for obvious reason), like sitting people. To see one fall apart is shocking because we don't expect furniture to do that with nothing near it (thus violating our innate sense of the physics of inanimate objects) and because there's a visceral raction to the image of something that's familiar and shaped like a person having their limbs fly off. Maybe that's too much projection, but I think that the initial shock, and the scene of the thing trying to pull itself back together elicit--and are probably designed to elicit--feelings that we wouldn't normally feel for a self-assembling object that didn't have such strong cultural and psychological connections. That's what makes it art, rather than design.

To me, the chair pushes the "uncanny valley of animism" button (excuse my mixed metaphor) less than the table, since it's more obviously mechanical, but it's still a great piece and an interesting exploration of the emotional relationships people have with their domestic objects. And it looks to be a great piece of engineering, too.

Posted by at 11:05 AM | Permalink | Comments (1)

I returned from a week at the Milan Furniture Fair yesterday (I've put a large collection of unsorted pictures from the Fair on Flickr). It's my third year going to the Fair, and I always find it a fascinating experience. This year was special, because it marked the first time I was able to participate. Over the last four months I worked with Whirlpool's Global Consumer Design group and and Syneo to put together a book-length description of their in.home experimental design project.

Every two years for the last 4 years Whirlpool does a large-scale conceptualizing and prototyping exercise to examine and extend the limits of appliance design. The previous installment, in.kitchen was two years ago and focused on the kitchen. The one before that, Project F on laundry.

This time Whirlpool decided to try something different. Recognizing that the role of appliances is moving away from their traditional places in the kitchen and laundry room, they decided to explore a household of linked appliances. This involved two major shifts in traditional appliance design thinking: conceptualizing new devices for locations where they've never been and inventing ways that they could all interact as a system, rather than just as standalone devices. Whirlpool and Syneo created 11 new appliance concepts that they took all the way to physical prototype and installed at the Future Technology in the Kitchen exhibition at the Fair. To envision how these products would interact, they developed four use scenarios and had the prototype house "act" them out. The four scenarios represented (roughly) an active morning, a quiet morning, a busy evening and a quiet evening. Together we added specific details to each scenario to create an engaging narrative.

My contribution was as researcher, writer and editor of the book describing the project and contextualizing some of the reasons for why this kind of exploration is important. I can't include the book here here, but I extracted some pages from a draft and compressed them so they would be reasonably downloadable.

Here are:

• My introductory essay (108K PDF), in which I try to contextualize the design of appliances as being on the continuum of ubiquitous computing and interaction design.
  
• A four-page layout (of some 16 pages) providing some context based on demographic and social trends (170K PDF) for why this kind of experimentation is important.
  
• A sample description of one of the prototype products, the Living Cube (164K PDF).
  

[note: all of the images in these PDFs have been severely compressed to make for downloadable PDFs]

I think that the project turned out very well and I am really honored to have been able to contribute. Appliance design, as a concept, has the potential to be on the forefront of making domestic technology actually useful, rather than merely exercises in technology. Compare the concepts of "appliance integration" and "home automation": the first one describes the unification of devices that have specific functionality, with--presumably--a similarly narrow focus in terms of the purposes of integration; the latter is an abstract concept that implies that in "automation" in itself is somehow beneficial, without specifying what's being automated.

But I digress. It was a great project to work on and I'm very grateful to Whirlpool for initiating it and to Syneo for bringing me in.

Here are some pictures I took of the setup at the Furniture Fair:

Posted by at 11:30 PM | Permalink | Comments (1)

Following up on my earlier post and attempting to include the terms that Adam and Bruce introduced, I wrote a sentence using all of the appropriate ubicomp buzzwords, as an attempt to create a narrative that ties together the fields, at least semantically. Here it is, in all its gory detail:

To create a world of ambient, ubiquitous intelligence, we will use physical, pervasive, often portable, information processing objects, which we will refer to individually as spimes and appliances, and collectively as everyware.

Liz is right of course when she tells me that it's a questionable exercise, when there is no master narrative and all of these terms emerge as products of different cultures, and Anne is right that creating rigid boundaries around terminology can be exclusionary of ideas and people, but I think it's valuable when trying to wrap my brain around the stuff. Or at least it'll make for magnetic poetry fodder.

Posted by at 4:39 PM | Permalink

Last week I had the pleasure and honor of presenting at O'Reilly's Emerging Technology Conference. It was a great party, and I had a great time presenting with Matt Cottam of Tellart. Our presentation was essentially a follow-up to a blog post I had made in 2004. Matt and I met at CHI 2004 in Vienna and some of his observations led me to that blog post, and we've been in touch ever since, discussing these ideas.

Our presentation consisted of two parts. I lead with a discussion of why sketching is a good metaphor for the kind of rapid hardware prototyping that is required as we move from the definition of basic technologies to designing products and experiences with those technologies. This dovetailed well with Bruce Sterling's keynote, in which he talked about defining the future of smart objects by defining the language we use. My point is related: the definition of our technological future rests in tools we use. It's not a new idea, but I think it's important to be thinking about it right now, as the field moves from the component engineering stage to subassemblies defined by end-user experience, rather than by engineering constraints.

Here's the abstract:

Robust physical computing prototyping systems are appearing continuously. More than just Lego Mindstorms, the BASIC stamp and microcontrollers, physical computing prototyping kits are a lightweight way to create real world objects that have interesting functionality without having to learn (too much) electronics or mechanical engineering. New software glue layers allow for much easier interfacing with existing software products, which opens up a whole new world of hacking beyond the screen and beyond basic circuit bending.

And here are my slides (330K PDF)

Matt's half of the talk introduced NADA his company's piece of software that's explicitly created to enable sketching in hardware by easily and rapidly connecting rapid hardware prototyping toolkits to software that's created for designers, rather than engineers (specifically, Flash).

Here's Regine Debatty's coverage, and here are Liz's notes. Thank you, both!

(photos by James Duncan Davidson/O'Reilly Media)

Watch this space for more action along these lines. WOOHOO! Thank you, Matt!

[Addendum: I forgot to mention that in the hour before our talk, Matt and Mike Migurski of Stamen adapted Mike's IRC backchannel visualizer to be a virtual controller for an old table lamp. As people typed in the Etech IRC back channel, they could control the brightness of the lamp. It was possibly the fastest real-world data visualization mashup, ever. Unfortunately, as we were lifting the foamcore to the table right before the talk, we must have created a short in the relay controlling the lamp, and it didn't really work while we were on stage. However, next time, we're planning on having it as realtime speaking rating ambient display, so speakers can know what the audience is really thinking about their talk. It'll be a good social experiment. ;-)]

Posted by at 3:43 PM | Permalink

Yesterday I was explaining what I do to a friend and started getting caught up in the usual tangle of terminology, so I came up with a structure for the different terms related to the fragmentation of information processing into everyday objects. As I see it, the different terms--pervasive computing, ubiquitous computing, ambient intelligence and physical computing--come from different historical contexts that are based on geography: PARC coined "ubiquitous computing," so it's big on the West Coast; IBM likes "pervasive," they get the East; Philips was responsible for "ambient intelligence," so that's what it's called in Europe. In reality, it's just a blind men and elephant problem. They're all describing the same idea, but alliances and territoriality create clusters of terminology. So here's how I described it to my friend:

"Who" is, of course, left as a a big question, but that's why there are so many anthropologists involved now, I suspect.

The definitions aren't totally separate, but it's an interesting exercise to see the focus of the groups who fly a particular flag. I still think it's all the same elephant and that maybe it needs an even yet different term. There's great value in creating a good term that encapsulates a set of ideas, but it has to accurately capture the essense of an idea as it is perceived by others to take off. Which means it needs to be externally-focused, and not about the process. I feel that none of these terms is sufficiently strong in that department, though I'm going to use "ubiquitous computing" and "ubicomp" for now, since I'm from the west coast and Weiser deserves mad props for having seen it first.

[1-31-06 update: Anne has written a typically thoughtful and insightful commentary to this note, to which I've replied. Thank you, Anne!]

[2-5-06 update: Doh! Peter tells me that if I had been paying attention, I would have noticed that my alma mater, Wired, is also on the disambiguation tip. More general than my take, but still. Peter generously said "must be in the air."]

Posted by at 9:29 AM | Permalink | Comments (4) | TrackBacks (2)

Back in February I talked about World of Warcraft and Feedback. I'm on to a new game, The Sims 2 from Electronic Arts/Maxis. My insight about Warcraft was that automatic feedback across a number of progress indicators is a seductive way to measure our success in the virtual world--part of the satisfaction of the game is watching the progress bars go up--and that it could be a model for how automated feedback could be created in a ubiquitous computing environment. The Sims 2 takes this to a new and more intimate level. The game is much closer to our reality, so its metrics start seeming eerily familiar, making its simulation that much more emotionally resonant. It's also a good place to study what kind of feedback would actually be appropriate in an instrumented environment.

(image from Wikipedia, showing a number of elements of the game)

Sims 2's progress indicators are divided into three categories: Needs, Aspirations and Skills.

Needs:

• Food
  
• Comfort
  
• Hygiene
  
• Bladder
  
• Energy
  
• Fun
  
• Social
  
• Environment
  

Aspirations:

• Wealth
  
• Knowledge
  
• Family
  
• Romance
  
• Popularity
  

These, in turn, spawn wants and fears, which are represented by icons

Skills:

• Cooking
  
• Mechanical
  
• Charisma
  
• Body
  
• Logic
  
• Creativity
  
• Cleaning
  

[Tangentially, this list starts to amusingly betray many of the deeply American and, specifically, Northern Californian values in the game. They don't seem to aspire to be great buck hunters or farmers, they carpool; they move house pretty easily; gene mixing is encouraged. They probably vote Democrat. ;-)]

Maxis must have thought about this pretty carefully during game design and it's interesting to see what they came up with. I just started playing the game, but it's already starting to seem like a good list to start thinking about the implications for where to start incorporating these ideas into the physical world. Some things are clearly more difficult than others, but taking a broad view, some interesting ideas emerge.

For example, you can immediately see how instrumenting basic needs with sensors and alarms is already the focus of people working in the medical and geriatric technology fields. It may not make sense for a healthy person to have a piece of hardware worry about their bladder, but in a hospital situation it becomes critical. For a person suffering from Alzheimer's Disease, notification of a need for food may be important (hell, I sometimes need to be reminded to eat.). How do you measure "fun"?

This list is profoundly user-centered. It's not starting with a monitoring technology and saying "well, if we tracked every single person everywhere and tried to identify what they're doing based on Baysian filtering, how could we make money off of that?" Instead, its focus is: here's what people desire from life and here are the wants they try to satisfy, how can we support them with technology that helps them track their progress? How can we help people feel less like pawns and more like Sims? (and I know that that still doesn't make them feel like people, but I feel it's a step in the right direction)

Posted by at 3:11 PM | Permalink | Comments (1)

I've put the photos I took of the big screen at the last two conferences I participated in, DUX05, which ended yesterday, and Ubicomp 2005, which was in Tokyo in September. I didn't take pictures of every slide, just the ones I was interested in, but I figured that others may be interested in 'em too, even though the quality of the photography isn't always great. I've put them up on Flickr and I encourage everyone to annotate and tag them as you feel appropriate.

Here they are:

DUX:

Ubicomp:

Also, in case you want to try this at home, here are my off-the-cuff thoughts for taking photos of slides at conferences:

1. Set your camera's ASA as high as possible (mine goes to 400)
2. Pay attention to the shutter speed. If it's super-low (like 1/4 of a second or less), you're not going to get much no matter what you do, so just listen to the talk.
3. Sit roughly in the middle and 5-10 rows back; you may be able to get a flatter perspective if you sit further back, but you have less light coming from the screen so you have to zoom, which makes it more likely that the images will smear beyond readability.
4. Take two pictures in rapid succession. Odds are that one of them will be less blurry than the other, usually the second one.
5. Keep you camera open and turned on, people flip through slides fast.
6. Don't worry about taking perfect pictures. It's just documentation for the sake of convenience. The actual slides are available elsewhere, if you really want them.
7. Hold your camera with both hands.
8. Never use the flash. It doesn't help and it's annoying.
9. Like with a sunset, sometimes you just have to put the camera away and experience the real thing.

Posted by at 1:36 PM | Permalink

Liz has written an excellent summary of Ubicomp 2005, warts and all. I admit there's some logrolling here, since I get namechecked in it (and, well, we live together and stuff), but I think it's a very articulate analysis of the conference and says everything I would have said. Although I had a great time at the conference, there was something missing, which Liz accurately identifies:

Especially in the systems-oriented long papers, what�s often missing is �design� � in the definition of design as a process that creates artifacts integrated into human contexts. I tend to prefer the shorter and less weighty formats such as posters and workshop papers. Paradoxically, this very lightness makes posters and workshops more immediately relevant and often more compelling. Unlike the longer papers, they don�t wear out their welcome.

Posted by at 11:27 PM | Permalink

At the recent NPUC there was a demo of the IBM SHARK gesture recognition system (pages with videos and a download of a software demo). As presented by Shumin Zhai, this is an alternate handwriting-based text input methodology. It works roughly like this: you draw a line between all of the letters in a word on a grid and the system recognizes the shape of the line you drew as the word, even if you didn't land exactly on all of the letters exactly. There's enough identifying information in the line shape to produce high accuracy of uniquely identifying the word. This similar to how Xerox's Graffiti method works for individual letters, but it's for whole words, and is significantly more efficient, apparently rivaling typing. Since the shapes of words on the grid are relatively unique, the grid arrangement is fairly arbitrary (and IBM is trying to find an optimal one for English). Although initially taking up more space than the Graffiti box, ultimately, the grid become unnecessary as people learn the shapes of the word-lines. Once that happens, the system can recognize the shape wherever it's drawn, and even recognize it despite severe distortions. Text entry ergonomics are generally a geeky thing, but this is way cool.

It's also seems like a much better way of introducing people to gestures command systems than having them memorize arbitrary gestures (even simple ones). First, people make the arbitrary shapes on a grid. Then, as they get good, they can put the grid away and the gestures still work. More importantly (for me, anyway), is that the shape of the lines is arbitrary. It could mean a word, or it could be a sequence of pictures, or a direction on a map. The basic recognition engine doesn't care.

As I was watching the demo, this thought came to me and I couldn't watch the rest of the demo without thinking it: magic wand. With an accelerometer embedded in a wand, or an IR LED and camera tracking system (like the PS2 Eyetoy), suddenly the waving of the wand, the patterns of loops, lines and intersections in the air, become meaningful. Commands can be issued by waving in specific patterns. Now-defunct Neurosmith made one for their Musini toy, and it did a lot less, but it may have been an early sign (like the Samba de Amigo maracas):

This dovetails well with what I've been thinking about the long-term effects of Harry Potter on the generation of kids growing up with it, the generation of ubiquitous computing consumers (call 'em Generation U or Generation H--maybe not, that's too close to Preparation H...but I digress). It seems clear where this is going. Low-power wireless networking, distributed computing, accelerometers combined with SHARK-like gesture recognition means action at a distance with a wave of the wand. Magic.

Now consider the pick-and-drop research from Jun Rekimoto's lab at Sony. Pick-and-drop is an interaction metaphor in which people "pick" up a virtual object (such as a window) on one device with a special pen, "carry" it to another device, then "drop" it there. This works by giving every pen a unique identity and having each device query a central pen server and file server before allowing the item to display on the local machine. However, all of this happens quickly, so the effect is that of tearing off a window from one screen and plopping it on another.

If we add this functionality to the device I described above, we get a thing that works using a paradigm that closely resembles a traditional magic device. And wands are devices that a significant portion of Generation U has grown up in comfort with. As Liz points out, one of the best things about Harry Potter is how mundane, comfortable and everyday magic is (to the wizard class). In reality, in the ubicomp version the magic is the interaction between networking hardware, displays, gesture recognition, window servers, access control lists and encryption (a public key may be the only data actually stored in the wand), but that's not how it feels.

Call it heresy--user-centered design is probably not supposed to make computers seem magical, just metaphorically appropriate--but maybe Harry Potter and its sisters (such as the His Dark Materials) provides a new model of ubiquitous computer experience design. After all, it's a distillation and update of classical metaphors for a new generation, which means that it's both (excuse me the marketing speak, but maybe this one time it's appropriate) timeless and contemporary. Maybe it's time to make magic objects.

[Before anyone mentions it: yes, I remember when, in 1994, Bill Gross said that if someone wanted to read his business plan for Knowledge Adventure Worlds, all they had to do was read Snow Crash.]

Posted by at 9:59 PM | Permalink | Comments (5)

Liz and I have been playing this game of adding and crossing social trends with ubiquitous computing technologies to see where things can go. A couple of months ago, we were introduced (by new parent friends of ours) to the Happiest Baby on the Block book. One part of it is the idea that infants calm down when presented with a lot of white noise, because it's like the kind of sound they hear in the womb. You can even buy a CD of white noise that features 3 different white noise flavors:

This CD includes 3 tracks of calming white noise. First it is fast and vigorous to get a crying baby's attention. Then it is moderate to gradually guide your fussy baby to calm. Finally, it is rhythmic and womb-like to keep your infant calm and sleeping longer.

A brainstorming session on a drive led to some ideas. Here's one:

A baby-sized digital satellite (XM/Sirius) radio headphones that dynamically detects Baby's mental state (happy, crying, sleeping) based on simple noise level/audio processing and switch between XM radio stations designed to provide the appropriate sounds for the baby's internal state, anywhere on earth. A subscription model that takes the child's age into account can automatially switch between stations that are appropriate for the child's development level when selecting the right music, or special subscriptions can be made for kids who have sensory integration dysfunction, which is apparently also helped by listening to specific types of sounds.

Posted by at 1:16 PM | Permalink | Comments (3)

I wrote a piece for the July/August issue of ACM interactions. It's a special issue on ambient intelligence and I wanted to share my thoughts about designing for ambient intelligent systems. To me the key is thinking about the design of everyday objects as projections of services into tools that support aspects of those services.

Now there's no reason that services must be coupled to specific objects or places, and of all of the possibilities AmI provides, the one I think may be most compelling is its ability to decouple services from individual objects.

[...]

Treating objects as representatives of a service, rather than the service itself, is a fundamental change in our relationship with the objects of our lives, but in a way that feels like a natural extension. The service becomes the focus, and the objects its avatars: the projections of a single idea into the world, rather than each embodying a different idea. This sounds lofty, but matching service to goals may lead to better overall user experiences than trying to match tools to goals. Traditional tool design focuses on enabling concrete tasks used to fulfill abstract goals. In many cases the tool becomes a necessary burden on the way to satisfying the goal, and the design of the tool is assumed to be unable to address the goal directly. Designing a service to satisfy a goal, and then designing tools that use the service to support tasks that satisfy the goal is a potentially "cleaner" way of thinking about creating a user experience than trying to enable the goal by way of designing task-specific tools.

In other words, in a world where tools communicate, store and process knowledge, every tool should not try to address every aspect of a given need. It may be easier to design tools by first designing services that satisfy the larger needs and then make tools that are tuned to facets of the service, rather than trying to do everything. (this is, btw, probably not unlike what ID calls service design)

The full text of the article(80K PDF) is available.

Posted by at 3:08 PM | Permalink | TrackBacks (1)

Matty Sallin, a former Interaction Design Ivrea student, and Greg Trefry have made a lamp that helps you visualize your pulse:

An ornamental lamp that detects and "echos" your own heartbeat in realtime. Just place your hands on either side of the lamp base and wait 6~10 seconds for it to detect your pulse, then watch as it throbs to match your heartbeat.

A nice visual design. I find it strangely reminiscent of the judge character from Pink Floyd's The Wall or a nun's hat thing, but that's just me and it is heart-shaped, which is what I think they were going for.

Lighting seems to be a common entrypoint into the augmented furniture world right now, which makes sense: there's nothing that has to move, it IS ambient display, and it's been established (by Ingo Maurer in high design, and Pottery Barn for a more traditional audience) that it can come in many forms and still be acceptable to a large audience (not so with the sofa, for example).

Posted by at 2:24 PM | Permalink

I'm always pleased to see people using smart home technology for something other than making a really killer, expensive simulation of the local $9 movie theater. The Gator Tech Smart House for seniors is a new construction house that's

designed to assist older persons in maximizing independence and maintaining a high quality of life. To this end we are innovating Pervasive Computing technology to create a supportive and assistive environment for the elderly and the disabled.

The house has a number of typical "smart home" home automation things in it--flat panel displays, remote controls, kitchen computers that pop out recipes--but there are also some nice innovations, since they're forcing themselves to think about the whole house, not just the living room and kitchen. There's a bed that monitors sleep patterns, there are floors that detect if someone has fallen, there's a pantry to warns you if some of the food in it is rotten (something that I could have certainly used in the past), there's a tub that prevents accidental scalding and adjusts settings based on whose in it (if they do it by weight--what seems the easiest way, to me--that would be way cool...or lukewarm, if that's your preference ;-). By creating a specific target market--the elderly--they've created an interesting set of constraints that's let them tune the capabilities of the technology to specific uses, which I think is great.

An interesting quote comes from a transcript of a TV show about the house. 78 year-old Minette Hendler says "It's the house taking care of me. I'm not really alone." Once again, this shows how people project human qualities onto technology. Minette really IS going to be alone, in the literal sense, but her hope is that some of the anxieties normally associated with being alone--being out of contact, no one being able to call for help on your behalf, no one watching your food and water intake, etc.--are going to be taken care of by the technology. That's almost certainly a good thing, but what still fascinates me is how 'smart' technology changes people's relationships with the inanimate objects in their lives. I was just thinking of the From Animals to Animats intelligent agent conference series, and how animat is an interesting term for something that's between the two worlds: animated, but not an animal. A smart house, in this definition (though not in the definition that the conference people would use) is a kind of animat.

Posted by at 12:01 PM | Permalink

The Consumer Electronics Association, which (I'm guessing) sets the agenda for the development of second-generation electronics manufacturers (the ones who are making stuff that was cutting edge last year cheaper this year) has put out a report (400K PDF) which has "hybrid white goods" as a technology to watch. By "to watch" I'm assuming they mean "to make very cheaply." They define the genre by example, which shows how lacking a specific focus makes defining a technology solution difficult:

the new category features such novelties as refrigerators that come with cable-ready TV screens,refrigerators that can monitor the shelflife of your in-box items; ovens that can download and execute recipes via the Internet; and even ovens that can be temperature-controlled during the day so they can store and eventually cook food via a cell phone request while you�re still at the office.

The reasons why people would want this technology are still somewhat unclear: "the prospect of making kitchen chores more efficient [...] will likely be appealing." There's that efficiency argument again, and the solution seems to be...remote controls (that big 1970s new technology hit) for appliances:

Despite the clear benefits, many consumers are intimidated by new technologies

[...]

it�s the industry goal to connect all devices to permit remote access.

[...]

Whirlpool,for example, now is testing an upgrade to let Polara owners control their ovens via a cell phone or the Net.

Remote control doesn't seem like a clear benefit to me, but it's interesting to see the marketing gears turning in terms of how it's going to be presented:

�Our research shows that busy consumers still blame themselves when they cannot provide their families with homecooked meals."

Parental guilt + efficiency = remote control. But pressing the guilt button isn't the same thing as giving people something that they're going to embrace for the long term. There needs to be genuine utility. Fortunately, the CEA does recognize this, to some extent:

As a result,consumers may hesitate if the smart kitchen appliance seems more complicated than it needs to be. The industry�s growth could be slowed if companies put the cart in front of the horse.

(and by "complicated" they mean "more work than is justified by the functionality")

John Dvorak makes another interesting point when he says:

The true goal of smart white goods is to get advertising-oriented LCD displays into the home and feed them promotional ads over an IP connection linked from the power lines.

I don't think that the companies are thinking in those terms, it's an awfully 1999 notion, but there's truth to the idea that it's going to be hard to resist cramming as much potentially revenue-generating functionality into the things, possibly at the expense of the user experience. And ads are one that gives companies dollar signs in their eyes, since it seems like free money.

The CEA says: "The industry needs to communicate the benefits in ways that demonstrate how technology makes life simpler and more efficient." However, as long as they're stuck in thinking that it's only simplicity and efficiency that people want, and that all technological functionality needs to stem from, or be crammed into, one of those pidgeon holes, and only guilt that'll get people interested, it's going to be tough to get people to buy. Yes, those are important, but they're only a small proportion of what people want from their household tools. If that's all people were interested in, why would there be so many (labor intensive) espresso makers in the world?

Now that there's an industry-friendly name for the idea and an official blessing, it's much more likely to get funded as part of R&D efforts. I hope they come up with some fuctionality for the technology other than remote-control fridges.

[in related news, iRobot just announced the Scooba a hard floor sibling to their Roomba vacuum cleaner. What's interesting is that these are appliances--white goods--that aren't based on existing white goods, but on much simpler household goods, and they're focused not on remote control, but on work elimination]

Posted by at 8:54 AM | Permalink

CNN is covering a Steelcase and IBM project out of IBM's Watson lab, Bluespace. Looking at the Bluespace site, things seem quiet, but the article has some interesting ideas in it:

[...]sensors and displays embedded in the furniture, which know when you arrive in the office and will automatically bring up your computer settings.

The wallpaper, or images on walls at least, will change color and pattern depending on your mood and preferences, even letting colleagues know whether you can be interrupted.

Most of the stuff they mention here, although they're talking about 2020, exists today.

One thing that troubles me about the article is that it implies that they're thinking of designing technologies for the office primarily from the perspective of the company's goals, rather than the goals of the individuals using it:

"Workers' productivity is the ultimate goal. We want to use technology in such a way to make workers as productive as possible."

Yeah, yeah, sure, sure, worker productivity is good, but that's not how people think about their jobs. Many failed technologies show that there need to be incentives for people to use the fancy tech on a day-to-day basis, or else it ends up making the organization as a whole less productive, as people struggle with the technology and central command creates new processes that require its use (because otherwise people don't use it). Bluespace does recognize this, to some extent, and they believe that this incentive is going to come in the form of attention-focusing technology:

"We're being bombarded, so we have to find a way -- we, the creators of technology -- to make this technology more aware of what the knowledge worker is doing so that we're not interrupting him or her at inopportune moments," she said.

I agree with the identification of the problem, but their solution, which is to make any surface a display (not an ambient display, but a replacement for screens), doesn't seem right:

A device called the "Everywhere Display Projector," a combination of a projector and computer vision technology, is used at Bluespace.

It projects information from a computer on to any surface, including, floors, desktops and chairs.

"The vision technology allows that surface to become interactive, so that while the computer is projecting the information on to, say, your table top, the worker can interact with that the thing which is basically just light by using light," Lai said.

This seems like it just means that there's no place you can get away from your work. The technology, as presented here, arbitrarily blurs the lines between work areas, rather than to defining them in a useful way.

They're also, somewhat surprisingly, working on detecting the emotions of workers:

"Once we get this emotion detection correct we can very quickly translate that into what we're projecting for wallpaper," Lai said.

That last part about emotion detection seems, uh, a bit out of left field relative to the other technologies mentioned. Maybe it's an attempt to bring a technological solution looking for a problem down to human scale. Frankly, I think it's potentially the most important part of the whole research, and by far the hardest, but I worry that because it doesn't instantly translate into hardware, it will probably get the fewest resources. That's kind of a shame.

Posted by at 11:04 AM | Permalink

Herman Miller Inc. wants to make it easier for private telephone conversations to remain private.

A device called Babble is the office furniture company's first foray into high-tech electronics. The wireless box duplicates and disassembles a user's voice before broadcasting it through a series of speakers to make phone conversations unintelligible to passers-by.

[...]

DeKruif said the burgeoning health care industry is the Zeeland-based company's most obvious customer for Babble.

"The industry has done a great job of managing paper, but there's nothing on the market to mask voice conversations," he said. "To this point, what we've seen is a piece of yellow tape on the floor and a sign that says "stand back.' That doesn't work very well."

DeKruif said the company expects Babble to also be attractive to lawyers and corporations that need to protect proprietary information in open work spaces.

A second generation Babble that protects face-to-face conversations in a similar fashion is in the works and will be marketed by Herman Miller.

(from this article)

I think that it's interesting H-M is moving away from just furnishing offices to thinking about the relationship the objects they make have, and can have, on the office environment, and I think it's interesting that they're thinking of specific markets and their needs. It seems intersting, but a product that is likely to be driven by the needs of upper management in a company, rather than the needs of the people on who are going to be using it, which I'm sure is going to limit its popularity.

Posted by at 11:51 PM | Permalink | TrackBacks (1)

Intel realizes that the context technology is presented in is important, specifically furniture:

Intel, the world�s largest chip maker, today unveiled a prototype designer furnishing for a new generation of home PC - The Ryan McElhinney Home Entertainment Shell for Intel.

Nicknamed the �E-Shell�, the Home Entertainment Shell showcases today�s entertainment PC capabilities in a modernist �60�s designer housing fit to grace the most stylish living spaces.

[...]

Inspired by Eero Aarnio�s iconic bubble and ball chairs of the early 1960s, the �E-Shell� harks back to an age when just two or three simple items of technology shared equal space in the living room with exciting new furniture designs and futuristic materials.

In contrast, a study into the 21st Century living room by Intel reveals that 42 percent of Brits complain technology hardware is now crowding them out of their rooms, with an average of five remote controls to cater for home entertainment needs. One in four living rooms (or 25 percent) are stacking up more than seven separate technology devices.

Despite recognising the benefits of having one multi-function entertainment device, half of British households (49 percent) claimed they wouldn�t allow a traditional PC near their living room because of its design shortfalls.

[Sounds like an attempt to recapture midcentury Italian futurism--the last time there was a unabashedly positive view of the technological future...but I digress, while feeling somewhat vindicated]

(from this press release)

[Update:
more info and pictures:

]

Posted by at 11:53 AM | Permalink

I was looking at the excellent Color Kinetics LED wall I saw at the Milan Furniture Fair:

and thinking about ambient display, when an idea popped into my head: what about an infrared LED dot matrix screen? On one hand it does come off as a kind of ironic "work that's in front of your eyes, but you can't see it" conceptual piece, such as the paintings in boxes that Art and Language did at one point. In those, there's a box and they tell you there's a painting inside it, but you're not allowed to open the box, so the painting is there, but you can't see it. On the other hand, I think that there's are legitimately interesting uses for a data panel that's only visible to technology, and yet is still within human line of sight.

Unlike wireless technology, an IR LED wall would have to be within eyeshot in order to be used. This impacts both security and it cuts down on the potential crosstalk of a bunch of wireless devices talking on the same set of channels creates. It also has the quality that it's effectively invisible unless someone wanted to see what it was broadcasting, thus moving control back into the user's sphere. The way you would access it would be to look at it through an IR-sensitive device, such as a cell phone camera (most CCDs are sensitive to near infrared). This creates a kind of effect, where your special glasses let you see information that's hidden in plain sight all around you. The ability to do that may be more comforting to people than invisible radio waves (assuming that's important).

Posted by at 6:55 PM | Permalink

The Call for Proposals for C4F3, the cafe project I'm co-chair of for ISEA is up.

This is THE big project for me for the next year and a half. It's designed to be the first large scale public space of fully-functional objects augmented by information processing technology. I'm hoping that we can create an environment where everything--from the furniture, to the walls, to the dishes--is somehow changed through active information processing. And, unlike an R&D lab space, it's supposed to be fun, interesting and actively used by the visitors to the ISEA2006/ZeroOne electronic art festival, so there is also room for humor and entertainment. I'm really excited about the possibilities, so much so that I'm tempted to gush, but I'm going to restrain myself and suggest that you look at CFP, spread the word and submit a proposal.

Posted by at 2:52 PM | Permalink | TrackBacks (1)

At the 2005 IA Summit in Montreal last week, I presented a talk on ubiquitous computing. Here's the abstract:

Ubiquitous computing, ubicomp, is the introduction of information-processing devices into the background of people's lives. Unprecedented networking and computational power and miniaturization define a ubicomp future that presents challenges which go beyond the expertise of traditional design disciplines. These devices will be more flexible than those created by traditional industrial designers, yet more narrowly task-oriented than general-purpose computers and software. Thus, just as the definition of experience design starts to stabilize, ubicomp poses new questions about information presentation and organization.

What do "navigation" and "search" mean in environments where your personal information cloud includes not just your laptop and phone, but your running shoes, your pacemaker and the sidewalk you're jogging on?
How do we share knowledge where there are no screens or keyboards?
How do we maintain our users' ethical prerogatives of privacy and choice?

This presentation will cover issues specifically addressing the ways that information architecture is a critical component of ubicomp user experience.

The talk is available as a 700K PDF of my PowerPoint, (drop me a note and let me know if you want the actual PPT). It's mostly pictures and is missing the closing Betty Boop cartoon, but the ideas are there, if skeletal.

[Andrew reminds me: Adam's original essay, which served as impetus and original title for this talk but which is a much deeper examination of the ethical in issues ubicomp, is available on Boxes and Arrows.]

Posted by at 12:50 PM | Permalink | Comments (1)

I think this is great: a book-shaped network storage device. What's great about it is not that it's book-shaped, but that it's designed to act like a book. That shows an understanding of how to incorporate technology appliances into people's environments and a valuable way of humanizing this technology through analogy. Plus, the engineers probably conceded optimal functionality (neighboring books will probably block some of the radio waves, and heat dissipation is certainly worse than in a standalone device) in favor of a better user experience, which is a tradeoff that should happen more often.

Here's a picture of an earlier one from the same company that has a more book-like design, but doesn't sit upright like a book:

Posted by at 8:45 AM | Permalink

As part of the Design Engaged conference last November there was a day when we were broken up into groups and encouraged to walk around Amsterdam and brainstorm ideas about design, technology, cities and human interaction. It was a pretty broad mandate, but we shared a similar set of interests, so many of the ideas resonates and we quickly assembled a bunch of interesting ideas about new technologies. During that exercise, I started thinking about heat maps as ways of representing things other than heat.

Heat maps are maps like the heat weather maps that appear in newspapers. They represent shifting gradients of temperature, as mapped to geography. In the newspaper they consolidate a lot of potential information (think of the listing of temperatures and cities that scrolls on CNN International) by keeping one variable (geography) stable and mapping another one to it. I thought "I wonder what else can be mapped like that?" and, of course, WiFi was the immediate choice, since I'm constantly checking for WiFi strength when sitting in cafes. Mapping WiFi to heat maps has been done by several groups, with the following map taken from Chris Lentz's 2003 Dartmouth Senior Thesis as one example:

My next thought was "OK, assuming that someone is going to do some semi-automated way of doing this based on the Netstumbler database [link[, what to do with this information?"
Then, I thought of all of the other information that's already bombarding me as I walk down the street looking at potential places to sit with my laptop: I often have headphones in my ears, or I'm talking on the phone, I'm looking out for traffic and reading menus, I'm trying to find a place with a table near the window. Finding the strength of WiFi connectivity is a secondary consideration at that point, and is always going to be, but the information is there, so how to get to it?
OK, I thought, so our visual sense is totally overloaded, and our auditory sense is pretty close to it, but we have a bunch of other senses that are perfectly good and relatively underused. We can map information to these other senses. One of these is heat, so why not reverse the polarity, so to speak, and take the WiFi heat map and map it back to actual heat.

That's the long windup for the pitch of the following idea:

The WiFi heat sensor

This type of shifting of data to a non-visual/non-auditory sense could have all kinds of other possibilities. Connected to a GPS (or some other way to find your place) with a database, it could tell you where there was an ATM, or a point of historic interest. Connected to a Lovegetty-like (or "Familiar Stranger"-like) device, it could tell you when there was an acquaintance nearby. It would give you a sense of the data space around you in a way that only you would know about.

My next step is to make a working one. No. Really. I have the Peltier Junctions and a hackable WiFi detector. Spidey sense here we come!

Posted by at 8:01 PM | Permalink | Comments (4)

Another smart carpet project as blogged by We Make Money Not Art (who is astoundingly, shockingly, depressingly prolific in her blogging talents--I can only hope she's a grad students procrastinating from doing her dissertations; if she's not, then she can't possibly be getting anything done and if she is, then I may as well go back to streetsweeping).

Posted by at 9:18 PM | Permalink | Comments (1)

I'm co-organizing a workshop at the Pervasive 2005 conference in May with Lucia Terrenghi, Irma Lindt and Andreas Butz. We just extended the deadline and I figured I'd announce it again.

Here's the announcement:

Paper Submission Deadline: 1st March, 2005

WORKSHOP ON USER EXPERIENCE DESIGN FOR PERVASIVE COMPUTING

http://www.fluidum.org/events/experience05/

Associated with the Pervasive 2005 Conference (http://www.pervasive2005.org)

12 May 2005, Munich, Germany

Organized by Lucia Terrenghi, Irma Lindt, Andreas Butz, Mike Kuniavsky

---

Experience design is a design approach which focuses on the quality of the user experience during the whole period of engagement with a product: from the first approach, through its usage, to the reflection and memory of the complete relationship.

As technologies for wireless networks, image capture, storage and display get cheaper and more performing, and as the internet drives up the availability of a pervasive information and communication infrastructure, it becomes possible to embed computing capabilities into a variety of environments and bring communication in a much broader set of contexts. Thus, pervasive computing and context sensitive systems allow for the design of new stimuli from which people could create their own meaningful experiences, individual or shareable. These goals raise new challenges, suggesting the need of new methods and forms of interaction patterns between users and environments, and between different groups of users. Design can play a key role in shaping new toolkits for contextualized experiences, and enhance the natural evolutions of users' sense of place and time towards the experience of living in a mixed reality, in which physical and virtual spaces are blending together, and social relationships become fluid and distributed.

---

Submissions:

You are kindly invited to post a position paper no longer than 4 pages describing your work and interests. Submissions must be in Adobe PDF format and should conform to the Springer-Verlag LNCS style. Themes that are relevant for this workshop include, but are not limited to:

• interaction design for pervasive computing: thus addressing new scenarios for pervasive computing, interaction within instrumented environments, multi-user interaction and interfaces, multimodal interfaces, tangible user interfaces, interaction paradigms.
• design for experience management: techniques and activity theory approaches focusing on how to stimulate and support users in creating their experience within a pervasive scenario, how to engage them within the experience, how to collect, store, reflect on, share experiences.
• user experience evaluation: user experience taxonomies, understanding of users' needs and specification of requirements, evaluation methods and assessment approaches.
• design of toolkits for the authoring of blended experiences: design for ambiguity, allowing for a transparency of the infrastructure that can stimulate and support users' and designers' creativity while making a semantic relationship between physical and virtual spaces of a mixed reality.

---

Contacts:

For further information visit the page
http://www.fluidum.org/events/experience05/

for contact and submission
lucia.terrenghi@ifi.lmu.de

Posted by at 10:57 AM | Permalink

Liz and I had a conversation yesterday based on my post about Neil Gershenfeld's book; specifically the part about the Web distracting people from ubicomp for most of the late 90s and early 00s.

Her point is that the Web had two big advantages over ubicomp in 1997 (whether or not it actually took resources away from ubicomp--it's not a zero-sum game):

• The barrier to entry was low. Hardware is hard. HTML is easy.
• The Web was about bringing people together, ubicomp was about making cool stuff.

The second point is really important, I think. Something that I've always talked about in UI design, and seem to have forgotten in my recent excitement with physical computing and the embedding of computation in everyday objects, is that people's ultimate goals in using any tool is to communicate with other people. There's little that's done solely by yourself, for yourself. That's a pretty boring closed system...and it's boring by definition since, by definition, we find purely solitary behavior kinda boring, on the whole.

Thus, forgetting that smart objects need to be in the service of social effects is a surefire way of making stuff that will be popular to a narrow niche, at best. An important lesson in design. Thanks, Liz!

Posted by at 1:19 PM | Permalink | Comments (1)

Homebound with a cold in the unusually sunny Portland winter, I read Neil Gershenfled's book, "When things start to think." It's an interesting book, and still relevant to the ubicomp world, even through it was published six years ago and he was writing it eight years ago. It's an autobiography, an introduction to technological and scientific concepts, wide-eyed speculation, a polemic against everything Gershenfeld doesn't like, and a big ad for the MIT Media Lab and the students there. All in 200 pages. Sometimes, it's all of those things in a single chapter, as when he starts by talking about telemarketing, moves into a detailed history of the Protestant Reformation, slides over to the Bill of Rights and end up creating his own version of Isimov's Three Laws of Robotics. It's not as much about making deep connections as it is about associative idea surfing. It's sitting with Gershenfeld while he gives you a braindump of everything that's on his ADD mind. It's quite blog-like, actually, and if blogs had existed in 1997 that may have been a more appropriate forum than a book.

The most interesting thing about the book, which is on the whole about ubiquitous computing--even though Gershenfled doesn't like the term--is how little progress has been made since he was writing 8 years ago. Over and over I see people re-discovering the same ideas he was talking about in the book, which were old to him even then, as if they were something amazing and new. His iconic image is the shoe computer. When did the first real shoe computer come out? 2004 (the Adidas 1).

Why did this happen? I blame The Web. The Web sucked up the "best and brightest" for 10 years, and only now are people getting bored enough with it to start thinking about hardware. The pieces are there:

• Chinese manufacturing (which everyone has heard about and owns at least one product that benefitted from) is there to make it.
  
• The DIY movement has spawned a bunch of interest into what goes into the black boxes.
  
• And the ephemerality and increasing sameness of the Web is pushing smart people toward physical objects (confirming what Chicken John said years ago: "All these dotcom people, you know what they really want to be doing? They want to be working with wood.").
  

This book is an interesting reminder of where all of this came from, and how it's neither new nor revolutionary. However, reading it shows me how important momentum and timing are: Gershenfeld was first in many ways, but without the support of companies and individuals running with the ideas, they stagnated, and only now are we picking up the crumbs of 1997.

The end of the book introduces Gershenfeld's Things That Think Consortium. Gershenfeld is no longer a director of that program, though I get the feeling he was at the time.

Posted by at 1:28 PM | Permalink

In a 1996 essay (Word document) Mark Weiser, the person who coined the term ubiquitous computing and who really defned the space, predicts:

The "Smart House" of 1955 dared to put a TV and a telephone in every room. And the "Smart House" of 2005 will have computers in every room. But what will they do?

I have yet to see a good consumer-oriented kitchen computing device, and we still mostly have dumb toilets. So is the revolution in personal ubicomp? Did the jump go from computers sitting on desks to intimate computing? iPods and cameraphones are way more popular than computers in the kitchen.

That said, it's only January, but--as always with Weiser--it's good to go back and see what he was thinking.

We will dwell with these computers, whose presence we will ignore most of the time, and they will provide us with constant clues about our environment, our loved ones, our own past, the objects around us and the world beyond our home. Computers will act like books, windows, walks around the block, phone calls to relatives. They won't replace these, but augment them, make them easier, more fun.

And, finally, some advice against focusing on technology, when it's not the technology that's interesting, it's the distractions it removes that--in his, and my view--provides the ultimate value:

We become smarter as we put our roots deeper into what is around us. The house of the future will become one giant connection to the world-- quietly and unobtrusively, as naturally as we know it is raining, or cold, or that someone is up before us in the kitchen making breakfast.

Ubiquitous computing just might help to
free our minds from unnecessary work, and
connect us to the fundamental challenge
that humans have always had:
to understand the patterns in the
universe and ourselves within them.

Posted by at 5:33 PM | Permalink

Found here through ACM's technews:

The new �attentive and more considerate� office cubicle helps increase work focus for those who share space with many others. It automatically �mediates audiovisual communications� between co-workers by using information gained about their �social orientation� in an office, says Dr. Vertegaal.

[...]

The attentive cubicle�s walls are constructed of a translucent material called Privacy Glass� that consists of a glass pane with an embedded layer of liquid crystals. Overhead cameras mounted in the ceiling track the �social geometry� between co-workers. When potential communication partners are detected, the cubicle�s walls automatically change from opaque to transparent, allowing for visual interaction.

A little creepy--"HAL, please let me talk to Bob." "Dave, I'm afraid I can't do that."--but interesting.

Posted by at 8:46 PM | Permalink

William sends a link to a short story about smart furniture and ubicomp.

Blebs had been around for about twenty years now, almost as long as I had been alive. Their roots could be traced back to several decisions made by manufacturers�decisions which, separately, were completely intelligent, foresighted, and well conceived, but which, synergistically, had caused unintended consequences�and to one insidious hack.

The first decision had been to implant silicon RFID chips into every appliance and product and consumable sold. These first chips, small as a flake of pepper, were simple transceivers that merely aided inventory tracking and retail sales by announcing to any suitable device the product's specs and location. But when new generations of chips using adaptive circuitry had gotten cheaper and more plentiful, industry had decided to install them in place of the simpler tags.

At that point millions of common, everyday objects�your toothbrush, your coffee maker, your shoes, the box of cereal on your shelf�began to exhibit massive processing power and interobject communication. Your wristwatch could monitor your sweat and tell your refrigerator to brew up some electrolyte-replenishing drink. Your bedsheets could inform the clothes-washer of the right settings to get them the cleanest. (The circuitry of the newest chips was built out of undamageable and pliable buckytubes.) So far, so good. Life was made easier for everyone.

Then came the Volition Bug.

The story also has many animist elements to it. I've written about animist reactions to ubicomp before, but it's nice to see it as part of a story, since that brings the point home even better. There's also a nod to the Power Tool Drag Races.

Posted by at 1:08 PM | Permalink

I was visiting my friends Moses and Lucie and their kids, Felix and Milo, in Berkeley last week. We went to a public playground near their house that had all kinds of toys scattered about (it's tellingly Berkeley that the toys don't walk away and parents regularly bring more for everyone to share). The thought of a store based on that model occurred to me: have a mall store--malls, of course are primary suburban safe play spaces for kids--where parents swipe their credit card to get themselves and their kids in. They get one of those amusement park or hospital-like plastic wristbands. Inside, it's a large open playspace and cafe, with lots of toys for the kids to play with, and new ones constantly being added. Each toy has an RFID tag in it. If a child wants the toy, falls in love with the toy, then the parents and kids just walk out with it--the RFID tags automatically charge the price of the toy to the credit card the parents swiped to get in, based on proximity with their wristband RFID.

I can see that there are a bunch of potential logistical and financial model problems with this (there's a reason that toys are kept in sturdy packages in toy stores, after all, and I bet it's primarily because of the margins). However, in terms of how a retail model can use technology and change to compete experientially in a realm where straight retail is now difficult, it seemed interesting to me.

Posted by at 10:58 AM | Permalink | Comments (2)

Here's a pretty hilarious story from Sync magazine, where Jake Bronstein--the author--tries to outfit his house with as many domestic robots as possible. I won't give away what happens, but his conclusion is that maybe robots that look like dogs and people aren't the future of robotics:

If I had to place a bet, I'd say when the robots take over the world, they won't look like anything we've seen in the movies. They'll probably look more like sinks. Or deep fryers. Or toilets.

I know I'm biased, but he's right.

Posted by at 12:37 AM | Permalink

A few weeks, old, but Victor sent me this link (to an Engadget piece) on a smart baby cradle.

Beds are a good platform for introducing technology and, more imporantly, young parents are a prime market for gadgets that make their infants safer or make their lives easier, so this makes a lot of sense. The amount of money that goes to kid tech is pretty huge, and with the continuing shift toward greater shared responsibility between parents, more dads--a prime technology consumer--are buying toys for their kids. Why else would there be $2000 titanium and leather baby strollers? (though, granted, that was a limited edition from Maclaren--but they still sold 1000--that's $2 million of titanium baby strollers sold!) Of course parents are also intensely cost-conscious as they realize how they've underestimated the costs of Baby, so technology needs to be cheaper (Neurosmith, the toy maker, probably went under because their very innovative technology toys had a price point of about $40, rather than the $20 or whatever parents were willing to pay for the technology), but technology is constantly getting cheaper, so it's a matter of time before the lines cross.

Posted by at 12:32 PM | Permalink

From boingboing by way of Cassidy, the gravity lamp from Front Design in Stockholm. It has a sensor and some very basic "robotics" in it (I think it pulls some wires and the rods straighten like tentpoles), but it's cute. This is the same group that did a memorable non-tech conceptual furniture piece I saw in Milan: it looks like the vase version of Duchamp's "Nude Descending a Staircase" (to see it, go to their site, click on Projects, then Design By, then Motion). They've also done some other clever technology-based design, including a robotic table that figures out how to stand, and an amusing 3D version of the glitchcore music philosophy (that's the one where the byproducts of digital recording are used as the basis of new music--Matmos and Oval are probably the ones who do it best) in the form of a candle holder, as misrendered by a rapid prototyping machine.

Cool stuff.

Posted by at 3:01 PM | Permalink | Comments (1)

Slashdot brings a link to a fake window made of LCD panels. Superficially, this is another wacky outgrowth of the casemod world, judging by the presentation of the how the thing was put together (it's classic casemod style, complete with random anime babe desktop pattern), but I think it's also an interesting interaction between the casemod world and the ambient display world. He could easily adapt it to be like the now-classic Ljungblad and Holmquist Mondrian Ambient Weather Display and present literal content, leveraging the domestic context.

OK, that's starting to sound convoluted. I guess what I mean is that it fits into a tritely domestic setting much more easily than the minimal Modernism of a lot of ambient display, and that makes it much more likely to be accepted by the large group of people who find even IKEA Modernism too cold. Familiarity is important when designing objects for the mass home market, and this feels very familiar, even if it's actually a pretty profoundly weird thing, the inverse of the old bricked-up window.

Posted by at 7:27 PM | Permalink

Sorry to beat this dead horse, but several people have recenly asked me for the version of the Smart Furniture Manifesto that was published in Metropolis Magazine in June. It's one of the articles from that issue that they didn't put online, so I can't just point to it. It's philosophically similar to what I wrote last year, but it expands and clarifies the ideas somewhat.