Making Digital Tangible: The Battle Against the “Pixel Empire”

SIGCHI Lifetime Research Award Lecture
CHI 2019 in Glasgow, UK, May 6th, 2019
Hiroshi Ishii
MIT Media Lab
Tangible Media Group

【生涯研究賞】Thanks to ACM SIGCHI for the SIGCIH Lifetime Research Award! Here are my award lecture (May 6th, 2019 in Glasgow) video, slides, and news.
【VIDEO】 https://vimeo.com/348062352
【SLIDES】 https://speakerdeck.com/ishii_mit/making-digital-tangible-the-battle-against-the-pixel-empire
【PHOTOS】https://www.facebook.com/ishii.mit/media_set?set=a.10157219676408544&type=3
https://www.facebook.com/ishii.mit/media_set?set=a.10157225616253544&type=3
https://www.facebook.com/ishii.mit/media_set?set=a.10157219791028544&type=3
【NEWS】 https://sigchi.org/awards/sigchi-award-recipients/2019-sigchi-awards/
http://news.mit.edu/2019/mit-professor-hiroshi-ishii-wins-acm-sigchi-lifetime-research-award-0220
https://www.media.mit.edu/posts/tangible-influence-a-stroll-through-hiroshi-ishii-s-living-legacy/

Transcript

1. Making Digital Tangible The Battle Against the “Pixel Empire” SIGCHI Lifetime Research Award Lecture CHI 2019 in Glasgow, UK, May 6th, 2019 Hiroshi Ishii MIT Media Lab Tangible Media Photo courtesy of Nobukazu Kuriki @ishii_mit ishii.mit
2. Photo courtesy of Nobukazu Kuriki thanks a million! My sincere appreciation to the Tangible Media Group & MIT Media Lab students over the past 25 years, the CHI, TEI, UIST, CSCW & SIGGRAPH communities over the past 30 years, and the NTT Human Interface Lab colleagues from the early 90s!
3. October 2015 MIT Media Lab 30th Anniversary Tangible Media Group 20th Anniversary
4. CHI 1990 Prof. Bill Buxton & Prof. Marilyn Mantei CHI 1997 Tangible Bits with Brygg Ullmer CHI 1992 ClearBoard with Minoru Kobayashi CHI 1998 Dr. Douglas Engelbart
5. CHI 2006 CHI Academy: SIGCHI Award Banquet CHI 2016 Materiable & Media Lab Party CHI 2017 Prof. Ben Shneiderman, Prof. Jun Rekimoto, … CHI 2018 Prof. Sean Follmer, Prof. Lining Yao & party! CHI 2015 bioLogic & TRANSFORM
6. SIGCHI Award SIGCHI Award SIGCHI Award CHI Community The Human-Computer Interaction (HCI) Pioneers Project by Prof. Ben Shneiderman https://hcipioneers.wordpress.com Prof. Ben Shneiderman
7. My Heroes & Gurus Mark Weiser 1952 - 1999 Douglas Engelbart 1925 - 2013 William Mitchell 1944 - 2010
8. Evolution Photo courtesy of Nobukazu Kuriki 2006 CHI Academy Tangible Bits @ MIT Media Lab (1995-2010) 2000 NTT ICC, Tokyo “Tangible Bits” Exhibition 1997 Tangible Bits Paper at CHI ‘97 2001-04 Ars Electronica, Linz “Getting in Touch” Exhibition 1990 2000 2010 2020 2016-19 Ars Electronica, Linz “Radical Atoms” Exhibition 2019 SIGCHI Lifetime Research Award Radical Atoms @ MIT Media Lab (2005-….) 1995 NTT! MIT Seamless Media @ NTT HI Labs (1990-95) 1992-93 Univ. of Toronto
9. https://foursquare.com/v/nttԣਢլݚڀ։ൃηϯλ/ 4b62542ef964a52097422ae3?openPhotoId=597ec130898bdc5ce5364cd8 1980-1995 NTT Yokosuka ECL & NTT Human Interface Labs NTT envisioned the future of an “Information Superhighway” enabled by B-ISDN (Broadband Integrated Services Digital Network) and ATM (Asynchronous Transfer Mode) Technologies in the 1980’s-90’s. My team at NTT Human Interface Labs focused on the creation of a new bi-directional realtime video-based distributed collaboration service, leading to the inventions of TeamWorkStation (1990) and ClearBoard (1992).
10. TeamWorkStation Hiroshi Ishii & Kazuho Arita CSCW 90, CACM 91, CACM 94 1990 1990
11. TeamWorkStation Translucent Video Overlay of Desktop and Computer Screen for Remote & Realtime Collaboration • Seamlessness (continuity) with existing work practices (desktop & computers) • Seamlessness (smooth transition) between individual and shared workspaces Hiroshi Ishii & Kazuho Arita, et.al. NTT Human Interface Labs CSCW 90, CACM 91, CACM 94
12. 1992 ClearBoard-1 NTT Human Interface Labs with Minoru Kobayashi, et.al. CHI 92, CSCW 92
13. ClearBoard • Seamless Integration of Inter-Personal and Shared Work Space • Gaze-awareness Hiroshi Ishii & Minoru Kobayashi, NTT Human Interface Labs CHI ’92, CSCW ’92, TOIS ’93, CACM ’94
14. Sources of Inspiration • VideoDraw by John C. Tang & Scott L. Minneman, Xerox PARC John C. Tang and Scott L. Minneman. 1990. VideoDraw: a video interface for collaborative drawing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '90), Jane Carrasco Chew and John Whiteside (Eds.). ACM, New York, NY, USA, 313-320. • Colab Meeting Room by Mark Stefik, et.al. Xerox PARC Mark Stefik, Gregg Foster, Daniel G. Bobrow, Kenneth Kahn, Stan Lanning, and Lucy Suchman. 1987. Beyond the chalkboard: computer support for collaboration and problem solving in meetings. Commun. ACM 30, 1 (January 1987), 32-47. • NLS by Douglas Engelbart, Bootstrap Institute D. Engelbart. 1988. The augmented knowledge workshop. In A history of personal workstations, Adele Goldberg (Ed.). ACM, New York, NY, USA 185-248
15. 1994 Atlanta Vanguard Conf
16. 1994 Vanguard Conference in Atlanta Alan Kay Nicholas Negroponte
17. https://www.lwa-architects.com/project/mit-media-lab/ MIT Media Lab 1995
18. Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp @ishii_mit The reason I chose to come to MIT… It was a mountain so high that the peak was beyond the clouds. There seemed to be no path leading to the top. But… (cont.) Mountain
19. Nicholas Negroponte “REBOOT!” Feb. 10th, 1995 at MIT Media Lab
21. October 1995- MIT Media Lab
22. October 1995- MIT Media Lab
23. Strategy https://www.nasa.gov/launchschedule/ main stream GUI • intangible: visual (eyes) • remote control • single user • general purpose new stream TUI • tangible: tactile & kinesthetic (hands & body) • direct manipulation • multi-user, multi-hands • special purpose https://www.art.com/products/p34962790556- sa-i9385929/zeppelin-airship-caught-in- searchlights-during-a-bombing-raid-over- england-1916.htm
24. Tangible Media MIT Media Lab 1995 Photo Credit: Flavia Sparacino
25. Make Digital Tangible Photo Credit: Flavia Sparacino
26. Invent new tangible interactions that inspire and engage people Photo Credit: Flavia Sparacino
27. MIT Media Lab FRAMES 1996 #54 TMG members 1996 Photo Credit: Webb Chappell MIT
28. Tangible Bits graspable media & ambient media foreground (center) & background (periphery) MIT Media Lab FRAMES 1996 “Getting in Touch with the Digital World” Photo Credit: Webb Chappell
29. ABACUS: The Origin of Tangible Bits 1960 2004 2009 PERVASIVE 2004 in Vienna Tokyo, Japan AXIS magazine Vol. 142 ABACUS
30. http://en.wikipedia.org/wiki/File:Grand_orrery_in_Putnam_Gallery,_2009-11-24.jpg
31. Orrery Tangible Representation of Knowledge handle
32. hands — handle
33. A Philosopher Giving a Lecture on the Orrery (sometimes called simply The Orrery) is a painting (oil on canvas, ca. 1766) by Joseph Wright of Derby depicting a public lecture about a model solar system, with a lamp—in place of the sun—illuminating the faces of the audience. http:// en.wikipedia.org/?title=Portal:History_of_science/Previous_pictures#/media/File:Wright_of_Derby,_The_Orrery.jpg collaboration
34. tangible aesthetic interactive
35. tangible - embody aesthetic - inspire interactive - engage
36. Tangible Bits CHI ’97 paper 1997 Photo Credit: Flavia Sparacino
37. Tangible Bits: Towards Seamless Interfaces between People, Bits March 1997 Presented at CHI ‘97 in Atlanta http://dl.acm.org/citation.cfm?id=258715
38. Mark Weiser 1952-1999 “Ubiquitous Computing” "The Computer for the 21st Century" - Scientific American Special Issue on Communications, Computers, and Networks, September, 1991 "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” Mark Weiser
39. Mark Weiser’s message Jan. 26, 1997
 Date: Sun, 26 Jan 1997 23:34:10 PST
 To: ishii@media.mit.edu, ullmer@media.mit.edu
 From: Mark Weiser <weiser@xerox.com>
 Subject: "Tangible Bits" Dear Hiroshi and Brygg, I recently had a chance to read your CHI 97 paper "Tangible Bits"! 
 Great work! In my opinion this is the kind of work that will characterize the technological landscape in the twenty-first century…. I do have a request. My request is that you help me stop the spread of misunderstanding of ubiquitous computing based simply on its name… "Tangible Bits" is very nice, and maybe could serve as an overall umbrella, but then you might lose it as the name of your research project! …
40. Sources of Inspiration 1 • Ubiquitous Computing (1991) by Mark Weiser Mark Weiser, The Computer for the 21st Century. Scientific American. September 1991. • DigitalDesk (1991) by Pierre Wellner, Rank Xerox EuroPARC, Cambridge, UK Pierre Wellner. 1993. Interacting with paper on the DigitalDesk. Commun. ACM 36, 7 (July 1993), 87-96. Pierre Wellner. 1991. The DigitalDesk calculator: tangible manipulation on a desk top display. In Proceedings of the 4th annual ACM symposium on User interface software and technology (UIST '91). ACM, New York, NY, USA, 27-33. • Dangling String (1995) by Natalie Jeremijenko, Xerox PARC, CA, U.S.A. Mark Weiser and John Seely Brown, Designing Calm Technology, Xerox PARC December 21, 1995 https://people.csail.mit.edu/rudolph/Teaching/weiser.pdf
41. Sources of Inspiration 2 • Marble Answering Machine (1992) by Durrell Bishop, RCA, London, UK Durrell Bishop. 2009. Visualising and physicalising the intangible product: "What happened to that bloke who designed the marble answer machine?". In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI '09). ACM, New York, NY, USA, Article 2, 1 pages. • Graspable UI (1995) by George Fitzmaurice, et.al., U. of Toronto, Canada George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), Irvin R. Katz, Robert Mack, Linn Marks, Mary Beth Rosson, and Jakob Nielsen (Eds.). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 442-449.
42. Tangible Bits Physical embodiment of 
 digital information and 
 computation (CHI ’97) graspable media ambient media
43. Painted Bits (GUI) and Tangible Bits (TUI) Graphical User Interface • Intangible representation (pixels on a screen & sound) • Generic input devices as “remote-controllers” Tangible User Interface • Tangible representation as an interactive control mechanism to manipulate the information directly with hands Urp running on the Sensetable Xerox Star
44. Painted Bits (GUI) and Tangible Bits (TUI) Hiroshi Ishii. 2008. Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI '08). ACM, New York, NY, USA, xv-xxv. DOI=http://dx.doi.org/10.1145/1347390.1347392
45. THE PHYSICAL WORLD THE DIGITAL WORLD GUI TUI RADICAL ATOMS PAINTED BITS TANGIBLE BITS A Graphical User Interfaces only let users see digital information through a screen, as if looking through a surface of the water. We interact with the forms below through remote controls such as a mouse, a keyboard or a touch screen. A Tangible User Interface is like an iceberg: there is a portion of the digital that emerges beyond the surface of the water - into the physical realm - that acts as physical manifestations of computation, allowing us to directly in- teract with the ‘tip of the iceberg.’ Radical Atoms is our vision for the future of interaction with hypothetical dynamic materials, in which all digital information has physical manifesta- tion so that we can interact directly with it - as if the iceberg had risen from the depths to reveal its sunken mass.
46. Tangible Bits embody digital information to interact with directly with hands GUI TUI 1997 painted bits tangible bits
47. Evolution of Tangible Bits 1 Background - Ambient Media graspable media Center Periphery Foreground Background ambient media ambientROOM (CHI ’97, ’98) pinwheels & water lamp (CoBuild ‘98, CHI ’01) metaDESK (CHI ’97, UIST ’97) Foreground - Graspable Media inTouch (CHI ’97, CSCW ’98) curlybot (SIGGRAPH ’99, CHI ’00) topobo (CHI ’04, ‘08) I/O coincident actuated tangibles topobo topobo (CHI ’04, ‘08) Triangles (DIS ’97, CHI ‘98) constructive assembly mediaBlocks (SIGGRPH’98) HandSCAPE (CHI ’00) TouchCounters (CHI ’99) musicBottles (SIGGRAPH ’99, CHI ’01) LumiTouch ComTouch (CHI ’01) (DIS ’02)
48. ambient media • to be aware of bits at the periphery using ambient display media such as light, sound, airflow, and water movement. Center and Periphery Foreground and Background Architectural Spaces as Interfaces • to grasp & manipulate bits in the center of the user's focus by coupling bits with physical objects and surfaces. graspable media Center Periphery Foreground Background ambient media graspable media
49. Foreground metaDESK & Tangible Geospace Ullmer and Ishii, CHI ’97, UIST ‘97 passiveLENS Background ambientROOM Architectural Spaces as Interfaces CHI ’97, ‘98 Ishii, H., Wisneski, C., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P, CHI ’97, ’98, CoBuild ’98 Ullmer, B. and Ishii, H. CHI ’97, UIST ‘97 phicons (physical icons) activeLENS
50. Background (Ambient Media) Pinwheels: wind of bits Ishii, H., Wisneski, C., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P, CHI ’97, ’98, CoBuild ’98 • The architectural space becomes an interface • Ambient information display spinning in a "wind of bits” • NTT ICC 2000 “Tangible Bits” Exhibit in Tokyo
51. musicBottles Ishii, Fletcher, Mazalek, Lee, Choo, Berzowska, Paradiso SIGGRAPH ’99, ’01, CHI ‘01 Jazz Techno Classical Weather • Seamless extension of metaphors and physical affordances into the digital domain • A Principle of Tangible Interface Design: Augmentation of everyday physical objects
52. musicBottles Ishii, Fletcher, Mazalek, Lee, Choo, Berzowska, Paradiso SIGGRAPH ’99, ’01, CHI ‘01 Classical
53. Origin: Weather Bottle present for my mother model: soy sauce bottle in her kitchen
54. ੴҪ ࿨ࢠ Kazuko ISHII 1926 - 1998 Origin: Weather Bottle
55. Bottles: A Transparent Interface as a Tribute to Dr. Mark Weiser IEICE TRANS. INF. & SYST., VOL.E87–D, NO.6 JUNE 2004
56. Coincidence of Input and Output Spaces A Principle of Tangible Interface Design curlybot Frei, P., Su, V., & Ishii, H. CHI ’00, SIGGRAPH ’99 record and playback physical motion topobo Raffle, H. S., Parkes, A. J., and Ishii, H. CHI ’04, ’06, ’08, SIGGRAPH ’07, IDC ‘07 building block with kinetic memory inTouch Brave, S., Dahley, A., Frei, P., & Ishii, H CHI ’97, CSCW ‘98 haptic interpersonal communication
57. Evolution of Tangible Bits 2 Tabletop TUIs Illuminating Light (CHI ’98) Urp (CHI ’99, SIGGRAPH ’99, ISMAR ’02,) metaDESK (CHI ’97, UIST ’97) PingPongPlus (CHI ’99, ACE ‘11) Sensetable (CHI ’01) Audiopad (NIME ’02, ACE ’06) IP NW WB (CHI ’03) Senseboard (CHI ’02) Urban Sim. (ISMAR ’02) Supply Chain Vis (Sys. Dyn. Rev. ’10) Sensetable platform Actuated Workbench (UIST ’02) PSyBench (CSCW ’98) PICO (CHI ’07) actuated tabletop TUIs Illuminating Clay & SandScape (CHI ’02) Phoxel Space (DIS ’04)
58. Seamless Integration of Tangibles and Digital Shadow (Projected Pixels) A Principle of Tangible Interface Design wind light reflections digital shadows Urp: Urban Planning Workbench John Underkoffler and Hiroshi Ishii, CHI ’98, 99, SIGGRAPH ‘99
59. Luminous Table in Urban Design Studio at MIT SA+P Ben-Joseph, Ishii, Underkoffler, Chak, Yeung, Piper CHI ’99, SIGGRAPH ’99, ISMAR ‘02 Urban Planning Workbench used in the Spring 2000 / 2001 MIT SA+P Courses
60. From a Scale Model of the Real World to an Abstract Computational Simulation Model A Principle of Tangible Interface Design Urp CHI ’98, 99, SIGGRAPH ‘99 Sensetable (CHI ’01) Audiopad (NIME ’02, ACE ’06) IP Network Design WB (CHI ’03) Supply Chain Vis (Sys. Dyn. Rev. ’10) Sensetable platform Supply Chain Visualization IP Network Design
61. Audiopad on the Sensetable Platform James Patten and Ben Recht, NIME ’02, ACE ’06 • A new way to perform electronic music. • Designed to combine the expressive power of traditional musical instruments with the modularity of a computer • Based on the Sensetable project.
62. Photo courtesy of Sergi Jordà ReacTable Sergi Jordà, Martin Kaltenbrunner, Günter Geiger, Ross Bencina Proceedings of the ICMC 2005, Barcelona
63. PingPongPlus Ishii, Lee, Wisneski, Orbanes, Chun, Paradiso SIGGRAPH ’98, CHI ’99, NTT ICC ’00, Ars Electronica ‘01 • Interactive Surface • Digital augmentation of ping pong play using a "reactive table" • Ball tracking using a microphone array underneath the table • Shift from competition to collaboration Photo Credit: Webb Chappell
64. Function Application Magnetic forces to move objects on a table in two dimensions. Realtime remote collaboration based on the synchronized distributed “Actuated PICO James Patten and Hiroshi Ishii CHI ‘07 • Simulation of cellular telephone networks • Mechanical constraints, coupled with computer-controlled actuation • Guide the motion of physical objects to inform the computational process Application Actuation of tangibles on table top TUIs Actuated Workbench Dan Maynes-Aminzade, Gian Pangaro & Hiroshi Ishii UIST ‘02
65. SandScape 2003 Illuminating Clay 2002 CHI ’02, TGIS ’04, BT Tech. J. ‘04
66. Hiroshi Ishii, Carlo Ratti, Ben Piper, Yao Wang, and Assaf Biderman Users can alter the form of the landscape model by Users can alter the form of the landscape model by manipulating sand while seeing the resultant effects of the computational manipulating sand while seeing the resultant effects of the computational analysis projected on the surface of the sand in real-time. analysis projected on the surface of the sand in real-time. SandScape Ars Electronic Center 2003 CHI ’02, TGIS ’04, BT Tech. J. ‘04 A Principle of TUI Design: Integration of form giving and computational analysis into one material
67. Lack of Continuity Between Physical and Digital Representation in Design Physical Ease of manipulation Clearer communication Aids spatial understanding Digital Greater precision Easy distribution Quantitative analysis How can we merge these media?
68. Tangible Design Media for Seamless Form Giving & Computational Reflection Physical Digital Upper Stream Lower Stream Rough and rapid form giving Precise and quantitative with hands for ideation computational reflection simultaneous form giving & computational reflection A Principle of Tangible Interface Design
69. Evolution of Tangible Bits 3 I/O Brush (CHI ’04, ’07) Remix+Robo Topobo (IDC ’07, SIGGRAPH ’07) Picture This! (INTERACT ’07, UbiComp ’08) jabberstamp (IDC ’07, SIGGRAPH ’07) Wetpaint (CHI ’09) Piezing (TEI ’09) OnObject (UIST ‘10) Psychohaptics (CHI ’09) Sourcemap (CHI ‘10) CopyCAD (UIST ‘10) deFORM & KidCAD (UIST ’11, CHI ’12) AR-Jig (ISMAR ’07) Beyond (CHI ‘10) g-stalt (TEI ‘10) T(ether) (SUI ’14) Kinected Conf. (CSCW ’11) AnnoScape (SUI ’14) Senspectra (CHI ’07) Glume (CHI ’06) Cord UIs (TEI ’15)
70. I/O Brush Kimiko Ryokai, Stefan Marti & Hiroshi Ishii CHI ’04, ’07, Ars Electronica ‘04 Your environment becomes a color palette to draw with I/O Brush Exhibition Ars Electronica Center Sep. 2004 ~ Aug. 2005
71. I/O Brush: History Mode Kimiko Ryokai, Stefan Marti & Hiroshi Ishii CHI ’04, ‘07 Where does the ink come from? Capturing and weaving the (hi)story for/with every stroke
72. “The World as the Palette” “The World as the Palette” “The World as the Palette” Colors in Barcelona Colors in Barcelona
73. Museums
 NTT ICC 2000 Tangible Bits Exhibition Photo Courtesy: NTT ICC
74. NTT ICC 2000 Tangible Bits Exhibition Photo Courtesy: NTT ICC
75. Ars Electronica Center 2001-04 Get in Touch Exhibition
76. Tangible Bits Get in Touch Exhibition 2001-2004 Ars Electronica Center
77. inTouch musicBottles PingPongPlus ClearBoard SandScape I/O Brush Urp PegBlocks curlybot Triangles pinwheels Tangible Bits Projects Exhibited at Ars Electronica 2001-2004
78. tangible bits 1997 radical atoms 2012 Radical Atoms Dynamic, Physical & Computational Materials that • Conform to structural constraints, • Transform structure & behavior, & • Inform new abilities.
79. 1. Frozen Atoms: inert, rigid, passive physical materials 2. Intangible Pixels: dynamic, virtual and intangible pixels (bits) 
 trapped behind a 2D flat screen Two Material Options Exist Today
80. 1. Frozen Atoms: inert, rigid, passive physical materials 2. Intangible Pixels: dynamic, virtual and intangible pixels (bits) 
 trapped behind a 2D flat screen 3. Radical Atoms: dynamic, physical and computational materials that transform and change 
 properties, driven by digital data and computation Introducing The Third Material Two Material Options Exist Today
81. Sources of Inspiration • Programmable matter (1991) by Toffoli and Margolus Toffoli, Tommaso; Margolus, Norman (1991). "Programmable matter: concepts and realization". Physica D. 47: 263–272. • Reconfigurable Robotics: M-Blocks (2015) by Daniela Rus, et.al., MIT CSAIL John W. Romanishin, Kyle Gilpin, Sebastian Claici, and Daniela Rus, 3D M-Blocks: Self-reconfiguring Robots Capable of Locomotion via Pivoting in Three Dimensions, 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, Washington, May 26-30, 2015 • Shape Display: FEELEX (2001) by Hiroo Iwata, et.al., U. of Tsukuba, Japan Hiroo Iwata, Hiroaki Yano, Fumitaka Nakaizumi, and Ryo Kawamura. 2001. Project FEELEX: adding haptic surface to graphics. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH '01). ACM, New York, NY, USA, 469-476
82. Linz, Austria, September 8 - 12, 2016 RADICAL ATOMS AND THE ALCHEMISTS OF OUR TIME
83. Radical Atoms Exhibition 2016~2019 @ Ars Electronica Center, Linz, Austria RADICAL ATOMS AND THE ALCHEMISTS OF OUR TIME
84. KinetiX Cilllia aeroMorph bioLogic inFORM musicBottles Radical Atoms Exhibition @ Ars Electronica Center Tangible Media Group | MIT Media Lab September 2016 ~ Summer 2019 in Linz, Austria Prof. Hiroshi Ishii
85. Radical Atoms Exhibition Linz Austria September 2016-2019 11 ML Projects musicBottles ZeroN Perfect Red SandScape topobo inFORM bioLogic LineFORM PneUI jamSheets Rovables Active Wood Self-Assembler Lab Lift-Bit Carlo Ratti Associati Infinite Cube ART+COM guests projects SPAXELS Ars Eelectronica Futurelab Media Lab, Responsive Environments Group
86. Tangible Media Group | MIT Media Lab Radical Atoms Exhibition 2016-19 @ Ars Electronica Center
87. ARS ELECTRONICA Linz, Austria Ars Electronica Center 2016-19 Radical Atoms Exhibition
88. Evolution of Radical Atoms 1 Actuated Tangibles inTouch (CHI ’97, CSCW ’98) curlybot (SIGGRAPH ’99, CHI ’00) topobo (CHI ’04, ‘08) I/O coincident actuated tangibles Actuated WB (UIST ’02) PSyBench (CSCW ’98) actuated tabletop TUIs PICO (CHI ’07) Weight/Volume Changing UI (TEI ’14) Amphorm (CogInfoCom ’12) Perfect Red & Radical Atoms (interactions ‘12) Kinetic Sketchup & Bosu (TEI ’09, DIS ’10) ZeroN (UIST ’11) levitating tangibles vision MirrorFugue (NIME ’13, CHI ’16) Art & Science
89. ACM interactions ‘12
91. Arts & Sciences
92. MirrorFugue III (2013) Xiao Xiao
93. MirrorFugue III (2013) Xiao Xiao
94. MirrorFugue III Xiao Xiao MirrorFugue III Xiao Xiao
95. Evolution of Radical Atoms 2 Shape Displays Static / Passive Tangible Bits Tangible Bits ! LineFORM & ChainFORM (UIST ’15, ’16 IEEE Perv. Comp. ’17) Illuminating Clay & SandScape (CHI ’02) Recom pose (UIST ‘11) Relief (TEI ’09, UIST ‘11) SUBLI MATE (CHI ‘13) inFORM (UIST ’13, ’14, Fast Comp. Design Award ’14) Tangible CityScape TRANSFORM (CHI ’15. A’Design Award ’15) Kinetic Blocks (UIST ‘15) Materiable (CHI ’16) inFORCE (TEI ‘19) AnimaStage (DIS ‘17) Active / Kinetic Shape Displays Physical Telepresence (UIST ’14) Radical Atoms
96. LineFORM & ChainFORM: Actuated Line/Curve Interfaces Ken Nakagaki, Artem Dementyev, Sean Follmer, Joseph Paradiso, and Hiroshi Ishii. UIST ’15, ‘16 LineFORM ChainFORM
97. Recompose based on the Relief Anthony DeVincenzi, David Lakatos, Matthew Blackshaw, Daniel Leithinger & Hiroshi Ishii Relief: A 2.5D Shape Display Daniel Leithinger, David Lakatos, Anthony DeVincenzi, Matthew Blackshaw, and Hiroshi Ishii. TEI ’10, UIST ‘11 Daniel Leithinger, Jinha Lee, Sean Follmer, Austin Lee, Matthew Chang & Hiroshi Ishii Daniel Leithinger & Hiroshi Ishii
98. Fast Company Innovation by Design Awards: Winner - Experimental Red Dot Award: Best of the Best - Design Concept Laval Virtual 2014 Award - INDUSTRIAL DESIGN & SIMULATION Core 77 Award - Interaction Student Winner IDSA IDEA Award Bronze inFORM Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, Hiroshi Ishii UIST ’13, ‘14
99. inFORM Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, Hiroshi Ishii. UIST ’13, ‘14
100. Cooper Hewitt Design Museum inFORM Exhibition Dec. 2014 - May 2015, New York Prof. Daniel Leithinger, Prof. Sean Follmer Philipp Schoessler, Jared Counts, Ken Nakagaki, David Doan, Basheer Tome and Prof. Hiroshi Ishii
101. inFORM ENGINES Designed by Daniel Leithinger & Sean Follmer, and Rendered by Amit Zoran
102. inFORM ENGINES Designed by Daniel Leithinger & Sean Follmer, and Rendered by Amit Zoran
103. The three panels of the triptych were sold separately in the mid-1970s.[9] Bacon was unhappy that the panels had been split up, writing on a photograph of the left-hand panel that it was "meaningless unless it is united with the other two panels." Triptych Francis Bacon
104. TextTextText TextTextText TRANSFORM Tangible Media MIT Media Lab (C) LEXUS DESIGN AMAZING 2014 MILAN
105. TRANSFORM TRANSFORM Tangible Media Tangible Media MIT Media Lab MIT Media Lab
106. Milano Design Week LEXUS DESIGN AMAZING 2014 MILAN April 8-13, 2014 Tangible Media Group MIT Media Lab
108. (C) LEXUS DESIGN AMAZING 2014 MILAN (C) LEXUS DESIGN AMAZING 2014 MILAN
109. https://www.flickr.com/photos/79203622@N02/9734698717/ TRANS-Disciplinary Finding opportunity in conflict between disciplines Breaking down old paradigms to create new archetypes
110. “Intriguing Elegance through Careful Juxtaposition of Opposing Elements” Lfinesse by LEXUS Design vs Technology Stillness vs Motion Atoms vs Bits
111. Milano Design Week 2014 TRANSFORM Exhibit 04/08-13/14, Milano, Italia Platinum A’DESIGN AWARD 2015 Prof. Hiroshi Ishii Prof. Hiroshi Ishii Prof. Daniel Leithinger Prof. Daniel Leithinger Prof. Sean Follmer Prof. Sean Follmer Prof. Amit Zoran Prof. Amit Zoran Philipp Schoessler Philipp Schoessler Jared Counts Jared Counts Tangible Media Group | MIT Media Lab @ LEXUS DESIGN AMAZING 2014 MILAN
112. Luke Vink, Viirj Kan, Ken Nakagaki, Daniel Leithinger, Sean Follmer, Philipp Schoessler, Amit Zoran, and Hiroshi Ishii. CHI ’15 Tangible Media Group | MIT Media Lab CHI 2015 Golden Mouse Award
113. A measure of a fluid’s resistance to gradual deformation by shear stress or tensile stress WATER HONEY The ability for a material to resist an applied force and to return to its original shape CLAY RUBBER The extent to which a material can be deformed in response to an applied force CONCRETE FOAM FLEXIBILITY ELASTICITY VISCOSITY Ken Nakagaki, Luke Vink, Jared Counts, Daniel Windham, Daniel Leithinger, Sean Follmer, and Hiroshi Ishii. CHI ‘16 Materiable: Rendering Material Properties in Response to Direct Touch with Shape Changing Interfaces
114. SHAPE CHANGE + DIRECT MANIPULATION + MID-AIR GESTURE MATERIAL PROPERTY REPRESENTATION + BI-DIRECTIONAL FORCE CONTROL Leithinger et al. (TEI10, UIST11, CHI11) Follmer, Leithinger, et al. (UIST13, UIST14, CHI15) Nakagaki, Vink, et al. (CHI16, TEI19) PHYSICAL TELEPRESENCE + INTERMATERIAL INTERACTION + DYNAMIC AFFORDANCES Relief + Recompose inForm + TRANSFORM MATERIABLE + inFORCE
115. Evolution of Radical Atoms 3 Programmable Materials Jamming UI (UIST ‘12) PneUI (UIST ‘13) jamSheets (TEI ‘14) optiElastic (UIST ‘14) Pneuduino (TEI ’16) aeroMorph (UIST ’16) Printflatables (CHI ’17) Pneumatic Shape Changing UIs bioPrint & xPrint (UIST ’14, CHI ’16) bioLogic (CHI ’15, A’Design Awards ’16, Sci. Adv. ‘17) Transfor- mative Appetite (CHI ’17) Hygromorphic Materials uniMorph (UIST ’15) Programmable Droplets (CHI ’18) HydroMorph (TEI ’16) Organic Primitives (CHI ’17) Cilllia (CHI ’16) SensorKnits (3D Prt. & Add. Manu. ’19) kinetiX (Comp. & Graphics ‘18)
116. PneUI (2012-13) UIST ‘13 jamSheets (2013-14) TEI ‘14 bioLogic (2014-15) CHI ‘15 aeroMoprh (2015-16) UIST ‘16 KinetiX (2014-18) Comp. & Graphics ‘18 Cilllia (2015-19) CHI ‘16 SensorKnit (2018-19) 3D Prt. & Add. Manu. ’19 Create responsive materials at the mesoscale to design novel sensors, actuators and displays uniMorph (2015) UIST ‘15 Programmable Materials
117. Cilllia 3D-Printing Functional Hair-like Structures Cilllia presents a computational method of 3D printing hair structures. It allows us to design and generate hair geometry at 50 micrometer resolution and assign various functionalities to the hair. The ability to fabricate customized hair structures enables us to create super fine surface texture; mechanical adhesion property; new passive actuators and touch sensors on a 3D printed artifact. 14,400 strands of hair on a 4cm by 4cm substrate Figures with detailed surface texture Figures with detailed surface texture Figures with detailed surface texture Figures with detailed surface texture Cilllia Jifei Ou, Gershon Dublon, Chin-Yi Cheng, Liang Zhou, Felix Heibeck and Hiroshi Ishii Golden A’Design Award 2017, CHI ’16
118. surface with mechanical adhesion sensing touch speed & direction with sound analysis Rotary Linear passive actuation with vibration: moving direction control Combined Cilllia Jifei Ou, Gershon Dublon, Chin-Yi Cheng, Liang Zhou, Felix Heibeck and Hiroshi Ishii Golden A’Design Award 2017, CHI ’16
119. Photo by Mattia Balsamini Fur & Feather as a medium for design Cilllia Jifei Ou, Gershon Dublon, Chin-Yi Cheng, Liang Zhou, Felix Heibeck and Hiroshi Ishii Golden A’Design Award 2017, CHI ’16
120. from Dr. Lining Yao’s Ph.D. thesis
121. Lining Yao, Jifei Ou, Chin-Yi Cheng, Helene Steiner, Wen Wang, Guanyun Wang, Hiroshi ishii. bioLogic: Natto Cells as Nanoactuators for Shape Changing Interfaces.In Proc. of CHI 2015. ACM BioLogic: from Build to Grow
122. bioLogic: Lining Yao, Wen Wang, Guanyun Wang, Helene Steiner, Chin-Yi Cheng, Jifei Ou, Oksana Anilionyte, Hiroshi Ishii. 3D Print. & Additive Manuf.’15, CHI ’15, Science Advances ’17, 3 A’Design Awards ’16, Fast Co.Design 2016 Innovation By Design Award, DIA Excellence Award '16
123. bioLogic: Lining Yao, Wen Wang, Guanyun Wang, Helene Steiner, Chin-Yi Cheng, Jifei Ou, Oksana Anilionyte, Hiroshi Ishii. 3D Print. & Additive Manuf.’15, CHI ’15, Science Advances ’17, 3 A’Design Awards ’16, Fast Co.Design 2016 Innovation By Design Award, DIA Excellence Award '16
124. Hiroshi Ishii MIT Media Lab “Bio is the new Interface” bioLogic Exhibit at MIT Media Lab E14 lobby in October 2015 Beyond Human-Machine Interfaces Towards Human-Living Machine Interfaces
125. Prof. Lining Yao, concept creation, interaction design and fabrication, MIT Media Lab …now CMU professor Dr. Wen Wang, biotechnology and material science, MIT Dept. of Chemical Engineering Guanyun Wang, industrial design and fabrication, MIT Media Lab/Zhejiang University Helene Steiner, interaction design, MIT Media Lab/ Royal College of Art Chin-Yi Cheng, computational design and simulation, MIT Architecture Jifei Ou, concept design and fabrication, MIT Media Lab Oksana Anilionyte, fashion design, MIT Media Lab/ Royal College of Art Prof. Hiroshi Ishii, advising and directing, Tangible Media Group, MIT Media Lab bioLogic Team 3 A’DESIGN AWARDS 2016 Textile Wearable Fashion Platinum Gold Silver
126. bioLogic: Lining Yao, Jifei Ou, Chin-Yi Cheng, Helene Steiner, Wen Wang, Guanyun Wang, Hiroshi ishii CHI ‘15 Grow rather than Build Radical Atoms The Bacterium Bacillus subtilis taken with a Tecnai T-12 TEM. Taken by Allon Weiner, The Weizmann Institute of Science, Rehovot, Israel. 2006.
127. TRANSFORM: Hiroshi Ishii, Daniel Leithinger, Sean Follmer, Amit Zoran, Philipp Schoessler, and Jared Counts. CHI ‘15 bioLogic: Lining Yao, Jifei Ou, Chin-Yi Cheng, Helene Steiner, Wen Wang, Guanyun Wang, Hiroshi ishii CHI ‘15 Programmable Droplets Udayan Umapathi, Patrick Shin, Ken Nakagaki, Daniel Leithinger and Hiroshi Ishii CHI ‘18 Dance Radical Atoms
128. CHI 2018 Golden Mouse Award Programmable Droplets Udayan Umapathi, Patrick Shin, Ken Nakagaki, Daniel Leithinger and Hiroshi Ishii CHI ‘18
129. Ω Levitate Radical Atoms
130. Fmag + mg = 0N ZeroN Jinha Lee, Rehmi Post, Hiroshi Ishii, MIT Media Lab UIST ‘11
131. ZeroN: Tangible Media Group Defy Gravity SPAXELS: Ars Electronica Futurelab
132. https://www.flickr.com/photos/79203622@N02/9734698717/ INVENT & INSPIRE
133. https://www.flickr.com/photos/79203622@N02/9734698717/ Be Artistic & Analytic Be Poetic & Pragmatic
134. Arts & Sciences Arts & Sciences Arts & Sciences
135. ART DESIGN SCIENCE TECHNOLOGY questions the world around us explains the world around us articulates the solution enables the solution
136. TRANSCENDING ART | DESIGN | SCIENCE | TECHNOLOGY Sagrada Familia Cathedral by Gaudi - spiral staircase The Tower of Babel by Pieter Bruegel the Elder (1563)
137. BUILDING SPIRAL TOWER OF THE BABEL TRANS-DISCIPLINARY STUDY THAT TRANSCENDS ART | DESIGN | SCIENCE | TECHNOLOGY BUILDING SPIRAL TOWER OF THE BABEL TRANS-DISCIPLINARY STUDY THAT TRANSCENDS ART | DESIGN | SCIENCE | TECHNOLOGY The Tower of Babel by Pieter Bruegel the Elder (1563) & ART DESIGN SCIENCE TECHNOLOGY Spiral by Hiroshi Ishii
138. Evolution Photo courtesy of Nobukazu Kuriki 2006 CHI Academy Tangible Bits @ MIT Media Lab (1995-2010) 2000 NTT ICC, Tokyo “Tangible Bits” Exhibition 1997 Tangible Bits Paper at CHI ‘97 2001-04 Ars Electronica, Linz “Getting in Touch” Exhibition 1990 2000 2010 2020 2016-19 Ars Electronica, Linz “Radical Atoms” Exhibition 2019 SIGCHI Lifetime Research Award Radical Atoms @ MIT Media Lab (2005-….) 1995 NTT! MIT Seamless Media @ NTT HI Labs (1990-95) 1992-93 Univ. of Toronto ?
139. What’s Next? Body Syntonic …
140. The Future is not to predict, but to invent Alan Kay 1971 This is the century in which you can be proactive about the future; you don't have to be reactive. The whole idea of having scientists and technology is that those things you can envision and describe can actually be built. Photo courtesy of Nobukazu Kuriki
141. vision
142. Vision Needs Technologies > 100 years ~10 years ~1 year Lifespan
143. Photo courtesy of Nobukazu Kuriki Envision vision vision vision vision
144. Envision Embody Inspire Photo courtesy of Nobukazu Kuriki
145. Envision Embody Inspire Photo courtesy of Nobukazu Kuriki Art Design, Science & Technology Art
146. ग़ߌྗ ಓఔྗ ଄ࢁྗ
147. ग़ߌྗ ಓఔྗ ଄ࢁྗ Three Driving Forces 1D: Stick Out 2D: Blaze the Trail 3D: Build the Mountain
148. ग़ߌྗ Stick Out The nail that just pokes out gets hammered down. Stick out far enough that no hammer can reach you. Stand out among the crowd. @ishii_mit
149. ಓఔྗ Blaze the Trail There is no road laid out before me. I charge forward, and a road emerges behind me. Neither spectators nor referees nor stopwatches exist here. http://www.mot-art-museum.jp/exhibition/148/3
150. Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp @ishii_mit The reason I chose to come to MIT…It was a mountain so high that the peak was beyond the clouds. There seemed to be no path leading to the top. ଄ࢁྗ Build the Mountain
151. Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp @ishii_mit But…I soon realized there was no existing mountain to climb. To create the mountain from the ground up, to be the first to reach the peak. That’s how you survive in MIT. ଄ࢁྗ Build the Mountain The reason I chose to come to MIT…It was a mountain so high that the peak was beyond the clouds. There seemed to be no path leading to the top.
152. Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
153. Today today Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
154. today 2050 2050 Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
155. today 2050 2100 2100 Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
156. today 2050 2100 2200 2200 Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
157. today 2050 2100 2200 Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp What do you want to pass on to those living in 2200? How do you want to be remembered? What do you want your legacy to be?
158. future 2200 Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
159. 2200 future Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
160. 2200 Life has a set end point But the future is never-ending Photo courtesy of Nobukazu Kuriki http://kurikiyama.jp
161. 2200 Technology soon becomes obsolete But true vision is ever-lasting “Cradle of Stars” by Scott Cresswell https://www.flickr.com/photos/scott-s_photos/11763686274/
162. MIT Media Lab 2200 “Cradle of Stars” by Scott Cresswell https://www.flickr.com/photos/scott-s_photos/11763686274/
163. Thanks! ੴҪ ༟ Hiroshi Ishii MIT Media Lab @ishii_mit ishii.mit ishii.mit
164. Thanks! ੴҪ ༟ Hiroshi Ishii MIT Media Lab @ishii_mit ishii.mit @ishii_mit ishii.mit
165. Making Digital Tangible The Battle Against the “Pixel Empire” SIGCHI Lifetime Research Award Lecture CHI 2019 in Glasgow, May 6th, 2019 Hiroshi Ishii MIT Media Lab Tangible Media Photo courtesy of Nobukazu Kuriki @ishii_mit ishii.mit