Interaction modalities, technologies and tools for interactive art
Presentation about interaction modalities and technologies that can be used for interactive art projects, with emphasis on devices available at the School of Arts (Catholic University of Portugal), Porto.
• Examples of technologies and tools that can be used to implement those interactions o Available tools at EArtes/UCP • Most prominent people/references in these areas • No code (for now)
that contains information. Waving goodbye is a gesture. Pressing a key on a keyboard is not a gesture because the motion of a finger on its way to hitting a key is neither observed nor significant. All that matters is which key was pressed". -- Kurtenbach and Hulteen (1990), cited by Billinghurst & Buxton (2011)
• Various technologies for detecting touch o which have different limits for the number of simultaneous touches it can detect • Gestures o Fingers only (small devices) o Hands/forearms (large(r) devices, depending on technology used) o 2d movements only • Number of users o one - personal devices (phones, tablets) o few - large surfaces (tables, walls)
o Laser light plane (LLP) technique (see Sandler 2012) o Multi-touch surface (in development) o Vertical orientation o One-two person interaction o Fingers, hands
Multi-Touch Screen o Reality touchscreen University of Groningen o http://www.youtube.com/watch?v=AlWFtF06RFo Several-person interaction, fingers, hands
cameras) • Various kinds of surfaces/scenes • Gestures o Varied, depends on positioning of camera § fingers § hands § whole body o mostly 2d, but can also infer distance o not very precise § Can be very precise with controlled conditions
uses bluetooth to connect to a computer o Has various accelerometer sensors to sense movement o IR sensor for pointing • Gestures o 3d movement, rotation o combined with the sensor bar, allows pointing gestures • Various wiimotes available @ EArtes
distance of each pixel to the camera (Borenstein 2012). • Gestures o Detects a "point cloud" - pixels and their distances to the camera o With extra software it's possible to detect limbs (skeleton) positioning o More accurate than simple cameras • Two available @ EArtes Gestural interaction: Kinect
Kiefer • Using various Kinect to model a dancer and then process the data into visual effects • https://vimeo.com/38505448 • honorary mention from the Prix Ars Electronica
things see" (Borenstein 2012) • Processing/Java o OpenKinect - http://www.shiffman.net/p5/kinect/ o SimpleOpenNI - http://code.google.com/p/simple-openni/ o http://www.creativeapplications.net/processing/ kinect-physics-tutorial-for-processing/ • Max/Msp o jit.freenect.grab - http://jmpelletier.com/freenect/
Interaction, http://www.billbuxton.com/ input14.Gesture.pdf • Borenstein, Greg (2012). Making Things See: 3D vision with Kinect, Processing, Arduino, and MakerBot, O'Reilly Media / Make • Buxton, Bill (2012). Multi-Touch Systems that I Have Known and Loved, http:// www.billbuxton.com/multitouchOverview.html • Kurtenbach, G. & Hulteen, E. (1990). Gestures in Human-Computer Communications. In B. Laurel (Ed.) The Art of Human Computer Interface Design. Addison-Wesley, 309-317, http:// www.billbuxton.com/inputManuscript.html • Levin, Golan. Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers, http://www.flong.com/texts/essays/essay_cvad/ • Sandler, Seth (2012). LASER LIGHT PLANE MULTITOUCH TECHNIQUE, http:// sethsandler.com/multitouch/llp/
information, employing physical artifacts both as representations and controls for computational media. TUIs couple physical representations (e.g., spatially manipulable physical objects) with digital representations (e.g., graphics and audio), yielding user interfaces that are computationally mediated, but generally not identifiable as "computers" per se." - Ullmer & Ishii (2000)
of printable objects for tangible prototypes • http://vimeo.com/38793875 • Paper o http://tei-conf.org/12/Main/Sketchatui • Printable objects o http://project-premium.org/sketch-a-tui/index.html
Malthe Borch, and Andrew Spitz • Tangible interface to Last.fm & Spotify Radio • http://www.creativeapplications.net/maxmsp/skube- tangible-interface-to-last-fm-spotify-radio/
W. A. S. Buxton, “Bricks: Laying the Foundations for Graspable User Interfaces,” in Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’95, 1995, pp. 442–449. • H. Ishii and B. Ullmer, “Tangible bits,” in Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’97, 1997, pp. 234–241. • Ullmer, B. and Ishii, H. (2000). “Emerging frameworks for tangible user interfaces,” IBM Systems Journal, vol. 39, no. 3, pp. 915–931, Jul. 2000.
Using brain signals to interact with computers. • Usually based on EEG readings • Various commercial devices available • http://en.wikipedia.org/wiki/Brain %E2%80%93computer_interface#Low-cost_BCI- based_Interfaces
inventing, designing, building, or using miniature body-borne computational and sensory devices. Wearable computers may be worn under, over, or in clothing, or may also be themselves clothes" (Mann, 2012)
Soegaard, Mads and Dam, Rikke Friis (eds.). "Encyclopedia of Human-Computer Interaction". Aarhus, Denmark: The Interaction Design Foundation. Available online at http://www.interaction-design.org/encyclopedia/wearable_computing.html
interactive physical systems by the use of software and hardware that can sense and respond to the analog world. (...) In the broad sense, physical computing is a creative framework for understanding human beings' relationship to the digital world. In practical use, the term most often describes handmade art, design or DIY hobby projects that use sensors and microcontrollers to translate analog input to a software system, and/or control electro-mechanical devices such as motors, servos, lighting or other hardware." - (Physical computing 2012)
Wikipedia, The Free Encyclopedia. Retrieved 14:37, September 22, 2012, from http://en.wikipedia.org/w/index.php? title=Physical_computing&oldid=503476286 • Dan O'Sullivan, Tom Igoe. (2004). Physical Computing: Sensing and Controlling the Physical World with Computers, Thomson Course Technology, Boston, Massachusetts,
anyone to see/interact with them • Traditionally, public displays have been used to broad information from one active source to millions of passive receivers • Digital public displays can change this o Turning receivers into active receivers o Supporting dynamic locally relevant information o Supporting user-generated content (many to many content distribution) • Everyone can interact o Everyone can see you interacting
show "Big Love" • Users can listen to people's thoughts by plugin headphones into the billboard • http://www.trendhunter.com/trends/audio-jack-billboard- for-big-love
al. 2003 • Display bus information (only the bus number 16 - two in each direction) o Size of square: amount of time before bus leaves o Color: Time to leave to the bus stop o Position: right - buses to the city center; left - buses from the city center
• Paper o http://www.mendeley.com/research/mobilenin-combining-multi-track- music-video-personal-mobile-phones-public-display-multi-user-interac o Allows users to vote on the next video clip
& Tanaka, K. (1999). The Information Percolator : Ambient Information Display in a Decorative Object. Proceedings of the 12th annual ACM symposium on User interface software and technology - UIST ’99 (pp. 141–148). New York, New York, USA: ACM Press. doi:10.1145/320719.322595 Prante, T., Röcker, C., Streitz, N., Stenzel, R., Magerkurth, C., van Alphen, D., & Plewe, D. (2003). Hello.Wall - Beyond Ambient Displays. Video Track and Adjunct Proceedings of the 5th Intern. Conference on Ubiquitous Computing (UBICOMP’03). Seattle, Wash., USA. Skog, T., Ljungblad, S., & Holmquist, L. E. (n.d.). Between aesthetics and utility: designing ambient information visualizations. IEEE Symposium on Information Visualization 2003 (IEEE Cat. No. 03TH8714) (pp. 233–240). IEEE. doi:10.1109/INFVIS.2003.1249031
computer is expected to be transported during normal usage. Mobile computing involves mobile communication, mobile hardware, and mobile software." - (Mobile computing 2012)
game, peddy paper like • Players have to discover a set of clues spread across the city (Porto) using a mobile device that plays videos pointing to next clues and that tracks players GPS locations • http://jpn.c2com.up.pt/2005/10/29/ xfetch_quando_o_real_e_o_virtual_se_confundem.html
Wikipedia, The Free Encyclopedia. Retrieved 17:56, September 30, 2012, from http://en.wikipedia.org/w/index.php? title=Mobile_computing&oldid=511364202
jorgecardoso
signer
learnenergy2
2720japanoke
asial_corp
masanari
copilot
takuro_nakajima
utokyoissr2360
d_date
edds
maggiecrowley
kneath
thoeni
swwweet
tmm1
afnizarnur
maltzj
bryan
lynnandtonic
searls
pauljervisheath
![Tangible interaction "[...] tangible interfaces give physical form to digital](https://files.speakerdeck.com/presentations/506caf717e8497000201b1cc/slide_43.jpg){kind=link}
