Introduction - Lecture 1 - Next Generation User Interfaces (4018166FNR)

This lecture forms part of a course on Next Generation User Interfaces given at the Vrije Universiteit Brussel.

Transcript

1. 2 December 2005 Next Generation User Interfaces Introduction Prof. Beat Signer Department of Computer Science Vrije Universiteit Brussel beatsigner.com
2. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 2 February 12, 2024 Course Organisation ▪ Prof. Beat Signer Vrije Universiteit Brussel PL9.3.60 (Pleinlaan 9) +32 2 629 1239 bsigner@vub.be wise.vub.ac.be/beat-signer ▪ Maxim Van de Wynckel Vrije Universiteit Brussel PL9.3.58 (Pleinlaan 9) +32 2 629 3487 mvdewync@vub.be wise.vub.ac.be/maxim-van-de-wynckel
3. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3 February 12, 2024 Course Organisation … ▪ Ingela Rossing Vrije Universiteit Brussel PL9.3.56 (Pleinlaan 9) +32 2 629 3749 ingela.rossing@vub.be wise.vub.ac.be/ingela-rossing
4. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4 February 12, 2024 Prerequisites ▪ Note that this is an advanced Master's level course and the following previous knowledge is required ▪ good programming skills ▪ We further expect that you have some basic knowledge about Human-Computer Interaction ▪ otherwise, it is best to follow the 'Human Computer Interaction' (1023841ANR) Bachelor course as a "repair course" ▪ It is not impossible to follow the course without these prerequisites, but in this case, you should not complain about the potential additional workload!
5. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5 February 12, 2024 Course Goals ▪ After attending the course on Next Generation User Interfaces, the student has an understanding of the interaction principles introduced by new devices such as smartphones, multi-touch tables or gesture-based inter- faces as well as the theoretical background behind these interaction principles. The student is able to reflect on the qualities and shortcomings of different interaction styles, while placing the user at the core of the interface design process. ▪ The student can apply the theoretical background and interaction principles discussed in the course and create interfaces that go beyond the classical WIMP metaphor.
6. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6 February 12, 2024 Course Goals … ▪ The course should give students the ability to under- stand the possibilities and limitations of next generation user interfaces and to recognise variations of such inter- faces. They should further be able to understand and evaluate new developments and technologies in fields related to the course and have the skills to independently study and master these new technologies.
7. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7 February 12, 2024 Exercises ▪ The course content is further investigated in a number of exercise sessions ▪ exercise sessions might also be helpful for the assignment ▪ assistants: Maxim Van de Wynckel - Monday 16:00–18:00 (exact slots and locations on the lecture schedule) ▪ Lab Sessions ▪ work on the assignment and get feedback from the assistant - Monday, 29.4.2024 and 15.5.2024, 16:00–18:00 in E.1.04 - you can also meet and get feedback (appointment via email) ▪ Additional content might be covered in exercise sessions ▪ strongly recommended to attend all exercise sessions! ▪ exam covers content of lectures and exercises
8. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8 February 12, 2024 Course Material ▪ All material will be available on Canvas ▪ lecture slides, exercises, research papers, tutorials, ... ▪ Make sure that you are subscribed to the Next Generation User Interfaces course on Canvas ▪ https://canvas.vub.be/courses/32926 ▪ Handouts normally available on Canvas at least the day before the lecture ▪ Similar information is also available on the WISE website (including links to last year's lecture slides) ▪ https://wise.vub.ac.be/course/next-generation-user-interfaces
9. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9 February 12, 2024 Lecture Schedule Exercise 1: Phidgets, Arduino and ESP32 23 24 25 26 Lecture 2: Information Architectures Lecture 3: Multimodal Interaction Initial Project Presentations Lecture 4: Pen-based Interaction Short Progress Presentations I.1.04 I.1.04 I.1.04 I.1.04 E.1.04 E.1.04 E.1.04 27 28 Lecture 1: Introduction 22 Information about Assignment and Group Formation E.1.04 Exercise 2: 3D Printing E.1.04 I.1.04 Lecture 5: Interactive Tabletops and Surfaces Exercise 3: Evaluation I.1.04 E.1.04 Lecture 6: Gesture-based Interaction Lecture 7: Tangible, Embedded and Embodied Interaction Exercise 4: Positioning Systems and Location Awareness I.1.04 E.1.04
10. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10 February 12, 2024 Lecture Schedule … Lecture 8: Virtual and Augmented Reality Exercise 5: WebXR 31 32 33 34 35 Lecture 9: Data Physicalisation Lecture 11: Human-AI Interaction Exercise 6: Discussion of Papers Lecture 12: Course Review Interim Project Presentations (Working Prototype) 29 30 Lab Session No Lecture E.1.04 E.1.04 E.1.04 E.1.04 I.1.04 I.1.04 I.1.04 I.1.04 No Exercise ⋮ 36 TBD Final Project Presentations No Lecture Lecture 10: Implicit and Cross-Device Interaction I.1.04 Lab Session E.1.04
11. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11 February 12, 2024 Assignment ▪ Next generation user interface ▪ realisation of a user interface for the domain of your choice (e.g. information management, analysis or gaming) - various presentations and reports - evaluated based on creativity, exploration of technologies, documentation, presentations, requirements analysis and evaluation, source code, … ▪ Assignment handed out later this week ▪ group project with 3 students per group - send an email with the 3 group members and your team name to Maxim Van de Wynckel by Friday, February 16 (mvdewync@vub.be) - final presentation (week of May 20), final report and code (May 26) ▪ assignment counts for 60% for the final grade
12. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12 February 12, 2024 Available Hardware Phidgets Raspberry Pi Smartphones and Tablets Pico Projectors Kinect M5 Stack
13. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13 February 12, 2024 Available Hardware … Web Cams Leap Motion Hand Tracking Digital Pens Multi-Touch Tabletop Myo Gesture Control Armband
14. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14 February 12, 2024 Available Hardware … HTC VIVE Pro 2 Ultimaker S5 3D Printer Microsoft HoloLens Meta Quest 3
15. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15 February 12, 2024 Available Hardware … ▪ You might further make use of the VUB FabLab ▪ laser cutter, CNC machines, 3D printers, …
16. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16 February 12, 2024 Exam ▪ Oral exam in English ▪ covers content of lectures and exercises ▪ counts 40% for the overall grade ▪ 5 mins questions about the assignment ▪ 15 mins questions about the course content (no preparation time) ▪ Overall grade = oral exam (40%) + assignment (60%) ▪ assignment is composed out of two grades - overall grade for project where students have some flexibility in distributing the grades (±2 points) (70%) - your contribution/knowledge to the project as checked in oral exam (30%) ▪ note that the grade for the oral exam as well as for the assign- ment have to be 8/20 or higher in order to pass the exam!
17. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17 February 12, 2024 Course Outline 1. Introduction ▪ interface types ▪ interaction design ▪ requirements analysis, prototyping and evaluation 2. Information Architectures ▪ personal information management (PIM) ▪ cross-media information systems 3. Multimodal Interaction ▪ human senses ▪ multimodal fusion and fission
18. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18 February 12, 2024 Course Outline … 4. Pen-based Interaction ▪ affordances of pen and paper ▪ digital pen and paper solutions ▪ pen and touch 5. Interactive Tabletops and Surfaces ▪ frameworks and technologies ▪ applications 6. Gesture-based Interaction ▪ single and multi-touch gestures ▪ offline vs. online gestures ▪ mid-air gestures
19. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19 February 12, 2024 Course Outline … 7. Tangible, Embedded and Embodied Interaction ▪ from tangible bits to radical atoms ▪ characteristics of tangible interfaces 8. Virtual and Augmented Reality ▪ technologies ▪ virtual and augmented reality applications 9. Data Physicalisation ▪ physical variables ▪ data physicalisation frameworks ▪ tangible holograms (TangHo)
20. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20 February 12, 2024 Course Outline … 10.Implicit and Cross-Device Interaction ▪ context-aware implicit human-computer interaction (HCII) ▪ cross-device and distributed user interfaces 11.Human-AI Interaction ▪ issues and guidelines ▪ use cases 12.Course Review & Final Project Presentations
21. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21 February 12, 2024 Video: Microsoft Office Labs Vision 2019
22. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22 February 12, 2024 "Evolution" of Interfaces date January 1984 February 2022 + 38 years price $2500 $3199 x 1.28 CPU 68000 Motorola 8 MHz 0.7 MIPS 8-core Intel Core i7 3.8 GHz 238 310 MIPS x 450 x 340000 memory 128 kB 16 GB x 132000 storage 400 kB floppy drive 1 TB SSD drive x 2684000 monitor 9" black and white 512 x 342 68 dpi 27" colour 5120 x 2880 218 dpi x 3 x 84 x 3.2 devices mouse keyboard mouse keyboard same same GUI desktop WIMP desktop WIMP same [partly based on Beaudouin-Lafon 2004] original Macintosh 27-inch iMac comparison
23. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23 February 12, 2024 Recent Changes and Opportunities in HCI ▪ Advances in graphical interfaces, speech, gesture and handwriting recognition ▪ Emergence of the Internet, cell phones, wireless networks, sensor technologies as well as large and small screens ▪ innovative interaction with digital information and services ▪ combining the physical and digital in new ways - mixed reality, cross-media spaces, tangible interfaces, wearable computing, ... ▪ collaborative interfaces with social interaction
24. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24 February 12, 2024 Interface Types ▪ Command-based ▪ WIMP and GUI ▪ Multimedia ▪ Virtual reality ▪ Web ▪ Consumer electronics ▪ Mobile ▪ Speech (Voice) ▪ Pen ▪ Touch ▪ Gesture ▪ Haptic ▪ Multimodal ▪ Shareable ▪ Tangible ▪ Augmented and mixed reality ▪ Wearable ▪ Robots ▪ Brain-computer ▪ Smart interfaces
25. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25 February 12, 2024 Towards Natural User Interfaces? ▪ Natural User Interfaces (NUI) enable a user to interact with a computer in the same way as they interact with the physical world ▪ use of speech, touch, mid-air gestures, face recognition etc. ▪ How natural are natural user interfaces? ▪ is it more natural to say “open” rather than to flick a switch to open door? ▪ is it more natural to raise both arms rather than to press a button on a remote control to change a TV channel? ▪ NUIs are effective for certain domains but might not replace existing user interfaces
26. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26 February 12, 2024 Which Interface Should We Use? ▪ In the last few years there is a significant increase in the number of user interface types ▪ How to decide which interface is preferable for a given task or activity? ▪ multimedia vs. tangible interface for learning ▪ speech vs. command-based interface ▪ multimodal vs. monomodal interface ▪ wearable vs. mobile interface ▪ virtual reality vs. augmented reality ▪ Many of these questions are currently being researched ▪ this course will provide you some more insights about the design process and different types of interfaces
27. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27 February 12, 2024 What to Design ▪ Who (user) is going to use an interactive product, how (task) is it going to be used and where (context)? ▪ Have to understand the activities people are doing while using the products ▪ How to optimise a user’s interactions with a system, environment or product to support and extend their activities in effective, useful and usable ways? ▪ take into account what people are good and bad at (human technology teamwork) ▪ what might help people with the way they currently do things ▪ listen to what people want and get them involved in the design ▪ use established user-based techniques during the design process
28. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28 February 12, 2024 Interaction Design Lifecycle Model Establishing requirements Designing alternatives Prototyping Evaluating Final product
29. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29 February 12, 2024 Interaction Design Process The interaction design process involves four basic activities 1. Establishing requirements ▪ know target users and the required support 2. Designing alternatives that meet the requirements ▪ conceptual design ▪ physical design 3. Prototyping the alternative designs in order that they can be communicated and assessed ▪ paper-based prototypes ▪ software prototypes
30. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30 February 12, 2024 Interaction Design Process 4. Evaluating ▪ determining the usability and acceptability ▪ observing or talking to users ▪ interviews or questionnaires
31. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31 February 12, 2024 Designing Alternatives ▪ Should not just stick to a solution that is “good enough” but also consider alternative solutions ▪ Creativity of the designer plays an important role ▪ discussion with other designers ▪ studying other designs - pay attention to copyrights and patent laws ▪ solve new problems based on knowledge gained from solving previous similar problems ▪ creativity workshops and brainstorming sessions “The best way to get a good idea, is to get lots of ideas.” Linus Pauling
32. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32 February 12, 2024 Types of Requirements ▪ Functional requirements ▪ what should the product do (fundamental) ▪ Data requirements ▪ type of data ▪ amount of data ▪ data accuracy ▪ … ▪ Environmental requirements (context of use) ▪ physical environment - lighting, noise, movement, dust, … ▪ social environment - synchronous or asynchronous sharing of data, co-located or distributed, …
33. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33 February 12, 2024 Types of Requirements … ▪ Environmental requirements (context of use) … ▪ organisational environment - user support, resources for training, how hierarchical is the management, … ▪ technical environment - technologies the product will run on, compatibility, technological limitations, … ▪ User characteristics ▪ key attributes of intended user group - abilities and skills - nationality and educational background - preferences - physical or mental disabilities - level of expertise (novice, expert, casual user, frequent user, …) ▪ user profile consists of a collection of attributes for a typical user
34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34 February 12, 2024 Types of Requirements … ▪ Usability goals ▪ define measures for agreed usability goals - objective measure of a user’s performance - effectiveness, efficiency, safety, utility, learnability, memorability ▪ User experience goals ▪ Different forms of data gathering for requirements ▪ interviews, focus groups, questionnaires, direct observation, indirect observation, studying documentation or researching similar products
35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35 February 12, 2024 User-Centred Design 1. Early focus on users and tasks ▪ who will be the users ▪ study the characteristics of users 2. Empirical measurement ▪ identify specific usability and user experience goals - helps to choose between alternative designs - can be used to check the progress ▪ sketches, description in natural language and prototypes help to observe and analyse the performance and reactions of users 3. Iterative design ▪ problems identified in user testing are fixed and evaluated in a next iteration - iteration is particularly important when trying to innovate
36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36 February 12, 2024 Prototyping ▪ “It is often said that users can’t tell you what they want, but when they see something and get to use it, they soon know what they don’t want.” ▪ A prototype is a manifestation of design that allows stakeholders to interact with it and explore its suitability ▪ paper-based storyboard or outline of a screen ▪ electronic picture or video simulation of a task ▪ 3D cardboard mock-up or object printed with a 3D printer ▪ piece of software ▪ … ▪ Prototypes help to choose between design alternatives ▪ Building a prototype encourages reflection in design
37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37 February 12, 2024 Low-Fidelity Prototyping ▪ Sketching ▪ hand-drawn sketches containing computer components, icons, dialogue boxes, … ▪ Prototyping with index cards ▪ each card represents one screen or one element of a task ▪ Wizard of Oz experiment ▪ assumes that one has a software-based prototype ▪ human operator (wizard) simulates the software’s response to the user ▪ PowerPoint ▪ balances the provisionality of paper with the polished appearance of software prototypes - characteristics of low and high fidelity
38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38 February 12, 2024 Evaluation ▪ Evaluation is an integral part of the design process ▪ usability of the system ▪ user experience ▪ Observe participants and measure their perfor- mance ▪ usability testing ▪ experiments ▪ field studies
39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39 February 12, 2024 Why, What, Where and When to Evaluate ▪ Why evaluate ▪ do we fulfill user requirements? ▪ ensure that users can use the product and they like it ▪ What to evaluate ▪ conceptual models ▪ early low-fidelity prototypes or high-fidelity prototypes ▪ individual function, complete workflow, aesthetic design, safety, … “User experience encompasses all aspects of the end-user’s interaction … the first requirement for an exemplary user experience is to meet the exact needs of the customer, without fuss or bother. Next come simplicity and elegance, which produces products that are a joy to own, a joy to use.” Nielsen Norman Group
40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40 February 12, 2024 Why, What, Where and When to Evaluate … ▪ Where to evaluate ▪ laboratory ▪ natural setting (in-the-wild studies) - better for user experience ▪ living labs ▪ When to evaluate ▪ formative evaluations - throughout the design process - what and how to redesign? ▪ summative evaluations - assess the final product - how well did we do? Aware Home, Georgia Tech
41. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41 February 12, 2024 UEQ+ User Experience Questionnaire ▪ Tool to build customised UX questionnaire ▪ modular extension of UEQ ▪ customised selection of UX scales ▪ Word templates with questions - more than 20 languages - answers on a 7-point Likert scale ▪ UEQ+ Data Analysis Tool ▪ Excel document creating graphics etc.
42. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42 February 12, 2024 UEQ+ Available Scales
43. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43 February 12, 2024 Online Survey Tools Qualtrics
44. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 44 February 12, 2024 Assignment ▪ Focus first on problem and user needs ▪ different interface types and hardware technologies should only be considered when designing alternatives ▪ Consider the use of user-centred design ▪ requirements, prototyping as well as evaluation ▪ Iterative design process with rapid low-fidelity prototyping ▪ Do not forget to evaluate your solution!
45. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 45 February 12, 2024 References ▪ Interaction Design: Beyond Human-Computer Interaction, Yvonne Rogers, Helen Sharp and Jenny Preece, Wiley (6th edition), April 2023 ISBN-13: 978-1119901099 ▪ Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Guidelines, Jeff Johnson, Morgan Kaufmann (3rd edition), November 2020, 978-0128182024 ▪ Human-Computer Interaction, Alan Dix, Janet E. Finlay, Gregory D. Abowd and Russell Beale, Prentice Hall (3rd edition), December 2003 ISBN-13: 978-0130461094
46. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 46 February 12, 2024 References ... ▪ Research Methods in Human-Computer Inter- action, Jonathan Lazar, Jinjuan Heidi Feng and Harry Hochheiser, Morgan Kaufmann (2nd edition), May 2019, ISBN-13: 978-0128053904 ▪ Designing for Interaction: Creating Innovative Applications and Devices, Dan Saffer, New Riders (2nd edition), August 2009 ISBN-13: 978-0321643391 ▪ The Design of Everyday Things, Don Norman, Basic Books (revised and expanded edition), November 2013, ISBN-13: 978-0465050659
47. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 47 February 12, 2024 References ... ▪ Human Computer Interaction (1023841ANR) course ▪ https://wise.vub.ac.be/index.php/course/human-computer- interaction ▪ Microsoft Office Labs Vision 2019 ▪ https://www.youtube.com/watch?v=Zp-_oUwdSeY ▪ MAXQDA ▪ https://www.maxqda.com ▪ https://softweb.vub.be ▪ Qualtrics XM ▪ https://www.qualtrics.com ▪ https://softweb.vub.be
48. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 48 February 12, 2024 References ... ▪ UEQ+: A Modular Extension of the User Experience Questionnaire ▪ https://ueqplus.ueq-research.org
49. 2 December 2005 Next Lecture Information Architectures