Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction - Lecture 1 - Next Generation User Interfaces (4018166FNR)

Introduction - Lecture 1 - Next Generation User Interfaces (4018166FNR)

This lecture forms part of a course on Next Generation User Interfaces given at the Vrije Universiteit Brussel.

Beat Signer

February 11, 2024
Tweet

More Decks by Beat Signer

Other Decks in Education

Transcript

  1. 2 December 2005
    Next Generation User Interfaces
    Introduction
    Prof. Beat Signer
    Department of Computer Science
    Vrije Universiteit Brussel
    beatsigner.com

    View full-size slide

  2. Beat Signer - Department of Computer Science - [email protected] 2
    February 12, 2024
    Course Organisation
    ▪ Prof. Beat Signer
    Vrije Universiteit Brussel
    PL9.3.60 (Pleinlaan 9)
    +32 2 629 1239
    [email protected]
    wise.vub.ac.be/beat-signer
    ▪ Maxim Van de Wynckel
    Vrije Universiteit Brussel
    PL9.3.58 (Pleinlaan 9)
    +32 2 629 3487
    [email protected]
    wise.vub.ac.be/maxim-van-de-wynckel

    View full-size slide

  3. Beat Signer - Department of Computer Science - [email protected] 3
    February 12, 2024
    Prerequisites
    ▪ Note that this is an advanced Master's level
    course and the following previous knowledge is required
    ▪ good programming skills
    ▪ We further expect that you have some basic knowledge
    about Human-Computer Interaction
    ▪ otherwise, it is best to follow the 'Human Computer Interaction'
    (1023841ANR) Bachelor course as a "repair course"
    ▪ It is not impossible to follow the course without these
    prerequisites, but in this case, you should not complain
    about the potential additional workload!

    View full-size slide

  4. Beat Signer - Department of Computer Science - [email protected] 4
    February 12, 2024
    Course Goals
    ▪ After attending the course on Next Generation User
    Interfaces, the student has an understanding of the
    interaction principles introduced by new devices such as
    smartphones, multi-touch tables or gesture-based inter-
    faces as well as the theoretical background behind these
    interaction principles. The student is able to reflect on the
    qualities and shortcomings of different interaction styles,
    while placing the user at the core of the interface design
    process.
    ▪ The student can apply the theoretical background and
    interaction principles discussed in the course and create
    interfaces that go beyond the classical WIMP metaphor.

    View full-size slide

  5. Beat Signer - Department of Computer Science - [email protected] 5
    February 12, 2024
    Course Goals …
    ▪ The course should give students the ability to under-
    stand the possibilities and limitations of next generation
    user interfaces and to recognise variations of such inter-
    faces. They should further be able to understand and
    evaluate new developments and technologies in fields
    related to the course and have the skills to independently
    study and master these new technologies.

    View full-size slide

  6. Beat Signer - Department of Computer Science - [email protected] 6
    February 12, 2024
    Exercises
    ▪ The course content is further investigated in
    a number of exercise sessions
    ▪ exercise sessions might also be helpful for the assignment
    ▪ assistants: Maxim Van de Wynckel
    - Monday 16:00–18:00 (exact slots and locations on the lecture schedule)
    ▪ Lab Sessions
    ▪ work on the assignment and get feedback from the assistant
    - Monday, 29.4.2024 and 15.5.2024, 16:00–18:00 in E.1.04
    - you can also meet and get feedback (appointment via email)
    ▪ Additional content might be covered in exercise sessions
    ▪ strongly recommended to attend all exercise sessions!
    ▪ exam covers content of lectures and exercises

    View full-size slide

  7. Beat Signer - Department of Computer Science - [email protected] 7
    February 12, 2024
    Course Material
    ▪ All material will be available on Canvas
    ▪ lecture slides, exercises, research papers, tutorials, ...
    ▪ Make sure that you are subscribed to the
    Next Generation User Interfaces course on Canvas
    ▪ https://canvas.vub.be/courses/32926
    ▪ Handouts normally available on Canvas at least the day
    before the lecture
    ▪ Similar information is also available on the WISE website
    (including links to last year's lecture slides)
    ▪ https://wise.vub.ac.be/course/next-generation-user-interfaces

    View full-size slide

  8. Beat Signer - Department of Computer Science - [email protected] 8
    February 12, 2024
    Lecture Schedule
    Exercise 1: Phidgets, Arduino and ESP32
    23
    24
    25
    26
    Lecture 2: Information Architectures
    Lecture 3: Multimodal Interaction
    Initial Project Presentations
    Lecture 4: Pen-based Interaction
    Short Progress Presentations
    I.1.04
    I.1.04
    I.1.04
    I.1.04
    E.1.04
    E.1.04
    E.1.04
    27
    28
    Lecture 1: Introduction
    22
    Information about Assignment and Group Formation E.1.04
    Exercise 2: 3D Printing E.1.04
    I.1.04
    Lecture 5: Interactive Tabletops and Surfaces
    Exercise 3: Evaluation
    I.1.04
    E.1.04
    Lecture 6: Gesture-based Interaction
    Lecture 7: Tangible, Embedded and Embodied Interaction
    Exercise 4: Positioning Systems and Location Awareness
    I.1.04
    E.1.04

    View full-size slide

  9. Beat Signer - Department of Computer Science - [email protected] 9
    February 12, 2024
    Lecture Schedule …
    Lecture 8: Virtual and Augmented Reality
    Exercise 5: WebXR
    31
    32
    33
    34
    35
    Lecture 9: Data Physicalisation
    Lecture 11: Human-AI Interaction
    Exercise 6: Discussion of Papers
    Lecture 12: Course Review
    Interim Project Presentations (Working Prototype)
    29
    30
    Lab Session
    No Lecture
    E.1.04
    E.1.04
    E.1.04
    E.1.04
    I.1.04
    I.1.04
    I.1.04
    I.1.04
    No Exercise

    36
    TBD
    Final Project Presentations
    No Lecture
    Lecture 10: Implicit and Cross-Device Interaction I.1.04
    Lab Session E.1.04

    View full-size slide

  10. Beat Signer - Department of Computer Science - [email protected] 10
    February 12, 2024
    Assignment
    ▪ Next generation user interface
    ▪ realisation of a user interface for the domain of
    your choice (e.g. information management, analysis or gaming)
    - various presentations and reports
    - evaluated based on creativity, exploration of technologies, documentation,
    presentations, requirements analysis and evaluation, source code, …
    ▪ Assignment handed out later this week
    ▪ group project with 3 students per group
    - send an email with the 3 group members and your team name to Maxim Van
    de Wynckel by Friday, February 16 ([email protected])
    - final presentation (week of May 20), final report and code (May 26)
    ▪ assignment counts for 60% for the final grade

    View full-size slide

  11. Beat Signer - Department of Computer Science - [email protected] 11
    February 12, 2024
    Available Hardware
    Phidgets
    Raspberry Pi
    Smartphones and Tablets
    Pico Projectors
    Kinect
    M5 Stack

    View full-size slide

  12. Beat Signer - Department of Computer Science - [email protected] 12
    February 12, 2024
    Available Hardware …
    Web Cams
    Leap Motion Hand Tracking
    Digital Pens
    Multi-Touch Tabletop
    Myo Gesture Control Armband

    View full-size slide

  13. Beat Signer - Department of Computer Science - [email protected] 13
    February 12, 2024
    Available Hardware …
    HTC VIVE Pro 2
    Ultimaker S5 3D Printer
    Microsoft HoloLens
    Meta Quest 3

    View full-size slide

  14. Beat Signer - Department of Computer Science - [email protected] 14
    February 12, 2024
    Available Hardware …
    ▪ You might further make use of the VUB FabLab
    ▪ laser cutter, CNC machines, 3D printers, …

    View full-size slide

  15. Beat Signer - Department of Computer Science - [email protected] 15
    February 12, 2024
    Exam
    ▪ Oral exam in English
    ▪ covers content of lectures and exercises
    ▪ counts 40% for the overall grade
    ▪ 5 mins questions about the assignment
    ▪ 15 mins questions about the course content (no preparation time)
    ▪ Overall grade = oral exam (40%) + assignment (60%)
    ▪ assignment is composed out of two grades
    - overall grade for project where students have some flexibility in distributing
    the grades (±2 points) (70%)
    - your contribution/knowledge to the project as checked in oral exam (30%)
    ▪ note that the grade for the oral exam as well as for the assign-
    ment have to be 8/20 or higher in order to pass the exam!

    View full-size slide

  16. Beat Signer - Department of Computer Science - [email protected] 16
    February 12, 2024
    Course Outline
    1. Introduction
    ▪ interface types
    ▪ interaction design
    ▪ requirements analysis, prototyping and evaluation
    2. Information Architectures
    ▪ personal information management (PIM)
    ▪ cross-media information systems
    3. Multimodal Interaction
    ▪ human senses
    ▪ multimodal fusion and fission

    View full-size slide

  17. Beat Signer - Department of Computer Science - [email protected] 17
    February 12, 2024
    Course Outline …
    4. Pen-based Interaction
    ▪ affordances of pen and paper
    ▪ digital pen and paper solutions
    ▪ pen and touch
    5. Interactive Tabletops and Surfaces
    ▪ frameworks and technologies
    ▪ applications
    6. Gesture-based Interaction
    ▪ single and multi-touch gestures
    ▪ offline vs. online gestures
    ▪ mid-air gestures

    View full-size slide

  18. Beat Signer - Department of Computer Science - [email protected] 18
    February 12, 2024
    Course Outline …
    7. Tangible, Embedded and Embodied Interaction
    ▪ from tangible bits to radical atoms
    ▪ characteristics of tangible interfaces
    8. Virtual and Augmented Reality
    ▪ technologies
    ▪ virtual and augmented reality applications
    9. Data Physicalisation
    ▪ physical variables
    ▪ data physicalisation frameworks
    ▪ tangible holograms (TangHo)

    View full-size slide

  19. Beat Signer - Department of Computer Science - [email protected] 19
    February 12, 2024
    Course Outline …
    10.Implicit and Cross-Device Interaction
    ▪ context-aware implicit human-computer interaction (HCII)
    ▪ cross-device and distributed user interfaces
    11.Human-AI Interaction
    ▪ issues and guidelines
    ▪ use cases
    12.Course Review & Final Project Presentations

    View full-size slide

  20. Beat Signer - Department of Computer Science - [email protected] 20
    February 12, 2024
    Video: Microsoft Office Labs Vision 2019

    View full-size slide

  21. Beat Signer - Department of Computer Science - [email protected] 21
    February 12, 2024
    "Evolution" of Interfaces
    date January 1984 February 2022 + 38 years
    price $2500 $3199 x 1.28
    CPU 68000 Motorola
    8 MHz
    0.7 MIPS
    8-core Intel Core i7
    3.8 GHz
    238 310 MIPS
    x 450
    x 340000
    memory 128 kB 16 GB x 132000
    storage 400 kB floppy drive 1 TB SSD drive x 2684000
    monitor 9" black and white
    512 x 342
    68 dpi
    27" colour
    5120 x 2880
    218 dpi
    x 3
    x 84
    x 3.2
    devices mouse
    keyboard
    mouse
    keyboard
    same
    same
    GUI desktop WIMP desktop WIMP same
    [partly based on Beaudouin-Lafon 2004]
    original Macintosh 27-inch iMac comparison

    View full-size slide

  22. Beat Signer - Department of Computer Science - [email protected] 22
    February 12, 2024
    Recent Changes and Opportunities in HCI
    ▪ Advances in graphical interfaces, speech, gesture and
    handwriting recognition
    ▪ Emergence of the Internet, cell phones, wireless
    networks, sensor technologies as well as large and small
    screens
    ▪ innovative interaction with digital information and services
    ▪ combining the physical and digital in new ways
    - mixed reality, cross-media spaces, tangible interfaces, wearable computing, ...
    ▪ collaborative interfaces with social interaction

    View full-size slide

  23. Beat Signer - Department of Computer Science - [email protected] 23
    February 12, 2024
    Interface Types
    ▪ Command-based
    ▪ WIMP and GUI
    ▪ Multimedia
    ▪ Virtual reality
    ▪ Web
    ▪ Consumer electronics
    ▪ Mobile
    ▪ Speech (Voice)
    ▪ Pen
    ▪ Touch
    ▪ Gesture
    ▪ Haptic
    ▪ Multimodal
    ▪ Shareable
    ▪ Tangible
    ▪ Augmented and mixed reality
    ▪ Wearable
    ▪ Robots
    ▪ Brain-computer
    ▪ Smart interfaces

    View full-size slide

  24. Beat Signer - Department of Computer Science - [email protected] 24
    February 12, 2024
    Towards Natural User Interfaces?
    ▪ Natural User Interfaces (NUI) enable a user to interact
    with a computer in the same way as they interact with
    the physical world
    ▪ use of speech, touch, mid-air gestures, face recognition etc.
    ▪ How natural are natural user interfaces?
    ▪ is it more natural to say “open” rather than to flick a switch
    to open door?
    ▪ is it more natural to raise both arms rather than to press a button
    on a remote control to change a TV channel?
    ▪ NUIs are effective for certain domains but might not
    replace existing user interfaces

    View full-size slide

  25. Beat Signer - Department of Computer Science - [email protected] 25
    February 12, 2024
    Which Interface Should We Use?
    ▪ In the last few years there is a significant increase in the
    number of user interface types
    ▪ How to decide which interface is preferable for a given
    task or activity?
    ▪ multimedia vs. tangible interface for learning
    ▪ speech vs. command-based interface
    ▪ multimodal vs. monomodal interface
    ▪ wearable vs. mobile interface
    ▪ virtual reality vs. augmented reality
    ▪ Many of these questions are currently being researched
    ▪ this course will provide you some more insights about the design
    process and different types of interfaces

    View full-size slide

  26. Beat Signer - Department of Computer Science - [email protected] 26
    February 12, 2024
    What to Design
    ▪ Who (user) is going to use an interactive product,
    how (task) is it going to be used and where (context)?
    ▪ Have to understand the activities people are doing while
    using the products
    ▪ How to optimise a user’s interactions with a system,
    environment or product to support and extend their
    activities in effective, useful and usable ways?
    ▪ take into account what people are good and bad at (human
    technology teamwork)
    ▪ what might help people with the way they currently do things
    ▪ listen to what people want and get them involved in the design
    ▪ use established user-based techniques during the design process

    View full-size slide

  27. Beat Signer - Department of Computer Science - [email protected] 27
    February 12, 2024
    Interaction Design Lifecycle Model
    Establishing
    requirements
    Designing
    alternatives
    Prototyping
    Evaluating
    Final
    product

    View full-size slide

  28. Beat Signer - Department of Computer Science - [email protected] 28
    February 12, 2024
    Interaction Design Process
    The interaction design process involves four basic activities
    1. Establishing requirements
    ▪ know target users and the required support
    2. Designing alternatives that meet the requirements
    ▪ conceptual design
    ▪ physical design
    3. Prototyping the alternative designs in order that they
    can be communicated and assessed
    ▪ paper-based prototypes
    ▪ software prototypes

    View full-size slide

  29. Beat Signer - Department of Computer Science - [email protected] 29
    February 12, 2024
    Interaction Design Process
    4. Evaluating
    ▪ determining the usability and acceptability
    ▪ observing or talking to users
    ▪ interviews or questionnaires

    View full-size slide

  30. Beat Signer - Department of Computer Science - [email protected] 30
    February 12, 2024
    Designing Alternatives
    ▪ Should not just stick to a solution that is “good enough”
    but also consider alternative solutions
    ▪ Creativity of the designer plays an important role
    ▪ discussion with other designers
    ▪ studying other designs
    - pay attention to copyrights and patent laws
    ▪ solve new problems based on knowledge gained from solving
    previous similar problems
    ▪ creativity workshops and brainstorming sessions
    “The best way to get a good idea, is to get lots of ideas.”
    Linus Pauling

    View full-size slide

  31. Beat Signer - Department of Computer Science - [email protected] 31
    February 12, 2024
    Types of Requirements
    ▪ Functional requirements
    ▪ what should the product do (fundamental)
    ▪ Data requirements
    ▪ type of data
    ▪ amount of data
    ▪ data accuracy
    ▪ …
    ▪ Environmental requirements (context of use)
    ▪ physical environment
    - lighting, noise, movement, dust, …
    ▪ social environment
    - synchronous or asynchronous sharing of data, co-located or distributed, …

    View full-size slide

  32. Beat Signer - Department of Computer Science - [email protected] 32
    February 12, 2024
    Types of Requirements …
    ▪ Environmental requirements (context of use) …
    ▪ organisational environment
    - user support, resources for training, how hierarchical is the management, …
    ▪ technical environment
    - technologies the product will run on, compatibility, technological limitations, …
    ▪ User characteristics
    ▪ key attributes of intended user group
    - abilities and skills
    - nationality and educational background
    - preferences
    - physical or mental disabilities
    - level of expertise (novice, expert, casual user, frequent user, …)
    ▪ user profile consists of a collection of attributes for a typical user

    View full-size slide

  33. Beat Signer - Department of Computer Science - [email protected] 33
    February 12, 2024
    Types of Requirements …
    ▪ Usability goals
    ▪ define measures for agreed usability goals
    - objective measure of a user’s performance
    - effectiveness, efficiency, safety, utility, learnability, memorability
    ▪ User experience goals
    ▪ Different forms of data gathering for requirements
    ▪ interviews, focus groups, questionnaires, direct observation,
    indirect observation, studying documentation or researching
    similar products

    View full-size slide

  34. Beat Signer - Department of Computer Science - [email protected] 34
    February 12, 2024
    User-Centred Design
    1. Early focus on users and tasks
    ▪ who will be the users
    ▪ study the characteristics of users
    2. Empirical measurement
    ▪ identify specific usability and user experience goals
    - helps to choose between alternative designs
    - can be used to check the progress
    ▪ sketches, description in natural language and prototypes help to
    observe and analyse the performance and reactions of users
    3. Iterative design
    ▪ problems identified in user testing are fixed and evaluated in
    a next iteration
    - iteration is particularly important when trying to innovate

    View full-size slide

  35. Beat Signer - Department of Computer Science - [email protected] 35
    February 12, 2024
    Prototyping
    ▪ “It is often said that users can’t tell you what
    they want, but when they see something and
    get to use it, they soon know what they don’t want.”
    ▪ A prototype is a manifestation of design that allows
    stakeholders to interact with it and explore its suitability
    ▪ paper-based storyboard or outline of a screen
    ▪ electronic picture or video simulation of a task
    ▪ 3D cardboard mock-up or object printed with a 3D printer
    ▪ piece of software
    ▪ …
    ▪ Prototypes help to choose between design alternatives
    ▪ Building a prototype encourages reflection in design

    View full-size slide

  36. Beat Signer - Department of Computer Science - [email protected] 36
    February 12, 2024
    Low-Fidelity Prototyping
    ▪ Sketching
    ▪ hand-drawn sketches containing computer components, icons,
    dialogue boxes, …
    ▪ Prototyping with index cards
    ▪ each card represents one screen or one element of a task
    ▪ Wizard of Oz experiment
    ▪ assumes that one has a software-based prototype
    ▪ human operator (wizard) simulates the software’s response
    to the user
    ▪ PowerPoint
    ▪ balances the provisionality of paper with the polished appearance
    of software prototypes
    - characteristics of low and high fidelity

    View full-size slide

  37. Beat Signer - Department of Computer Science - [email protected] 37
    February 12, 2024
    Evaluation
    ▪ Evaluation is an integral
    part of the design process
    ▪ usability of the system
    ▪ user experience
    ▪ Observe participants and
    measure their perfor-
    mance
    ▪ usability testing
    ▪ experiments
    ▪ field studies

    View full-size slide

  38. Beat Signer - Department of Computer Science - [email protected] 38
    February 12, 2024
    Why, What, Where and When to Evaluate
    ▪ Why evaluate
    ▪ do we fulfill user requirements?
    ▪ ensure that users can use the product and they like it
    ▪ What to evaluate
    ▪ conceptual models
    ▪ early low-fidelity prototypes or high-fidelity prototypes
    ▪ individual function, complete workflow, aesthetic design, safety, …
    “User experience encompasses all aspects of the end-user’s
    interaction … the first requirement for an exemplary user experience
    is to meet the exact needs of the customer, without fuss or bother.
    Next come simplicity and elegance, which produces products that are
    a joy to own, a joy to use.”
    Nielsen Norman Group

    View full-size slide

  39. Beat Signer - Department of Computer Science - [email protected] 39
    February 12, 2024
    Why, What, Where and When to Evaluate …
    ▪ Where to evaluate
    ▪ laboratory
    ▪ natural setting
    (in-the-wild studies)
    - better for user experience
    ▪ living labs
    ▪ When to evaluate
    ▪ formative evaluations
    - throughout the design process
    - what and how to redesign?
    ▪ summative evaluations
    - assess the final product
    - how well did we do?
    Aware Home, Georgia Tech

    View full-size slide

  40. Beat Signer - Department of Computer Science - [email protected] 40
    February 12, 2024
    UEQ+ User Experience Questionnaire
    ▪ Tool to build customised
    UX questionnaire
    ▪ modular extension of UEQ
    ▪ customised selection of
    UX scales
    ▪ Word templates with
    questions
    - more than 20 languages
    - answers on a 7-point Likert scale
    ▪ UEQ+ Data Analysis Tool
    ▪ Excel document creating
    graphics etc.

    View full-size slide

  41. Beat Signer - Department of Computer Science - [email protected] 41
    February 12, 2024
    UEQ+ Available Scales

    View full-size slide

  42. Beat Signer - Department of Computer Science - [email protected] 42
    February 12, 2024
    Online Survey Tools
    Qualtrics

    View full-size slide

  43. Beat Signer - Department of Computer Science - [email protected] 43
    February 12, 2024
    Assignment
    ▪ Focus first on problem and user needs
    ▪ different interface types and hardware technologies
    should only be considered when designing alternatives
    ▪ Consider the use of user-centred design
    ▪ requirements, prototyping as well as evaluation
    ▪ Iterative design process with rapid low-fidelity prototyping
    ▪ Do not forget to evaluate your solution!

    View full-size slide

  44. Beat Signer - Department of Computer Science - [email protected] 44
    February 12, 2024
    References
    ▪ Interaction Design: Beyond Human-Computer
    Interaction, Yvonne Rogers, Helen Sharp and
    Jenny Preece, Wiley (6th edition), April 2023
    ISBN-13: 978-1119901099
    ▪ Designing with the Mind in Mind: Simple Guide to
    Understanding User Interface Design Guidelines,
    Jeff Johnson, Morgan Kaufmann (3rd edition),
    November 2020, 978-0128182024
    ▪ Human-Computer Interaction, Alan Dix,
    Janet E. Finlay, Gregory D. Abowd and Russell
    Beale, Prentice Hall (3rd edition), December 2003
    ISBN-13: 978-0130461094

    View full-size slide

  45. Beat Signer - Department of Computer Science - [email protected] 45
    February 12, 2024
    References ...
    ▪ Research Methods in Human-Computer Inter-
    action, Jonathan Lazar, Jinjuan Heidi Feng and
    Harry Hochheiser, Morgan Kaufmann (2nd edition),
    May 2019, ISBN-13: 978-0128053904
    ▪ Designing for Interaction: Creating Innovative
    Applications and Devices, Dan Saffer, New Riders
    (2nd edition), August 2009
    ISBN-13: 978-0321643391
    ▪ The Design of Everyday Things, Don Norman,
    Basic Books (revised and expanded edition),
    November 2013, ISBN-13: 978-0465050659

    View full-size slide

  46. Beat Signer - Department of Computer Science - [email protected] 46
    February 12, 2024
    References ...
    ▪ Human Computer Interaction (1023841ANR) course
    ▪ https://wise.vub.ac.be/index.php/course/human-computer-
    interaction
    ▪ Microsoft Office Labs Vision 2019
    ▪ https://www.youtube.com/watch?v=Zp-_oUwdSeY
    ▪ MAXQDA
    ▪ https://www.maxqda.com
    ▪ https://softweb.vub.be
    ▪ Qualtrics XM
    ▪ https://www.qualtrics.com
    ▪ https://softweb.vub.be

    View full-size slide

  47. Beat Signer - Department of Computer Science - [email protected] 47
    February 12, 2024
    References ...
    ▪ UEQ+: A Modular Extension of the User
    Experience Questionnaire
    ▪ https://ueqplus.ueq-research.org

    View full-size slide

  48. 2 December 2005
    Next Lecture
    Information Architectures

    View full-size slide