Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evaluating Interpreting in Virtual Reality

Evaluating Interpreting in Virtual Reality

New Computing Technologies Animation and Games Workshop- Presentation of the EU LLP project EVIVA, evaluating the performance of VLEs in interpreter-mediated communication and training

Panagiotis D. Ritsos

May 15, 2014
Tweet

More Decks by Panagiotis D. Ritsos

Other Decks in Research

Transcript

  1. Panagiotis “Panos” D. Ritsos, Robert Gittins and Jonathan C. Roberts
    {p.ritsos, r.gittins, j.c.roberts} @bangor.ac.uk
    NEW COMPUTER TECHNOLOGIES WALES – ANIMATION & GAMES WORKSHOP – 15TH MAY 2014

    View Slide

  2. Summary
    • Introduction to our use-case
    • Overview of project EVIVA
    • Brief overview of IVY – Virtual Environment (IVY-VE)
    • Challenges, solutions and opportunities
    • Future steps

    View Slide

  3. Our use case…
    Training and simulation for interpreter-mediated
    communications
    • The rise of migration and multilingualism in Europe requires professional
    interpreters in business, legal, medical and many other settings.
    • Future interpreters therefore need to master an ever broadening range of
    interpreting scenarios and skills.
    • This is difficult to achieve with traditional teaching methods and in times of
    reduced teaching contact hours.
    • Also, a client-side understanding of what working with an interpreter
    involves is crucial, but efforts to educate potential clients of interpreters are
    scarce and normally separate from interpreter education.

    View Slide

  4. Our projects…
    Interpreting in Virtual Reality (2011 – 2012)
    • …address the needs of future interpreters and users of interpreters in higher
    education, vocational training and adult learning contexts
    • …use 3D virtual environment technology to create an adaptive, learning
    environment that supports the acquisition and application of skills required in
    interpreter-mediated communication
    Evaluating the Education of Interpreters and their Clients through Virtual
    Learning Activities (2013 – 2014)
    • …evaluate the educational opportunities that three types of virtual learning
    environments — 3D virtual worlds, videoconference tools and video
    repositories of training material— offer for future interpreters and their clients

    View Slide

  5. Projects relationship
    Timeline to projects relative to EVIVA
    Bangor, Surrey, Tubingen, StC Steinbeis, U. Adam Mickiewicz, U. Cyprus, Bar-Ilan, UVEG
    2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
    EVIVA
    StC, Surrey
    Surrey, StC, UVEG
    Bangor, Surrey, StC, UAM, UCY, Bar-llan
    Bangor, Surrey, StC, UAM, UCY
    ELISA
    BACKBONE
    IVY
    Evaluation & analysis of user behaviour
    in relation to UX, pedagogy in VLEs
    used for interpreting, (including IVY-VE)
    Implementation of a 3D VLE based on the
    synergy of Second Life and web tech
    (Appfuse) through the SL URI namespace
    Repositories of audiovisual
    corpus material of interpreting-
    specific scenarios

    View Slide

  6. EVIVA - aims
    • The aims of EVIVA project are:
    – to investigate the efficiency of VLEs and how they support different learning activities
    – to develop innovative evaluation methods for this investigation
    – to formulate design recommendations and guidelines for VLEs in interpreting
    • By evaluating educational solutions that reflect real-life situations, EVIVA contributes to
    bringing together the worlds of work and education.
    • Outcomes will inform VLE design and will be transferable to other fields of education
    • One of the evaluated VLEs is IVY-VE

    View Slide

  7. Virtual Environment requirements
    • Re-create scenarios, using a large audio corpus, using a multi-user virtual world
    – ~75 scenarios shared around 20 locations
    • Enrich the corpus with more languages (for each scenario)
    – EN, GE, FR, GR, HE, RU, PL to begin with, followed by CH, IT, SP
    • Allow interpreting tutors (who can not program, sigh!) to add content on their own
    • Built three modes into the system: a training/simulation mode, a live mode and an
    exhibition mode.
    • Get the environment working in six months for students to use…

    View Slide

  8. IVY-VE – our plan (circa 2011)
    • Due to time limitations, previous experience and existing assets we chose to
    built our IVY-VE around Second Life virtual world,
    – Advantages
    • easy to fabricate locations for our scenarios
    • easy to `built’ a multiplayer world
    • no special needs for infrastructure
    • wide use in academia and education
    – Disadvantages
    • No instancing, zoning and replication (like in MMOs)
    • No mature implementations for NPCs
    • A bit erratic at the time with maintenance service breaks

    View Slide

  9. IVY-VE – Scenarios
    Interpreter
    Actor
    Observer Interpreter
    Actor A Actor B
    Observer
    Monologue Dialogue
    • Actors can be users or Non-player Characters (NPCs)
    • User is the interpreter – his audio is not recorded at this point

    View Slide

  10. IVY-VE – architecture
    • IVY-VE comprises of three components (plus a database)
    – A scenario management system (SMS) that allows tutors to upload, edit and manage
    content. The scenarios are associated with locations in SL (user class: administrators)
    – The IVY island where most settings are built (user class: all + observers)
    – A Heads-up display, (using the HTML-on-a-prim feature) working as an extension to the
    SL GUI, giving access to the scenarios and initiating in-world events through the SL URI
    namespace (user class: interpreter)
    Virtual
    World
    Scenario Selection Menu (HUD)
    Scenes
    Scenario Management
    IVY-VE Interface
    Second
    Life
    Web
    User
    Administrator
    Observer

    View Slide

  11. IVY-VE – our actors
    • Two types of actors used – `dummies’ and ‘bots’
    – Bots have better appearance, can be controlled by whispers and through a telnet server...
    … but seem to have a mind on their own on zone restarts
    – Dummies are more stable but not realistically animated and of mediocre quality
    – In-house built dummies, (imported from Autodesk Pinocchio Project, tweaked in Maya etc.)

    View Slide

  12. IVY-VE – HUD

    View Slide

  13. IVY-VE – without instancing…
    • Each location can hold more than one scenarios,
    e.g., more than one taking place in an office setting
    • Due to that, we replicated popular locations into our
    skyboxes
    • Once a scenario is selected by a user all scenarios
    taking place on that location become unavailable to
    the remaining users
    • Upon scenario selection the HUD becomes a player
    • Teleportation events (to the scenarios location) are
    initiated
    • The user is required to press a teleport button

    View Slide

  14. Preliminary evaluation
    • IVY-VE has been used in Interpreter training and modules, since 01/2012, in various sites:
    – University of Surrey, Center of Translation Studies
    – Adam Mickiewicz, Department of Translation Studies
    – University of Cyprus, Department of English Studies
    • Preliminary UX evaluations where conducted towards the end of 2012
    • Interpreters appeared quite excited to have IVY-VE…
    • …but are they biased from having a dedicated tool (which they did not in the past)?
    • Longer, more thorough evaluation is required, as well as comparison with other tools
    Interpreters may use
    • This led to the EVIVA project…

    View Slide

  15. EVIVA – adapting IVY-VE
    • Two new scenario hubs added on a seperate SL island.
    – A visitors centre, used for project promotion, access to the public etc.
    – An evaluation centre, used for the evaluation sessions taking place in
    2014
    – `Vanilla’ IVY-VE is still used by postgraduate students
    • New locations, using higher quality 3D assets added throughout the
    two additional hubs
    • Separate SMS and HUDs are used for each island

    View Slide

  16. EVIVA– locations
    Replication of locations on skyboxes Visitors Centre
    Evaluation Centre Additional scenario locations

    View Slide

  17. EVIVA - evaluation

    View Slide

  18. EVIVA – opportunities - I
    • In EVIVA we can explore UX using new interfaces, such as Oculus Rift and the CtrlAltStudio
    Viewer (and soon the SL viewer) and the Razer Hydra

    View Slide

  19. EVIVA – opportunities - II
    • We have also been experimenting with the implementation of a new bespoke system using various
    technologies, like WebGL/Three.js and Unity3D

    View Slide

  20. Future challenges
    • We still have lots to do regarding:
    – Avatars, movement, realism
    – Customizable NPCs
    – Multiplayer functionality (instancing, replication, zoning etc.)
    – Reuse of our Scenario Management System
    – Synchronized facial animations and gestures
    • If this is your area and expertise or share the same questions…
    lets have a chat about it!

    View Slide

  21. Thank you!
    http://www.virtual-interpreting.net
    @vr_interpreting
    @ritsos_p

    View Slide