Upgrade to Pro — share decks privately, control downloads, hide ads and more …

iGesture: A General Gesture Recognition Framework

Beat Signer
PRO
September 27, 2007

iGesture: A General Gesture Recognition Framework

Presentation given at ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil.

ABSTRACT: With the emergence of digital pen and paper interfaces, there is a need for gesture recognition tools for digital pen input. While there exists a variety of gesture recognition frameworks, none of them addresses the issues of supporting application developers as well as the designers of new recognition algorithms and, at the same time, can be integrated with new forms of input devices such as digital pens. We introduce iGesture, a Java-based gesture recognition framework focusing on extensibility and cross-application reusability by providing an integrated solution that includes tools for gesture recognition as well as the creation and management of gesture sets for the evaluation and optimisation of new or existing gesture recognition algorithms. In addition to traditional screen-based interaction, iGesture provides a digital pen and paper interface.

Research paper: https://beatsigner.com/publications/igesture-a-general-gesture-recognition-framework.pdf

Beat Signer
PRO

September 27, 2007
Tweet

More Decks by Beat Signer

Other Decks in Science

Transcript

  1. ICDAR 2007, September 26
    A General Gesture Recognition Framework
    Beat Signer, Ueli Kurmann and Moira C. Norrie
    Institute for Information Systems, ETH Zurich
    8092 Zurich, Switzerland

    View Slide

  2. ICDAR 2007, September 26 Beat Signer, [email protected] 2
    Interactive Paper
    Digital Pen and Paper
    https://beatsigner.com/interactive-
    paper.htmlhttps://beatsigner.com/interactiv
    e-paper.html

    View Slide

  3. ICDAR 2007, September 26 Beat Signer, [email protected] 3
    Requirements
    ▪ Java API
    ▪ Different types of input devices
    ▪ digital pen and paper, mouse, etc.
    ▪ Support application and algorithm developers
    ▪ Definition and evaluation of new gesture sets
    ▪ Simple acquisition of gesture data

    View Slide

  4. ICDAR 2007, September 26 Beat Signer, [email protected] 4
    iGesture Components
    Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  5. ICDAR 2007, September 26 Beat Signer, [email protected] 5
    iGesture Admin Tab Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  6. ICDAR 2007, September 26 Beat Signer, [email protected] 6
    Capture of Samples Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  7. ICDAR 2007, September 26 Beat Signer, [email protected] 7
    iGesture Test Bench Tab Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  8. ICDAR 2007, September 26 Beat Signer, [email protected] 8
    iGesture Test Data Tab Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  9. ICDAR 2007, September 26 Beat Signer, [email protected] 9
    Evaluation Tools Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  10. ICDAR 2007, September 26 Beat Signer, [email protected] 10
    Recogniser API
    Configuration configuration =
    XMLTool.importConfiguration(new File(RUBINE));
    GestureSet gestureSet =
    XMLTool.importGestureSet(new File(RUBINE_SET));
    configuration.addGestureSet(gestureSet);
    recogniser = new Recogniser(configuration);
    ...
    ResultSet result = recogniser.recognise(note);
    Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  11. ICDAR 2007, September 26 Beat Signer, [email protected] 11
    Gesture Representation Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  12. ICDAR 2007, September 26 Beat Signer, [email protected] 12
    Recognition Algorithm Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  13. ICDAR 2007, September 26 Beat Signer, [email protected] 13
    Persistence Common Data Structures
    Management
    Console
    Evaluation
    Tools
    Recogniser

    View Slide

  14. ICDAR 2007, September 26 Beat Signer, [email protected] 14
    Current Status
    ▪ Algorithms
    ▪ Rubine, SiGeR, E-Rubine, SiGrid
    ▪ Gesture Sets
    ▪ Graffiti numbers, Graffiti letters, MS application
    gestures, customised multi-stroke gestures
    ▪ Input Devices
    ▪ Anoto digital pen (e.g. Magicomm G303), mouse

    View Slide

  15. ICDAR 2007, September 26 Beat Signer, [email protected] 15
    Conclusions
    ▪ Simple gesture recogniser Java API
    ▪ Experimental platform for implementing and
    testing new gesture recognition algorithms
    ▪ Definition and evaluation of new gesture sets
    ▪ Extensibility in terms of
    ▪ input devices
    ▪ algorithms
    ▪ gestures sets

    View Slide

  16. ICDAR 2007, September 26 Beat Signer, [email protected] 16
    References
    ▪ B. Signer, M. C. Norrie and U. Kurmann. iGesture: A
    Java Framework for the Development and Deployment
    of Stroke-Based Online Gesture Recognition Algorithms.
    Technical Report ETH Zurich TR 561, September 2007
    ▪ iGesture Framework,
    Apache License, V2.0
    ▪ http://www.igesture.org

    View Slide