iGesture: A General Gesture Recognition Framework

Presentation given at ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil.

ABSTRACT: With the emergence of digital pen and paper interfaces, there is a need for gesture recognition tools for digital pen input. While there exists a variety of gesture recognition frameworks, none of them addresses the issues of supporting application developers as well as the designers of new recognition algorithms and, at the same time, can be integrated with new forms of input devices such as digital pens. We introduce iGesture, a Java-based gesture recognition framework focusing on extensibility and cross-application reusability by providing an integrated solution that includes tools for gesture recognition as well as the creation and management of gesture sets for the evaluation and optimisation of new or existing gesture recognition algorithms. In addition to traditional screen-based interaction, iGesture provides a digital pen and paper interface.

Research paper: https://beatsigner.com/publications/igesture-a-general-gesture-recognition-framework.pdf

Transcript

1. ICDAR 2007, September 26 A General Gesture Recognition Framework Beat Signer, Ueli Kurmann and Moira C. Norrie Institute for Information Systems, ETH Zurich 8092 Zurich, Switzerland
2. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 2 Interactive Paper Digital Pen and Paper https://beatsigner.com/interactive- paper.htmlhttps://beatsigner.com/interactiv e-paper.html
3. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 3 Requirements ▪ Java API ▪ Different types of input devices ▪ digital pen and paper, mouse, etc. ▪ Support application and algorithm developers ▪ Definition and evaluation of new gesture sets ▪ Simple acquisition of gesture data
4. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 4 iGesture Components Common Data Structures Management Console Evaluation Tools Recogniser
5. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 5 iGesture Admin Tab Common Data Structures Management Console Evaluation Tools Recogniser
6. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 6 Capture of Samples Common Data Structures Management Console Evaluation Tools Recogniser
7. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 7 iGesture Test Bench Tab Common Data Structures Management Console Evaluation Tools Recogniser
8. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 8 iGesture Test Data Tab Common Data Structures Management Console Evaluation Tools Recogniser
9. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 9 Evaluation Tools Common Data Structures Management Console Evaluation Tools Recogniser
10. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 10 Recogniser API Configuration configuration = XMLTool.importConfiguration(new File(RUBINE)); GestureSet gestureSet = XMLTool.importGestureSet(new File(RUBINE_SET)); configuration.addGestureSet(gestureSet); recogniser = new Recogniser(configuration); ... ResultSet result = recogniser.recognise(note); Common Data Structures Management Console Evaluation Tools Recogniser
11. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 11 Gesture Representation Common Data Structures Management Console Evaluation Tools Recogniser
12. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 12 Recognition Algorithm Common Data Structures Management Console Evaluation Tools Recogniser
13. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 13 Persistence Common Data Structures Management Console Evaluation Tools Recogniser
14. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 14 Current Status ▪ Algorithms ▪ Rubine, SiGeR, E-Rubine, SiGrid ▪ Gesture Sets ▪ Graffiti numbers, Graffiti letters, MS application gestures, customised multi-stroke gestures ▪ Input Devices ▪ Anoto digital pen (e.g. Magicomm G303), mouse
15. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 15 Conclusions ▪ Simple gesture recogniser Java API ▪ Experimental platform for implementing and testing new gesture recognition algorithms ▪ Definition and evaluation of new gesture sets ▪ Extensibility in terms of ▪ input devices ▪ algorithms ▪ gestures sets
16. ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 16 References ▪ B. Signer, M. C. Norrie and U. Kurmann. iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms. Technical Report ETH Zurich TR 561, September 2007 ▪ iGesture Framework, Apache License, V2.0 ▪ http://www.igesture.org