Amanda Dash, Nora Huang, Dany Cabrera, Tristan Partridge and Maria Ferman Milestone 2 SignCoach 1

Introduction We are developing an Educational Android Application to help users learn American Sign Language Spelling using Computer Vision. SignCoach 2

3 Users’ Feedback: - Susanne Harnden - Island Deaf + Hard of Hearing Centre - Nigel Howard - UVIC ASL Professor - Jen Moxam - Member of the UVIC ASL club Why did we change?

Persona: Susan ● Type: UVic student ● Role: user ● Problem: Susan is enrolled in ASL 100, and she would love to have an ASL education tool in her mobile phone to study and have an automatic feedback on whether she is right. 4

5

Mock UI (Interactive prototype) 6 https://projects.invisionapp.com/share/S49501VVU#/screens/201079137

Overall Progress ● Trained and can use Haar cascade classifiers to identify signs (Amanda, Nora) ● Mock UI for App (Maria, Dany) ● Basic NDK C++ integration(Dany) ● UI partially implemented (Tristan) ● Companion website and APK host (Maria) 7

Difficulties ● When we pivoted had to scrap all Android UI developed ● NDK C++ integration issues ● Small training dataset for classifiers ● Acquiring users 8

Milestone 3a 08/11/2016 Updated Milestones Milestone 3b 18/11/2016 Milestone 3c 4/12/2016 ● Improve training data ● Retrain classifiers ● Test framework ● Fix camera view bug ● Teaching and learning work flows ● Posting APK to companion website ● Refine UI visual aesthetics ● Improve accuracy ● Fixing bugs ● Responding to users feedback ● Stretch goals (assisted learning algorithm) ● Increase accuracy 9

10 Make a video doing the ASL Alphabet and you can win one signcoachasl@gmail.com

Thanks! Any questions? Amanda Dash - adash42@uvic.ca Nora Huang - norah@uvic.ca Dany Cabrera - dcabrera@uvic.ca Tristan Partridge - tpart526@uvic.ca Maria Ferman - mfermang@uvic.ca ◎ Presentation template by SlidesCarnival 11 SignCoach