Amanda Dash, Nora Huang, Dany Cabrera, Tristan Partridge and Maria Ferman SignTalker 

Demo 2 Server: https://drive.google.com/open?id=0B5GYwEJYfpXgbXl2cy1tY3BTRVU Client: https://drive.google.com/open?id=0Bwzg6oj1aTvZWklCdXotZUZIWmc 

What’s happening? 1. Grab color image from camera 2. Use K-Means to do color clustering, and find the biggest color cluster (hand) 3. Get the contour of the hand 4. Take the Discrete Fourier Transform of the contour to obtain a sampled, scale-invarient shape descriptor 5. Match the descriptor against a prepared set of 4 (A-D) ground truth samples using linear interpolation 6. The best match (within a certain margin of error) is returned 

Feature list 4 1. Researching and testing existing technologies M1 2. Shape descriptor for hand position M1 3. REST server to receive shape descriptor data from phone M1 4. REST server to respond with determined word M1 5. Machine Learning framework M1 6. Creating relationship with the ASL community (users) M2 7. Get Dataset M2 8. Train classifier to run on server M2 9. Menu options on phone M2 10. Companion Website M2 11. Phone gets word from REST and outputs as audio (Text -to-Speech) M3 12. Optimizing and improving shape descriptor M3 13. Tutorial/guide/help options M3 

Road Map Ground Truth Dataset Optimize features extraction & classification Text predictor Full App development ASL Users 5 Train Machine Learning Classifier Milestone 2 (4/Nov) 

Our Progress 1. Researching and testing existing technologies 2. Researching and meeting with ASL users 3. Shape descriptor for hand position 4. REST server to receive shape descriptor data from phone 5. REST server to respond with determined word 6. Machine Learning framework 7. Evaluated technical risks a. Time 6 

7 

ASL Learning tool Why to pivot? 1.- We know the letter we are matching 1.1 This makes the search space smaller 2.- Lower the Hardware requirements 3.- Real-time not required 4.- Larger user base 8 

Thanks! Any questions? Amanda Dash - [email protected] Nora Huang - [email protected] Dany Cabrera - [email protected] Tristan Partridge - [email protected] Maria Ferman - [email protected] Presentation template by SlidesCarnival 9 http://104.236.214.96:8080/signtalker/