SeeMeSpeak - A RailsRumble Story

SeeMeSpeak.org is a dictionary for sign language. The app was written during the Rails Rumble Hackathon this year and we got to the 8th place. In this talk I want to show you how we successfully wrote this webapp, how we worked and what we plan for the future of the site.

Transcript

1. A Rails Rumble Story Klaus Fleerkötter
2. “Hallo Berlin” http://youtu.be/BjIclPJAyHs
3. - Bodo Tasche (@bitboxer) “We created something I always dreamed of: a dictionary for sign language. One that would enable everyone to add participate in it. A real crowd-sourced solution to the problem that is the recording of thousands of signs.”
4. Team Florian Gilcher @Argorak Bodo Tasche @bitboxer Klaus Fleerkötter @kaeff Jan Nietfeld @JanNilpferd
7. http://www.flickr.com/photos/blogumentary/72204339 Contributing
8. Categorization
9. Challenges • Contributing signs must be frictionless • Multiple (sign and spoken) languages • Flexible & forward-compatible categorization
10. HTML to the resque!
11. Technology elasticsearch. Ruby On Rails TorqueBox (App server) elasticsearch (Search & DB) getUserMedia
12. Video Recording // Initialization navigator.getUserMedia({ video: true, audio: false }, function(stream) { mediaRecorder = new MediaStreamRecorder(stream); video.attr("src", window.URL.createObjectURL(stream)); }); ! // Later var captureSpan = function(milliSeconds) { mediaRecorder.ondataavailable = function(blob) { stop(); setBlob(blob); }; mediaRecorder.start(milliSeconds); }
13. MediaStream Recording [Constructor (MediaStream stream)] interface MediaRecorder : EventTarget { readonly attribute MediaStream stream; readonly attribute RecordingStateEnum state; readonly attribute unsigned long imageWidth; readonly attribute unsigned long imageHeight; readonly attribute DOMString mimeType; attribute EventHandler onrecording; attribute EventHandler onstop; attribute EventHandler ondataavailable; attribute EventHandler onpause; attribute EventHandler onresume; attribute EventHandler onerror; attribute EventHandler onwarning; void record (optional long? timeslice); void stop (); void pause (); void resume (); […] }; http://www.w3.org/TR/mediastream-recording/
 Polyfill: https://github.com/streamproc/MediaStreamRecorder W3C Working Draft
14. XHR2 over jQuery newEntryForm = $("form#new_entry"); form = new FormData(newEntryForm[0]); form.append("entry[video]", window.recorder.getBlob()); $.ajax({ url: newEntryForm.attr("action"), type: newEntryForm.attr("method"), processData: false, contentType: false, data: form });
15. Torquebox
16. Video Transcription with class VideosTranscoder include TorqueBox::Messaging::Backgroundable always_background :generate_video ! def initialize(entry, video_file) @entry = entry @video_file = video_file end ! def transcode! # ... end ! def self.generate_video(entry, video_file) VideosTranscoder.new(entry, video_file).transcode! end end Backgroundable
17. elasticsearch.
18. “Data model” { "transcription": "hello from seemespeak", "tags": ["funny"], "flags": ["casual"], "reviewed": false, "language": "ASL" }
19. Searching “Hello” { "query": { "match": { "transcription": "Hello" } }, "filter": { "bool": { "must": [ { "term": { "reviewed": true } } ], "must_not": [ { "terms": { "flags": [ "insult", "vulgar", "casual" ] }} ] } } }
20. …there’s more • popcorn.js: Cross-browser video playback! • Virtus: Declaring models & attributes, on steroids • Video Conversion: libav via custom shell wrapper • Bing Translator API for transcription translations
22. Saturday Sunday Monday 12:00 am 02:00 am 11:59 pm git tag rumble13
23. Prepare
24. You're a craftsman, even under pressure
25. Sleep
26. Please contribute! • Record signs • Feedback • Design • Code https://github.com/seemespeak
27. Thanks! Klaus Fleerkötter ! @kaeff hi@kaeff.net http://kaeff.net http://seemespeak.org/