Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SeeMeSpeak - A RailsRumble Story

F5cf061c14e74f2ac9acb509b46b955f?s=47 Klaus Fl
December 09, 2013

SeeMeSpeak - A RailsRumble Story

SeeMeSpeak.org is a dictionary for sign language. The app was written during the Rails Rumble Hackathon this year and we got to the 8th place. In this talk I want to show you how we successfully wrote this webapp, how we worked and what we plan for the future of the site.

F5cf061c14e74f2ac9acb509b46b955f?s=128

Klaus Fl

December 09, 2013
Tweet

Transcript

  1. A Rails Rumble Story Klaus Fleerkötter

  2. “Hallo Berlin” http://youtu.be/BjIclPJAyHs

  3. - Bodo Tasche (@bitboxer) “We created something I always dreamed

    of: a dictionary for sign language. One that would enable everyone to add participate in it. A real crowd-sourced solution to the problem that is the recording of thousands of signs.”
  4. Team Florian Gilcher @Argorak Bodo Tasche @bitboxer Klaus Fleerkötter @kaeff

    Jan Nietfeld @JanNilpferd
  5. None
  6. None
  7. http://www.flickr.com/photos/blogumentary/72204339 Contributing

  8. Categorization

  9. Challenges • Contributing signs must be frictionless • Multiple (sign

    and spoken) languages • Flexible & forward-compatible categorization
  10. HTML to the resque!

  11. Technology elasticsearch. Ruby On Rails TorqueBox (App server) elasticsearch (Search

    & DB) getUserMedia
  12. Video Recording // Initialization navigator.getUserMedia({ video: true, audio: false },

    function(stream) { mediaRecorder = new MediaStreamRecorder(stream); video.attr("src", window.URL.createObjectURL(stream)); }); ! // Later var captureSpan = function(milliSeconds) { mediaRecorder.ondataavailable = function(blob) { stop(); setBlob(blob); }; mediaRecorder.start(milliSeconds); }
  13. MediaStream Recording [Constructor (MediaStream stream)] interface MediaRecorder : EventTarget {

    readonly attribute MediaStream stream; readonly attribute RecordingStateEnum state; readonly attribute unsigned long imageWidth; readonly attribute unsigned long imageHeight; readonly attribute DOMString mimeType; attribute EventHandler onrecording; attribute EventHandler onstop; attribute EventHandler ondataavailable; attribute EventHandler onpause; attribute EventHandler onresume; attribute EventHandler onerror; attribute EventHandler onwarning; void record (optional long? timeslice); void stop (); void pause (); void resume (); […] }; http://www.w3.org/TR/mediastream-recording/
 Polyfill: https://github.com/streamproc/MediaStreamRecorder W3C Working Draft
  14. XHR2 over jQuery newEntryForm = $("form#new_entry"); form = new FormData(newEntryForm[0]);

    form.append("entry[video]", window.recorder.getBlob()); $.ajax({ url: newEntryForm.attr("action"), type: newEntryForm.attr("method"), processData: false, contentType: false, data: form });
  15. Torquebox

  16. Video Transcription with class VideosTranscoder include TorqueBox::Messaging::Backgroundable always_background :generate_video !

    def initialize(entry, video_file) @entry = entry @video_file = video_file end ! def transcode! # ... end ! def self.generate_video(entry, video_file) VideosTranscoder.new(entry, video_file).transcode! end end Backgroundable
  17. elasticsearch.

  18. “Data model” { "transcription": "hello from seemespeak", "tags": ["funny"], "flags":

    ["casual"], "reviewed": false, "language": "ASL" }
  19. Searching “Hello” { "query": { "match": { "transcription": "Hello" }

    }, "filter": { "bool": { "must": [ { "term": { "reviewed": true } } ], "must_not": [ { "terms": { "flags": [ "insult", "vulgar", "casual" ] }} ] } } }
  20. …there’s more • popcorn.js: Cross-browser video playback! • Virtus: Declaring

    models & attributes, on steroids • Video Conversion: libav via custom shell wrapper • Bing Translator API for transcription translations
  21. None
  22. Saturday Sunday Monday 12:00 am 02:00 am 11:59 pm git

    tag rumble13
  23. Prepare

  24. You're a craftsman, even under pressure

  25. Sleep

  26. Please contribute! • Record signs • Feedback • Design •

    Code https://github.com/seemespeak
  27. Thanks! Klaus Fleerkötter ! @kaeff hi@kaeff.net http://kaeff.net http://seemespeak.org/