Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Moodstocks - Paris Tech Talks #6

Moodstocks - Paris Tech Talks #6

An overview of Mobile Image Recognition, a presentation of Moodstocks' approach and a focus on Moodstocks SDK core library for Paris Tech Talks Meetup #6.

* https://moodstocks.com/
* http://www.meetup.com/Paris-Tech-Talks/events/165819522/
* https://github.com/Moodstocks/jpec

5d911b7a5d9158ad9fdcc6a493c7333c?s=128

Cédric Deltheil

March 11, 2014
Tweet

Transcript

  1. Supersonic image recognition for your mobile apps Cédric @deltheil Paris

    Tech Talks #6 Moodstocks Tech co-founder
  2. Mobile Image Recognition Your Mobile App Reference Images + ID-s

    Image Recognition Platform query match ID = 1234 index user developer ID = 1234
  3. The old way: thin client Mobile App Image Recognition API

    client server reference images send JPEG image via HTTP match ID or nil
  4. Drawback: network latencies!

  5. Our approach: thick client Mobile App Moodstocks API client server

    reference images up to millions images Moodstocks SDK image signatures up to 10k images query match ID or nil server-side search for large DB sync
  6. Benefit: speed a typical search runs in a few ms

  7. Moodstocks SDK Mobile App Wrapper Core Library Objective-C Java C

  8. Moodstocks SDK Mobile App Wrapper Core Library we will now

    focus on this part
  9. Design Goal #1 Portability • core library written in C99

    • +99% code in common between iOS / Android • e.g supporting Intel CPU-s on Android was… no effort!
  10. Design Goal #2 Efficiency • in-memory data structures for maximum

    speed • leverage NEON capabilities (SIMD) where applicable • a typical search runs in ~ 30 ms / 10k images (iPhone 5S)
  11. Design Goal #3 Low footprint • 100% home made, lightweight

    implementation • implement & embed only what we need! • around 800 kB for the iOS ARMv7s release build (-Os)
  12. in 1/2 day Our choices pay off

  13. Core Library Architecture Public Header File Internal Modules 3rd Party

    Libraries Image Recognition Pipeline
  14. Public Header File Internal Modules 3rd Party Libraries Image Recognition

    Pipeline
  15. Image Recognition Pipeline • full-stack: the whole recognition occurs on-device!

    • robust to camera noises (blur, lighting, …) • supports 1D and 2D barcodes too Image Recognition Pipeline
  16. Public Header File Internal Modules 3rd Party Libraries Image Recognition

    Pipeline
  17. Internal Modules (1/4) • master-slave sync engine for image signatures

    • efficient: works with diff-s & data bundles (less requests) • fail-safe: retries + the client always stays consistent Sync Storage Logs API client
  18. Internal Modules (2/4) Sync Storage Logs API client • collects

    search results & metadata (OS, etc) along time • a long-running thread posts batches every 30s • tolerant to network errors -> HTTP 409 (Already Exists)
  19. Internal Modules (3/4) • persists image signatures on-disk • persists

    search results events on-disk • used as a temp on-disk buffer at sync time Sync Storage Logs API client
  20. Internal Modules (4/4) • used for server-side recognition (send image

    via HTTP) • includes a pool to re-use connections • also used to post logs batches to the server (HTTP) Sync Storage Logs API client
  21. Public Header File Internal Modules 3rd Party Libraries Image Recognition

    Pipeline
  22. 3rd Party Libraries (1/4) libcurl Tokyo Cabinet jpec msgpack •

    the +500M users file transfer library • multi API -> multiple transfers in the same thread • we use it as the backend of our sync engine + HTTP reqs.
  23. 3rd Party Libraries (2/4) libcurl Tokyo Cabinet jpec msgpack •

    key / value embedded database (hash / table / B+tree) • … and also a fantastic C library (see tcutil.h) • we use it as the backend of our storage engine
  24. 3rd Party Libraries (3/4) libcurl Tokyo Cabinet jpec msgpack •

    our home-made, tiny JPEG encoder (600 LOC-s) • we open sourced it -> github.com/Moodstocks/jpec • we use it for server-side requests (send image)
  25. 3rd Party Libraries (4/4) libcurl Tokyo Cabinet jpec msgpack •

    efficient binary serialization format • we use it to pack the data we synchronize
  26. Thanks! Questions? Comments? cedric AT moodstocks DOT com | @deltheil

  27. Building • we use simple Makefile-s and shell scripts •

    we cross-compile w/ the iOS and Android toolchains • we combine all the arch-s into a fat static lib (armv7, …)
  28. Testing • we write unit tests with ct (Easy Unit

    Testing) by @krarick • we run them on device! e.g jailbreaked iPhone + openssh • we use a mock to test the sync engine
  29. Quality • no warnings: we compile with -Wall -Werror •

    static analysis with Clang Static Analyzer • memory leak checks with Valgrind