Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MEMS-sensors in Computer Vision: we underestima...

CEE-SECR
October 20, 2017

MEMS-sensors in Computer Vision: we underestimate them Anastasiya Kornilova, SPbSU, CEE-SECR 2017

A key challenge in such areas of computer vision as video stabilization, 3D-reconstruction, SLAM, VR, is to increase energy efficient and performance of algorithms being implemented. In most cases, a detection of camera movement and rotation is a significant consumer of a computational power. In this report, we discuss how to decrease significantly the number of calculations using MEMS motion sensors (gyroscope, accelerometer) in case of video stabilization task. We consider main difficulties that can be encountered when implementing these algorithms on specific platforms.

This topic would be interesting for specialists in the fields related to computer vision and digital signal processing. The main goal of the report is to present our ongoing research on digital video stabilization and to analyze its application in various fields.

CEE-SECR

October 20, 2017
Tweet

More Decks by CEE-SECR

Other Decks in Technology

Transcript

  1. MEMS-sensors in Computer Vision: we underestimate them Anastasiya KORNILOVA Iakov

    KIRILENKO SPBU, JetBrains Research Software Engineering Conference Russia October 2017, St. Petersburg
  2. Using MEMS-sensors • Instantly • Power consumption ー 2-5 mW

    • Widely used ◦ Smartphones ◦ Embedded systems 7
  3. More complex integration • “Digital Video Stabilization and Rolling Shutter

    Correction using Gyroscopes“ (2012, Standford) • Quaternions • 3D-rotation 14
  4. Oops... Auto calibration Rotation matrix Rolling shutter parameter • Focal

    length • Camera and sensor displacement • Synchronization sensor and camera • Bias of gyroscope 19
  5. • Real-time available • Simple model, but many problems •

    Codecs • 3D-reconstruction Conclusion 21
  6. Statistics • Whatsapp: 250 million messages with video per day

    • Periscope: 350 000 hours streamings per day • Snapchat: 6 billion videos per day 25
  7. Mobile video editors iMovie PowerDirector CameraPlus Pro Camera Genius 4$

    3.5$ 3.5$ 3$ 30 млн 5 млн 10 млн 3 млн 26