Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fitness Motion Recognition with Android Wear

Fitness Motion Recognition with Android Wear

Counting is what computers do best and we should let them do it whenever possible. That's true for spreadsheets and it's true for fitness. Wearables exist to count your steps, measure the distance you run, and track how your pulse races after a workout. So why are we still counting pushups, situps, and burpees like they did in the Stone Age?

In this presentation, I will talk about the steps necessary to implement this kind of motion recognition on Android Wear:

- Measuring the motion being recorded by the device
- Deriving a pattern that represents the motion you want to recognize
- Implementing the pattern recognition in the most battery-efficient manner possible

Edward Dale

June 05, 2015
Tweet

More Decks by Edward Dale

Other Decks in Technology

Transcript

  1. Fitness Motion
    Recognition
    with
    Android Wear
    Edward Dale
    Freeletics
    © Edward Dale, 2015 1

    View Slide

  2. http://www.someecards.com/usercards/viewcard/MjAxMy1hMjIwMWUzMTc4NDgyOTA1
    © Edward Dale, 2015 2

    View Slide

  3. Agenda
    • Define scope
    • Sensors
    • Algorithms
    • Battery Efficiency
    © Edward Dale, 2015 3

    View Slide

  4. Defining the problem scope
    • Segmenting exercise from non-exercise
    • Recognizing which exercise is being performed
    • Counting repetitions
    • Online
    © Edward Dale, 2015 4

    View Slide

  5. Defining the problem scope
    • Segmenting exercise from non-exercise
    • Recognizing which exercise is being performed
    • Counting repetitions
    • Online
    © Edward Dale, 2015 5

    View Slide

  6. Defining the problem scope
    • Segmenting exercise from non-exercise
    • Recognizing which exercise is being performed
    • Counting repetitions
    • Online
    © Edward Dale, 2015 6

    View Slide

  7. Defining the problem scope
    • Segmenting exercise from non-exercise
    • Recognizing which exercise is being performed
    • Counting repetitions !
    • Online
    © Edward Dale, 2015 7

    View Slide

  8. Defining the problem scope
    • Segmenting exercise from non-exercise
    • Recognizing which exercise is being performed
    • Counting repetitions !
    • Online !
    © Edward Dale, 2015 8

    View Slide

  9. Which "Fitness Motion"?
    • Running
    • Swimming
    • Cycling
    © Edward Dale, 2015 9

    View Slide

  10. This "Fitness Motion"
    • Pushups
    • Jumping Jacks
    • Burpees
    © Edward Dale, 2015 10

    View Slide

  11. Pushups
    © Edward Dale, 2015 11

    View Slide

  12. © Edward Dale, 2015 12

    View Slide

  13. Pushup Sensors
    • Proximity (Sensor.TYPE_PROXIMITY)
    © Edward Dale, 2015 13

    View Slide

  14. Pushup Sensors
    • Proximity (Sensor.TYPE_PROXIMITY)
    • Rotation (Sensor.TYPE_GAME_ROTATION_VECTOR,
    Sensor.TYPE_GEOMAGNETIC_ROTATION_VECTOR,
    Sensor.TYPE_GYROSCOPE, Sensor.TYPE_ROTATION_VECTOR)
    © Edward Dale, 2015 14

    View Slide

  15. Pushup Sensors
    • Proximity (Sensor.TYPE_PROXIMITY)
    • Rotation (Sensor.TYPE_GAME_ROTATION_VECTOR,
    Sensor.TYPE_GEOMAGNETIC_ROTATION_VECTOR,
    Sensor.TYPE_GYROSCOPE, Sensor.TYPE_ROTATION_VECTOR)
    • Acceleration (Sensor.TYPE_ACCELEROMETER,
    Sensor.TYPE_LINEAR_ACCELERATION,
    Sensor.TYPE_GRAVITY)
    © Edward Dale, 2015 15

    View Slide

  16. Pushup Sensors
    • Proximity (Sensor.TYPE_PROXIMITY)
    • Rotation (Sensor.TYPE_GAME_ROTATION_VECTOR,
    Sensor.TYPE_GEOMAGNETIC_ROTATION_VECTOR,
    Sensor.TYPE_GYROSCOPE, Sensor.TYPE_ROTATION_VECTOR)
    • Acceleration (Sensor.TYPE_ACCELEROMETER,
    Sensor.TYPE_LINEAR_ACCELERATION,
    Sensor.TYPE_GRAVITY) !
    © Edward Dale, 2015 16

    View Slide

  17. Acceleration Sensors
    TYPE_ACCELEROMETER uses the accelerometer and only the
    accelerometer. It returns raw accelerometer events, with minimal
    or no processing at all.
    TYPE_LINEAR_ACCELERATION and TYPE_GRAVITY ... are "fused"
    sensors
    — Mathias Agopian on android-developers
    Always returns 3 components of acceleration vector
    © Edward Dale, 2015 17

    View Slide

  18. Acceleration Vector
    What to do with acceleration direction?
    Pushup acceleration happens in primary one direction
    Ignore acceleration direction and just use magnitude
    © Edward Dale, 2015 18

    View Slide

  19. © Edward Dale, 2015 19

    View Slide

  20. © Edward Dale, 2015 20

    View Slide

  21. Not so fast
    Still have to count
    But there are well-known algorithms for that
    Google: Online peak detection algorithm
    © Edward Dale, 2015 21

    View Slide

  22. Peakdet
    A point is considered a maximum peak if it has the maximal value,
    and was preceded (to the left) by a value lower by DELTA.
    -- http://www.billauer.co.il/peakdet.html
    © Edward Dale, 2015 22

    View Slide

  23. © Edward Dale, 2015 23

    View Slide

  24. © Edward Dale, 2015 24

    View Slide

  25. Peakdet
    • Online !
    • Efficient !
    • Sensitive to DELTA parameter "
    © Edward Dale, 2015 25

    View Slide

  26. Battery Efficiency
    • Analyze fewer samples
    • Do less analysis per sample
    • Analyze sample on the phone
    • Choose less power-hungry sensors
    • Watch the Power Optimization for Android talk from day 1
    © Edward Dale, 2015 26

    View Slide

  27. Battery Efficiency
    Analyze fewer samples
    • Register for sensor updates with lowest sampling frequency
    necessary
    • SENSOR_DELAY_NORMAL (5Hz)
    • SENSOR_DELAY_UI (15Hz)
    • SENSOR_DELAY_GAME (50Hz)
    • SENSOR_DELAY_FASTEST (~∞Hz)
    © Edward Dale, 2015 27

    View Slide

  28. Battery Efficiency
    Analyze fewer samples
    • Register for sensor updates with lowest sampling frequency
    necessary
    • Also possible to suggest your own sampling frequency
    • Just a suggestion to the device
    © Edward Dale, 2015 28

    View Slide

  29. Battery Efficiency
    Do less analysis per sample
    • Choose an efficient algorithm
    • Peakdet is relatively efficient
    • More efficient than algorithms using derivates
    © Edward Dale, 2015 29

    View Slide

  30. Battery Efficiency
    Analyze samples on the phone
    Just use the watch as a wearable sensor that sends data to be
    analyzed on the phone.
    © Edward Dale, 2015 30

    View Slide

  31. Battery Efficiency
    Choose Less Power-Hungry Sensors
    • Sensor power drain will differ on different hardware
    • Ask the sensor how much power the sensor uses
    Sensor.getPower()
    © Edward Dale, 2015 31

    View Slide

  32. Thanks!
    Edward Dale (@scompt)
    Freeletics (We're hiring)
    © Edward Dale, 2015 32

    View Slide

  33. Links
    • Walk Detection and Step Counting on
    Unconstrained Smartphones
    • RecoFit: Using a Wearable Sensor to
    Find, Recognize, and Count Repetitive Exercises
    • Sample Project
    © Edward Dale, 2015 33

    View Slide