Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Convolutional Pose Machines

Convolutional Pose Machines

52cd889740cd8ef55de65d3babbca0df?s=128

contaconta

July 17, 2016
Tweet

Transcript

  1. Convolutional Pose Machines @conta_

  2. 緒方 貴紀 (twitter: @conta_) CTO@ABEJA, Inc. Computer Visionとか、Machine Learningを使った プロダクト開発をやっています。 Self

    Introduction
  3. Pose Estimation?

  4. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures

    [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]
  5. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures

    [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]
  6. Pose Machines? Confidence Maps [Ramakrishna et al., 2014]

  7. Pose Machines [Ramakrishna et al., 2014] パッチから特徴量を抽出し、各Parts or NotのClassifier を用いて、Confidence

    Mapsを作りたい
  8. Pose Machines [Ramakrishna et al., 2014] 局所的な特徴を用いた推定はWeak Partsによっては、局所的特徴だど推定できない。。。

  9. Part Contextは非常に有効な特徴 Pose Machines [Ramakrishna et al., 2014] 前段階での推定結果を用いて、各Partsの関係性を 事前情報無しにを活用出来ないか?

  10. Pose Machines Stage I Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features [Ramakrishna et al., 2014]
  11. Pose Machines Stage II Confidence g2 g1 g3 Context Features

    Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
  12. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
  13. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
  14. Pose Machines (Previous Work)

  15. Pose Machines (Previous Work) HoG Random Forests

  16. Convolutional Pose Machines (CPM) Deep Deep

  17. Architecture of CPM 階層的なCNNによるPose Machinesの実現

  18. Stage1 Stage1: Localな特徴量の学習 368x368のInputに対して160x160の範囲をカバー

  19. 各Parts + BackgroundのConfidence Maps (P+1) *MPII Human Pose Datasetだと P=14

    Outputs
  20. Stage 2 Stage2: Localな特徴量 + Part Contextによる学習

  21. Stage T Stage2と同じ構成のネットワークを積み上げていく *本研究ではStage6まで積み上げる

  22. 各StageのConfidence Mapsと教師データとのEuclidean Distance Loss 教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの Loss

    Function
  23. Stage2以降、全段階のConfidence Mapのおかげで 良い推定ができている Spatial context from belief maps

  24. 3つのDatasetsで実験 - MPII Human Pose Dataset - Leeds Sports Pose

    (LSP) Datase - FLIC Dataset Experiments
  25. Results

  26. Results Stageは積み上げるとイイんやで

  27. Results Pose Machinesを上回る精度 Stageを重ねるごとに精度は向上

  28. Results State-of-the-art Performance (ドヤァ

  29. 実装してみた

  30. 1. (色々頑張って実験した結果)いい感じの連続構成 CNNによって、暗黙的な空間モデルの学習ができた 2. Graphical Modelによる推論無しに、階層構造の Predictionができた Conclusion

  31. We are hiring! → https://www.wantedly.com/ companies/abeja 博士持ち大歓迎!