Convolutional Pose Machines

Transcript

1. Convolutional Pose Machines @conta_

2. 緒方　貴紀 (twitter: @conta_) CTO@ABEJA, Inc. Computer Visionとか、Machine Learningを使った プロダクト開発をやっています。 Self

   Introduction

3. Pose Estimation?

4. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures

   [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]

5. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures

   [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]

6. Pose Machines? Confidence Maps [Ramakrishna et al., 2014]

7. Pose Machines [Ramakrishna et al., 2014] パッチから特徴量を抽出し、各Parts or NotのClassifier を用いて、Confidence

   Mapsを作りたい

8. Pose Machines [Ramakrishna et al., 2014] 局所的な特徴を用いた推定はWeak Partsによっては、局所的特徴だど推定できない。。。

9. Part Contextは非常に有効な特徴 Pose Machines [Ramakrishna et al., 2014] 前段階での推定結果を用いて、各Partsの関係性を 事前情報無しにを活用出来ないか？

10. Pose Machines Stage I Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features [Ramakrishna et al., 2014]

11. Pose Machines Stage II Confidence g2 g1 g3 Context Features

    Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

12. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

13. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist

    g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

14. Pose Machines (Previous Work)

15. Pose Machines (Previous Work) HoG Random Forests

16. Convolutional Pose Machines (CPM) Deep Deep

17. Architecture of CPM 階層的なCNNによるPose Machinesの実現

18. Stage1 Stage1: Localな特徴量の学習 368x368のInputに対して160x160の範囲をカバー

19. 各Parts + BackgroundのConfidence Maps (P+1) ＊MPII Human Pose Datasetだと P=14

    Outputs

20. Stage 2 Stage2: Localな特徴量 + Part Contextによる学習

21. Stage T Stage2と同じ構成のネットワークを積み上げていく ＊本研究ではStage6まで積み上げる

22. 各StageのConfidence Mapsと教師データとのEuclidean Distance Loss 教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの Loss

    Function

23. Stage2以降、全段階のConfidence Mapのおかげで 良い推定ができている Spatial context from belief maps

24. 3つのDatasetsで実験 - MPII Human Pose Dataset - Leeds Sports Pose

    (LSP) Datase - FLIC Dataset Experiments

25. Results

26. Results Stageは積み上げるとイイんやで

27. Results Pose Machinesを上回る精度 Stageを重ねるごとに精度は向上

28. Results State-of-the-art Performance (ﾄﾞﾔｧ

29. 実装してみた

30. 1. (色々頑張って実験した結果)いい感じの連続構成 CNNによって、暗黙的な空間モデルの学習ができた 2. Graphical Modelによる推論無しに、階層構造の Predictionができた Conclusion

31. We are hiring! → https://www.wantedly.com/ companies/abeja 博士持ち大歓迎！