Convolutional Pose Machines

Transcript

1. Convolutional Pose Machines @conta_
2. 緒方　貴紀 (twitter: @conta_) CTO@ABEJA, Inc. Computer Visionとか、Machine Learningを使った プロダクト開発をやっています。 Self Introduction
3. Pose Estimation?
4. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]
5. Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]
6. Pose Machines? Confidence Maps [Ramakrishna et al., 2014]
7. Pose Machines [Ramakrishna et al., 2014] パッチから特徴量を抽出し、各Parts or NotのClassifier を用いて、Confidence Mapsを作りたい
8. Pose Machines [Ramakrishna et al., 2014] 局所的な特徴を用いた推定はWeak Partsによっては、局所的特徴だど推定できない。。。
9. Part Contextは非常に有効な特徴 Pose Machines [Ramakrishna et al., 2014] 前段階での推定結果を用いて、各Partsの関係性を 事前情報無しにを活用出来ないか？
10. Pose Machines Stage I Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features [Ramakrishna et al., 2014]
11. Pose Machines Stage II Confidence g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
12. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
13. Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]
14. Pose Machines (Previous Work)
15. Pose Machines (Previous Work) HoG Random Forests
16. Convolutional Pose Machines (CPM) Deep Deep
17. Architecture of CPM 階層的なCNNによるPose Machinesの実現
18. Stage1 Stage1: Localな特徴量の学習 368x368のInputに対して160x160の範囲をカバー
19. 各Parts + BackgroundのConfidence Maps (P+1) ＊MPII Human Pose Datasetだと P=14 Outputs
20. Stage 2 Stage2: Localな特徴量 + Part Contextによる学習
21. Stage T Stage2と同じ構成のネットワークを積み上げていく ＊本研究ではStage6まで積み上げる
22. 各StageのConfidence Mapsと教師データとのEuclidean Distance Loss 教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの Loss Function
23. Stage2以降、全段階のConfidence Mapのおかげで 良い推定ができている Spatial context from belief maps
24. 3つのDatasetsで実験 - MPII Human Pose Dataset - Leeds Sports Pose (LSP) Datase - FLIC Dataset Experiments
25. Results
26. Results Stageは積み上げるとイイんやで
27. Results Pose Machinesを上回る精度 Stageを重ねるごとに精度は向上
28. Results State-of-the-art Performance (ﾄﾞﾔｧ
29. 実装してみた
30. 1. (色々頑張って実験した結果)いい感じの連続構成 CNNによって、暗黙的な空間モデルの学習ができた 2. Graphical Modelによる推論無しに、階層構造の Predictionができた Conclusion
31. We are hiring! → https://www.wantedly.com/ companies/abeja 博士持ち大歓迎！