Tweet
Tweet

Slide 1

Slide 1 text

Convolutional Pose Machines @conta_

Slide 2

Slide 2 text

緒方　貴紀 (twitter: @conta_) CTO@ABEJA, Inc. Computer Visionとか、Machine Learningを使った プロダクト開発をやっています。 Self Introduction

Slide 3

Slide 3 text

Pose Estimation?

Slide 4

Slide 4 text

Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]

Slide 5

Slide 5 text

Related Work Pictorial structures Hierarchical models Sequential prediction Convolutional architectures [A. Toshev and C. Szegedy, CVPR’2013] [Tian et al., ICCV’2011] [Mykhaylo et al., CVPR’2009] [Ramakrishna et al., 2014]

Slide 6

Slide 6 text

Pose Machines? Confidence Maps [Ramakrishna et al., 2014]

Slide 7

Slide 7 text

Pose Machines [Ramakrishna et al., 2014] パッチから特徴量を抽出し、各Parts or NotのClassifier を用いて、Confidence Mapsを作りたい

Slide 8

Slide 8 text

Pose Machines [Ramakrishna et al., 2014] 局所的な特徴を用いた推定はWeak Partsによっては、局所的特徴だど推定できない。。。

Slide 9

Slide 9 text

Part Contextは非常に有効な特徴 Pose Machines [Ramakrishna et al., 2014] 前段階での推定結果を用いて、各Partsの関係性を 事前情報無しにを活用出来ないか？

Slide 10

Slide 10 text

Pose Machines Stage I Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features [Ramakrishna et al., 2014]

Slide 11

Slide 11 text

Pose Machines Stage II Confidence g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

Slide 12

Slide 12 text

Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

Slide 13

Slide 13 text

Pose Machines Stage III Confidence Head Neck L-Shoulder L-Elbow L-Wrist g2 g1 g3 Context Features Context Features Stage I Confidence Maps Stage II Confidence Maps Stage III Confidence Maps Image Features Head Neck L-Shoulder L-Elbow L-Wrist [Ramakrishna et al., 2014]

Slide 14

Slide 14 text

Pose Machines (Previous Work)

Slide 15

Slide 15 text

Pose Machines (Previous Work) HoG Random Forests

Slide 16

Slide 16 text

Convolutional Pose Machines (CPM) Deep Deep

Slide 17

Slide 17 text

Architecture of CPM 階層的なCNNによるPose Machinesの実現

Slide 18

Slide 18 text

Stage1 Stage1: Localな特徴量の学習 368x368のInputに対して160x160の範囲をカバー

Slide 19

Slide 19 text

各Parts + BackgroundのConfidence Maps (P+1) ＊MPII Human Pose Datasetだと P=14 Outputs

Slide 20

Slide 20 text

Stage 2 Stage2: Localな特徴量 + Part Contextによる学習

Slide 21

Slide 21 text

Stage T Stage2と同じ構成のネットワークを積み上げていく ＊本研究ではStage6まで積み上げる

Slide 22

Slide 22 text

各StageのConfidence Mapsと教師データとのEuclidean Distance Loss 教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの Loss Function

Slide 23

Slide 23 text

Stage2以降、全段階のConfidence Mapのおかげで 良い推定ができている Spatial context from belief maps

Slide 24

Slide 24 text

3つのDatasetsで実験 - MPII Human Pose Dataset - Leeds Sports Pose (LSP) Datase - FLIC Dataset Experiments

Slide 25

Slide 25 text

Results

Slide 26

Slide 26 text

Results Stageは積み上げるとイイんやで

Slide 27

Slide 27 text

Results Pose Machinesを上回る精度 Stageを重ねるごとに精度は向上

Slide 28

Slide 28 text

Results State-of-the-art Performance (ﾄﾞﾔｧ

Slide 29

Slide 29 text

実装してみた

Slide 30

Slide 30 text

1. (色々頑張って実験した結果)いい感じの連続構成 CNNによって、暗黙的な空間モデルの学習ができた 2. Graphical Modelによる推論無しに、階層構造の Predictionができた Conclusion

Slide 31

Slide 31 text

We are hiring! → https://www.wantedly.com/ companies/abeja 博士持ち大歓迎！