AAAI-2016 Invited Talk

1507f22ca8d84c83e52362f69e428698?s=47 Ozan Sener
February 15, 2016

AAAI-2016 Invited Talk

Extended presentation of my RSS paper rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

1507f22ca8d84c83e52362f69e428698?s=128

Ozan Sener

February 15, 2016
Tweet

Transcript

  1. rCRF - Recursive Conditional Random Fields for Human Activity Understanding

    Ozan Sener, Ashutosh Saxena
  2. courtesy of Koppula

  3. courtesy of Koppula

  4. None
  5. How to Find MAP Compute features Define the energy function

    Solve the Combinatorial Optimization Activity/Object labels for the past and future Sample Possible Futures [Koppula et al RSS/ICML, Kitani et al ECCV, Jiang et al. RSS/CVPR, Lan et al ECCV, Hu et al RSS/ICRA ]
  6. None
  7. Shortcomings Compute features Define the energy function Solve the Combinatorial

    Optimization Activity/Object labels for the past and future Sample Possible Futures We also need probabilities in addition to MAP solution (future is unknown)
  8. Shortcomings Compute features Define the energy function Solve the Combinatorial

    Optimization Activity/Object labels for the past and future Sample Possible Futures We also need probabilities in addition to MAP solution (future is unknown) on Space with dimension~ 1o6xT ~ 103600 (#ObjLabels#Objectsx#ActLabels)Time
  9. Tale of Two Approaches Graphical Models (discriminative) Can handle exponentially

    large space Recursive Bayesian (generative) Can compute full belief and reason about uncertinity
  10. Structured Diversity Although the state dimensionality is high, probability concentrates

    on a few modes
  11. Structured Diversity Modes are likely and structurally diverse yt,i =

    arg max y belt ( y ) s.t. ( y, yt,i ) 8j < i
  12. HMM – Recursive Belief Estimation HMM Derivation [Rabiner] belt( y

    ) / p( y t = y | x 1, . . . , x t) | {z } ↵t(y) p( x t+1, . . . , x T | y t = y ) | {z } t(y) ↵t( y t) = p( x t| y t) X yt 1 ↵t 1( y t 1)p( y t| y t 1) t( y t) = X yt+1 p( x t+1| y t+1) t+1( y t+1)p( y t+1| y t)
  13. rCRF: Structured Diversity meets HMM A Belief over rCRF is

    a CRF bel ( yt ) / exp 2 4 X v,w2Et ⇣ Eb ( v, w) ˜ Eb ( v, w) ⌘ X v2Vt 0 @Eu ( v) ˜ Eu ( v) + X yt 1 ↵t 1 ( yt 1 ) log p ( yt v |yt 1 v ) 1 X yt+1 t+1 ( yt+1 ) bel ( yt+1 ) log p ( yt+1 v |yt v) 1 A 3 5 Binary Term Unary Term
  14. rCRF: Structured Diversity w/ HMM Sampling over rCRF is equivalent

    to solving CRF Binary Term Unary Term penalize similarity to previous solutions
  15. rCRF: Algorithm in a Nutshell Compute energy function for frame-wise

    CRF Forward-Backward loop for message passing Compute the energy function of rCRF Sample by using Lagrangian relaxation [Batra et al]
  16. rCRF: Algorithm in a Nutshell All extensions of recursive belief

    estimation framework are applicable For example, •  Non-Markovian edges (eg. skip- connections, auto-regressive HMM, Input-Output HMM)
  17. Efficiency and Accuracy Improvement rCRF is 30x faster than the

    state-of-the-art algorithms and runs in real-time Accurate handling of uncertainty also increases accuracy
  18. Resulting Belief

  19. Efficiency and Accuracy Improvement Resulting belief also stays informative trough

    time
  20. Computes probabilities for past/present/future with no random sampling rCRF: Algorithm

    O ⇣ (TNOLOLA)3 ⌘ ! O ⇣ T (NOLOLA)3 ⌘ Finds all plausible solutions Extends existing learned models and inference algorithms
  21. Thank you