AAAI-2016 Invited Talk

1507f22ca8d84c83e52362f69e428698?s=47 Ozan Sener
February 15, 2016

AAAI-2016 Invited Talk

Extended presentation of my RSS paper rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

1507f22ca8d84c83e52362f69e428698?s=128

Ozan Sener

February 15, 2016
Tweet

Transcript

  1. 4.
  2. 5.

    How to Find MAP Compute features Define the energy function

    Solve the Combinatorial Optimization Activity/Object labels for the past and future Sample Possible Futures [Koppula et al RSS/ICML, Kitani et al ECCV, Jiang et al. RSS/CVPR, Lan et al ECCV, Hu et al RSS/ICRA ]
  3. 6.
  4. 7.

    Shortcomings Compute features Define the energy function Solve the Combinatorial

    Optimization Activity/Object labels for the past and future Sample Possible Futures We also need probabilities in addition to MAP solution (future is unknown)
  5. 8.

    Shortcomings Compute features Define the energy function Solve the Combinatorial

    Optimization Activity/Object labels for the past and future Sample Possible Futures We also need probabilities in addition to MAP solution (future is unknown) on Space with dimension~ 1o6xT ~ 103600 (#ObjLabels#Objectsx#ActLabels)Time
  6. 9.

    Tale of Two Approaches Graphical Models (discriminative) Can handle exponentially

    large space Recursive Bayesian (generative) Can compute full belief and reason about uncertinity
  7. 11.

    Structured Diversity Modes are likely and structurally diverse yt,i =

    arg max y belt ( y ) s.t. ( y, yt,i ) 8j < i
  8. 12.

    HMM – Recursive Belief Estimation HMM Derivation [Rabiner] belt( y

    ) / p( y t = y | x 1, . . . , x t) | {z } ↵t(y) p( x t+1, . . . , x T | y t = y ) | {z } t(y) ↵t( y t) = p( x t| y t) X yt 1 ↵t 1( y t 1)p( y t| y t 1) t( y t) = X yt+1 p( x t+1| y t+1) t+1( y t+1)p( y t+1| y t)
  9. 13.

    rCRF: Structured Diversity meets HMM A Belief over rCRF is

    a CRF bel ( yt ) / exp 2 4 X v,w2Et ⇣ Eb ( v, w) ˜ Eb ( v, w) ⌘ X v2Vt 0 @Eu ( v) ˜ Eu ( v) + X yt 1 ↵t 1 ( yt 1 ) log p ( yt v |yt 1 v ) 1 X yt+1 t+1 ( yt+1 ) bel ( yt+1 ) log p ( yt+1 v |yt v) 1 A 3 5 Binary Term Unary Term
  10. 14.

    rCRF: Structured Diversity w/ HMM Sampling over rCRF is equivalent

    to solving CRF Binary Term Unary Term penalize similarity to previous solutions
  11. 15.

    rCRF: Algorithm in a Nutshell Compute energy function for frame-wise

    CRF Forward-Backward loop for message passing Compute the energy function of rCRF Sample by using Lagrangian relaxation [Batra et al]
  12. 16.

    rCRF: Algorithm in a Nutshell All extensions of recursive belief

    estimation framework are applicable For example, •  Non-Markovian edges (eg. skip- connections, auto-regressive HMM, Input-Output HMM)
  13. 17.

    Efficiency and Accuracy Improvement rCRF is 30x faster than the

    state-of-the-art algorithms and runs in real-time Accurate handling of uncertainty also increases accuracy
  14. 20.

    Computes probabilities for past/present/future with no random sampling rCRF: Algorithm

    O ⇣ (TNOLOLA)3 ⌘ ! O ⇣ T (NOLOLA)3 ⌘ Finds all plausible solutions Extends existing learned models and inference algorithms
  15. 21.