$30 off During Our Annual Pro Sale. View Details »

Towards Realistic Predictors - EN

Leszek Rybicki
November 29, 2018

Towards Realistic Predictors - EN

Later version of the "Towards Realistic Predictors" paper review, in English.

Leszek Rybicki

November 29, 2018
Tweet

More Decks by Leszek Rybicki

Other Decks in Research

Transcript

  1. Towards Realistic Predictors
    Pei Wang and Nuno Vasconcelos
    Statistical and Visual Computing Lab, UC San Diego
    mlKitchen X
    2018.11.29
    @_lunardog_
    http://openaccess.thecvf.com/content_ECCV_2018/papers/Pei_Wang_Towards_Realistic_Predictors_ECCV_2018_paper.pdf

    View Slide

  2. Self-Introduction
    ● Call me Leszek
    ● Born in Poland
    ● Living in Japan since 2010
    ● Cookpad R&D since 2016
    ● I consume too much science fiction
    ● I’m bad at selfies

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. Towards Realistic Predictors
    Pei Wang and Nuno Vasconcelos
    Statistical and Visual Computing Lab, UC San Diego
    http://openaccess.thecvf.com/content_ECCV_2018/papers/Pei_Wang_Towards_Realistic_Predictors_ECCV_2018_paper.pdf

    View Slide

  8. Define “Realistic”
    optimistic
    pessimistic realistic

    View Slide

  9. https://snappygoat.com/

    View Slide

  10. https://snappygoat.com/

    View Slide

  11. https://www.abc15.com

    View Slide

  12. A more benign example...

    View Slide

  13. Classifier
    Food / non-food classifier
    food
    plant
    person
    pet

    other

    View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. Let’s start with what we can easily classify

    View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. What is “hard” anyway?

    View Slide

  25. HP-Net = Hardness Predictor
    HP-Net hardness

    View Slide

  26. Classifier
    HP-Net
    Adversarial training with a hardness predictor

    View Slide

  27. Hardness Predictor Loss
    bi y s e t y s u w r e s ma c
    mi zi t K l a k-Le b di g e b en t
    di r i n d a m i m n = 1 − p
    c

    View Slide

  28. Classifier Loss weighted by hardness
    we t ro -en p
    ma h er p e (la r ) mo po n , w i as e m s
    (lo s) ar en s or c

    View Slide

  29. Classifier
    HP-Net
    Training
    1. train classifier F and
    HP-Net S jointly on training
    set D
    2. run S on D and eliminate
    hard examples, to create
    realistic training set D′
    3. learn realistic classifier F′
    on D′, with S fixed
    4. output pair S, F′
    5. GOTO 1
    D
    F
    S

    View Slide

  30. Can’t we just use confidence scores?

    View Slide

  31. View Slide

  32. Hardness
    progression
    during training

    View Slide

  33. Do we need two separate models?

    View Slide

  34. Classifier
    +
    HP-Net
    +

    View Slide

  35. View Slide

  36. Do we need to fine tune?

    View Slide

  37. C - normal classifier
    F - realistic predictor without fine-tuning (just rejection)
    F’ - realistic predictor, fine-tuned on samples accepted by HP-Net

    View Slide

  38. View Slide

  39. Conclusions
    ● There are times when it’s OK to skip hard samples
    ● ...and times when it’s BEST to reject hard samples
    ● The paper introduces a GAN-like architecture to train
    any classifier with its own hardness predictor
    ● Training with a hardness predictor improves accuracy
    ● HP-Net should be trained jointly with the classifier, but in an alternating order
    ● HP-Net solves a different problem from the Classifier, should be a separate model
    ● ….but best results are when the architectures are the same

    View Slide

  40. https://ja.wikipedia.org/wiki/2001年宇宙の旅
       うちゅうのたび
    2001年宇宙の旅
    『2001年宇宙の旅』(にせんいちねんう
    ちゅうのたび、原題:2001: A Space
    Odyssey)は、アーサー・C・クラークとスタン
    リー・キューブリックのアイデアをまとめた
    ストーリーに基いて製作された、SF映画お
    よびSF小説である。
    2001: A Space Odyssey is a 1968 epic science
    fiction film produced and directed by Stanley
    Kubrick. The screenplay was written by Kubrick
    and Arthur C. Clarke, and was inspired by
    Clarke's short story "The Sentinel". A novel also
    called 2001: A Space Odyssey, written
    concurrently with the screenplay, was
    published soon after the film was released.
    2001: A Space Odyssey
    https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)

    View Slide

  41. It’s the movie with the mysterious black block, and classical music in space.

    View Slide

  42. Spaceships don’t make a “Whoosh!” sound, there’s classical music instead.

    View Slide

  43. Fra Dav
    HA 9000 is r pe
    in f e n a om
    d i n p e h
    Frank and Dave don’t trust HAL.
    HAL is a realistic predictor and doesn’t always follow orders.

    View Slide

  44. Open the pod bay doors, HAL!

    View Slide

  45. I’m sorry, Dave.
    I’m afraid I can’t do that.
    Fra
    HAL kills Frank.

    View Slide

  46. Dave has no choice but to deactivate HAL.
    Daisy… Daisy…

    View Slide

  47. Dedicated to Douglas Rain
    (March 13, 1928 – November 11, 2018)
    known as the Voice of HAL

    View Slide