Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Confronting_the_partition_function

tanimutomo
May 09, 2018
52

 Confronting_the_partition_function

分配関数との対峙
Chapter 18 in Deep learning Book
深層学習本(Deep learning book by Ian Goodfellow)
URL:http://www.deeplearningbook.org/

Tag: deep learning, machine learning, mathematics, nce, partition function, pcd, sml

tanimutomo

May 09, 2018
Tweet

More Decks by tanimutomo

Transcript

  1. / 1

  2. de • x No • k p L i a

    R I Si • mr • Pc i Lh Tc i • tn Pc i • v • lxN • u M s i gi • ag i gi • 8 9 8 B 8 D • D 6 CD 6 / G E 8 DB CD F8 F8B98 68 • 1C8E 8 • 6 B8 / D6 9 2 D / D6 9 • 8 C 9 6 B8 / D6 9 • C8 DB CD F8 CD D • CD D 9 D 8 1 BD D E 6D • 8 8 - A BD 68 A 9 • B 98 A 9
  3. 8 3 6 1 / 8 8 3 1 /

    8 6 1 / 8 9 1 / 8
  4. 1 9 8 6 7 logZ7 logp~ 1 7 /

    7 / 7 1 7 logp~ 1
  5. L D2 8 8 6 P 4 M 1 6

    9 / C 6 6/ S
  6. 1 2 1 9 /9 a,b,c 1 68 / •

    a 1 • b • c 1 8
  7. 9    generalized pseudolikelihood estimator → 8 31

    m 6 / m = 1 S(1) = 1,…,n → m = n S(i) = {i} →
  8. 1 1 8 26 / 2 6 9 3 /

    1 / 1 8 6 9
  9. 8 3 3 / SML / 8 3 1 /

    →SML 8 6 9 16
  10. 4 •     x4 • logp~(x)1 4

    46 • 46 • logp~(x)4 4 / • 8 49 • 4 4 4
  11. • 9 4 / • 4 4 2 8 •

    21 9 2 9 8 9 /6
  12. 8 • 5/ p model1 6 9 / 4 •

    / 4 1 6 4 6 • Kingma 1 6 4 8 6
  13. a c 7 1 - ,1 4 4 - -

    4 / 4 1 8 e c C E e Z c i • 9 –logZ(θ) • ΘE9 c  c NC6
  14. e sl v 1 / 1 3 1 - /

    5 • • r a →o c v • ta • ta 6 • 8m f e n i 9 • c sl v • f v c 5 • 6
  15. 6 4 • • 5 G 6 8 / •

    GAN • 5 G A / 1 9 8 G/
  16. A • 9 2 • 9 • 9 • •

    6 8 5 • 6 8 5/ 1 2 B
  17. 6 6 6 p0 p1 8 9 3 • p1

    / 1 • p1 / 1 p0 / 6 • p0 p1 p0 6 p1 1 Z16 1
  18. p0 p1 6 9 8 p0 p1 / 58 bridge

    the gap intermediate distributions 8 1. 1 annealed importance sampling, AIS 2. bridge sampling
  19. 8 A 6 8 0 <= j <= n-1 /

    pnj pnj+1 S → 6 1 8 Znj+1 / Znj →1 8 Z1 / Z0 8 9 → 9 p1 8 p0 8I
  20. 6 A9 9 8 I / I / 1 7

    1 log w(k) 1
  21. 6 7 91 A A S 9 7 A 8

    / 1 I 9 A 2
  22. 4 1 / d  7 6 7 8 a

    d be g → be 9 be 6