Confronting_the_partition_function

Slide 1

Slide 1 text

/ 1

Slide 2

Slide 2 text

de • x No • k p L i a R I Si • mr • Pc i Lh Tc i • tn Pc i • v • lxN • u M s i gi • ag i gi • 8 9 8 B 8 D • D 6 CD 6 / G E 8 DB CD F8 F8B98 68 • 1C8E 8 • 6 B8 / D6 9 2 D / D6 9 • 8 C 9 6 B8 / D6 9 • C8 DB CD F8 CD D • CD D 9 D 8 1 BD D E 6D • 8 8 - A BD 68 A 9 • B 98 A 9

Slide 3

Slide 3 text

8 3 6 1 / 8 8 3 1 / 8 6 1 / 8 9 1 / 8

Slide 4

Slide 4 text

8 / 1 9 46

Slide 5

Slide 5 text

9 65 1 / 8

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

7 8 / → / 96 / 1 θ/

Slide 8

Slide 8 text

9 1 / 6 9 positive phase negative phase → 8

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

8 / 16

Slide 12

Slide 12 text

6 8 → 8 3 21 444page 6 / 8 9 6

Slide 13

Slide 13 text

1 /6 6 ↓ 9 31 /6 6 ↓ 9 8 8 → =0

Slide 14

Slide 14 text

6 1 6 → 1 9 / 4 4 6 8

Slide 15

Slide 15 text

5 1 9 8 5 9 / 5 65 9 / 5 5

Slide 16

Slide 16 text

/18 hallucinations fantasy particles 6 9 /

Slide 17

Slide 17 text

1 9 8 6 7 logZ7 logp~ 1 7 / 7 / 7 1 7 logp~ 1

Slide 18

Slide 18 text

698 / 2 . 1

Slide 19

Slide 19 text

9 6 /1 || MCMC1 8 / 6 8 /

Slide 20

Slide 20 text

9 6C CD 9 6C 1 9D 1 2 8/ 0 9D 0

Slide 21

Slide 21 text

6 D / 8 CD8 1 9 C D / 2 || 2

Slide 22

Slide 22 text

/ 8 9 6 D / C 8 1 9 12 8 9

Slide 23

Slide 23 text

M SML P PCD 9 6 8 C S D 9 312 /L

Slide 24

Slide 24 text

L D2 8 8 6 P 4 M 1 6 9 / C 6 6/ S

Slide 25

Slide 25 text

S C 5 P L M2 SML8 6 CD6 9 D / 15 6

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

61 2 / 8 7

Slide 28

Slide 28 text

1 2 1 9 /9 a,b,c 1 68 / • a 1 • b • c 1 8

Slide 29

Slide 29 text

8 2 / 19 2 1 6

Slide 30

Slide 30 text

8 pseudolikelihood 3 / 6 3 1 p~0 k^n → k*n 9 /

Slide 31

Slide 31 text

9 generalized pseudolikelihood estimator → 8 31 m 6 / m = 1 S(1) = 1,…,n → m = n S(i) = {i} →

Slide 32

Slide 32 text

1 1 8 26 / 2 6 9 3 / 1 / 1 8 6 9

Slide 33

Slide 33 text

9 / /1 p~(x)/ 86 31 p~ 3→ / 8 8

Slide 34

Slide 34 text

4 4 3 / 68 1 8 4 9 3 8 64

Slide 35

Slide 35 text

5 13/ 13 8 9 6

Slide 36

Slide 36 text

8 3 3 / SML / 8 3 1 / →SML 8 6 9 16

Slide 37

Slide 37 text

/. 1 /. 1

Slide 38

Slide 38 text

6 3 98 Z/ 1 1 1

Slide 39

Slide 39 text

1/ 36 Z → xZ = 0 8 pdata 9

Slide 40

Slide 40 text

Slide 41

Slide 41 text

4 • x4 • logp~(x)1 4 46 • 46 • logp~(x)4 4 / • 8 49 • 4 4 4

Slide 42

Slide 42 text

• 9 4 / • 4 4 2 8 • 21 9 2 9 8 9 /6

Slide 43

Slide 43 text

31 . /

Slide 44

Slide 44 text

? pdata ? q(x|y)8 8y6 x 1 4 → 68 pdata48 4 / 4 9

Slide 45

Slide 45 text

8 • 5/ p model1 6 9 / 4 • / 4 1 6 4 6 • Kingma 1 6 4 8 6

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

a c 7 1 - ,1 4 4 - - 4 / 4 1 8 e c C E e Z c i • 9 –logZ(θ) • ΘE9 c c NC6

Slide 48

Slide 48 text

4 noise distribution p noise(x) 8 E/ 1 p noise(x) 6 96 C

Slide 49

Slide 49 text

69 4 1 8 1 /

Slide 50

Slide 50 text

59 p joint 0 p train 8 / 6 1 /

Slide 51

Slide 51 text

p joint E 8 9 C / 615 8 9

Slide 52

Slide 52 text

98 • logp~ model Back prop • p noise 25 51 6 →NCE /

Slide 53

Slide 53 text

e sl v 1 / 1 3 1 - / 5 • • r a →o c v • ta • ta 6 • 8m f e n i 9 • c sl v • f v c 5 • 6

Slide 54

Slide 54 text

6 4 • • 5 G 6 8 / • GAN • 5 G A / 1 9 8 G/

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

• 9 • Z 8 6 / • • / ) 6 5( 1

Slide 57

Slide 57 text

• • 5 8 • /1 657 9 657

Slide 58

Slide 58 text

A • 9 2 • 9 • 9 • • 6 8 5 • 6 8 5/ 1 2 B

Slide 59

Slide 59 text

6 || 6 89 / 1 5

Slide 60

Slide 60 text

9 6 8 9 9 Z(θ A ) / 01

Slide 61

Slide 61 text

/ → 9 691 9 8 / /

Slide 62

Slide 62 text

Slide 63

Slide 63 text

6 6 6 p0 p1 8 9 3 • p1 / 1 • p1 / 1 p0 / 6 • p0 p1 p0 6 p1 1 Z16 1

Slide 64

Slide 64 text

1 1 141 / / 8 9 / → /6 1 /

Slide 65

Slide 65 text

p0 p1 6 9 8 p0 p1 / 58 bridge the gap intermediate distributions 8 1. 1 annealed importance sampling, AIS 2. bridge sampling

Slide 66

Slide 66 text

DKL(p0||p1) I p0 p1 9 A6 8 S 1 / 1

Slide 67

Slide 67 text

69 7 1. Z0 Z1/ 2. Z0 I 1 8 3. A 2 8

Slide 68

Slide 68 text

8 A 6 8 0 <= j <= n-1 / pnj pnj+1 S → 6 1 8 Znj+1 / Znj →1 8 Z1 / Z0 8 9 → 9 p1 8 p0 8I

Slide 69

Slide 69 text

8 6 I / 8 9 A Tnj(x’|x) 1 Pnj(x) 1

Slide 70

Slide 70 text

0 / 7 98 6 01

Slide 71

Slide 71 text

6 A9 9 8 I / I / 1 7 1 log w(k) 1

Slide 72

Slide 72 text

6 7 91 A A S 9 7 A 8 / 1 I 9 A 2

Slide 73

Slide 73 text

3 1 / AIS motivation 8 9 7 61 p* p0 p1 9

Slide 74

Slide 74 text

4 1 / d 7 6 7 8 a d be g → be 9 be 6

Slide 75

Slide 75 text

/ 1 5 9 7 AIS / 1 6 8 6

Slide 76

Slide 76 text

89 AIS 1 / / 796 89 /

Slide 77

Slide 77 text

1 / 1 Desjardins/ 987 6 AIS 6 RBM