Bayesian statistics Tokyo.R#94

Slide 1

Slide 1 text

#94 @kilometer00 2021.09.11 BeginneR Session -- Bayesian statistics --

Slide 2

Slide 2 text

Who！？ Who?

Slide 3

Slide 3 text

Who！？・ @kilometer ・Postdoc Researcher (Ph.D. Eng.) ・Neuroscience ・Computational Behavior ・Functional brain imaging ・R： ~ 10 years

Slide 4

Slide 4 text

宣伝!!（書籍の翻訳に参加しました。）絶賛販売中！

Slide 5

Slide 5 text

宣伝!!（筆頭論⽂が出版されました！！）

Slide 6

Slide 6 text

BeginneR Session

Slide 7

Slide 7 text

-FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML

Slide 8

Slide 8 text

-FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML https://tokyor.connpass.com/

Slide 9

Slide 9 text

-FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML h0ps://tokyor.connpass.com/ SXBLBMBOH TMBDLXPSLTQBDF .FNCFSਓ

Slide 10

Slide 10 text

3Λ࢝ΊΑ͏ 【Step】 1. Install R 2. Install RStudio

Slide 11

Slide 11 text

*OTUBMM3 ☝

Slide 12

Slide 12 text

*OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%& ☝

Slide 13

Slide 13 text

☝ *OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%&

Slide 14

Slide 14 text

)PXUPVTF34UVEJP 4DSJQUFEJUPS $POTPMF &OWJSPONFOU QMPU FUD 1 write 2 select 3 run(⌘ + ↩) output

Slide 15

Slide 15 text

)PXUPVTF34UVEJP

Slide 16

Slide 16 text

)PXUPVTF34UVEJP

Slide 17

Slide 17 text

> x + y [1] 3 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP

Slide 18

Slide 18 text

> x + y [1] 4 ಉ͡ม਺໊ʹ୅ೖ͢Δͱ্ॻ͖͞ΕΔ DPNNFOUPVU 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP

Slide 19

Slide 19 text

QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ 2021.09.04

Slide 20

Slide 20 text

$dyverse: データサイエンス関連パッケージ群をまとめたパッケージ・dplyr: テーブルデータの加⼯・集計・ggplot2: グラフの描画・stringr: ⽂字列加⼯・$dyr: データの整形や変形・purrrr: 関数型プログラミング⽤・magri7r: パイプ演算⼦%>%を提供 *OTUBMMQBDLBHFGSPN$3"/ QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0⒏DJBM3QBDLBHFSFQPTJUPSZ https://cran.r-project.org/

Slide 21

Slide 21 text

0367*22(4*,1*/.6&41/6 ) $70-98.56.$' 20+5*59&4*,1*/. ) $70-98.56.$' 20+5*59&70-98.56.'###%# !" "UUBDIUIFQBDLBHF QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ *OTUBMMQBDLBHFGSPN$3"/

Slide 22

Slide 22 text

Stan A state-of-the-art platform for statistical modeling R A free so4ware environment for sta7s7cal compu7ng and graphics. {rstan} package A pla:orm using stan from R

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

BeginneR

Slide 25

Slide 25 text

Before After BeginneR Session BeginneR BeginneR

Slide 26

Slide 26 text

BeginneR Advanced Hoxo_m If I have seen further it is by standing on the shoulders of Giants. -- Sir Isaac Newton, 1676

Slide 27

Slide 27 text

#94 @kilometer00 BeginneR Session -- Bayesian statistics --

Slide 28

Slide 28 text

Experiment hypothesis observation principle phenotype model data Truth Knowledge f X (unknown)

Slide 29

Slide 29 text

“Hypothesis driven” “Data driven” Experimental design A B Front Back Right Left VerAcal Up A B

Slide 30

Slide 30 text

Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X

Slide 31

Slide 31 text

Strong hypothesis obs. principle phenotype f X Weak hypothesis obs. principle phenotype model Complex data f X model Simple data “Hypothesis driven” “Data driven” Experimental design ここが気になる（気になりだす）

Slide 32

Slide 32 text

Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with α faces (regular polyhedron) ! = 5 ?

Slide 33

Slide 33 text

Dice with α faces ! = 5 $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6 $ % = ! α = 8 = 1 8 $ % = ! α = 12 = 1 12 $ % = ! α = 20 = 1 20 likelihood maximum likelihood

Slide 34

Slide 34 text

Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood maximum likelihood

Slide 35

Slide 35 text

Could you ﬁnd α ? Yes, yes, yes. αis 6!! Why do you think so? Because, arg max! - . α = 6 !! Hmmm......, so......, how about ? $(α = 6) Oh, it is " #!"!! ......nnNNNNO!!! WHAT!!????

Slide 36

Slide 36 text

Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood ! α = 6 % = & !!??

Slide 37

Slide 37 text

Probability distribution $(% = !) ! % $(% = !|α = 6) #(% = '|α) parameter data

Slide 38

Slide 38 text

Probability distribution $(%) ! % arg max! -(2|α) 1 6!" α = 6 α = 8 α = 12 $(4) α 4 -(5 = α|2 = .) ! = # α = 20

Slide 39

Slide 39 text

Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1 6!" $$(4) α 4 -! (5 = α|2 = .) ! = # α = 6 α = 8 α = 12 α = 20

Slide 40

Slide 40 text

Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1 6!" $$(4) α 4 -! (5 = α|2 = .) ! = # ' 5 : α → & ' 6 : & → α α = 6 α = 8 α = 12 α = 20

Slide 41

Slide 41 text

CondiEonal probability "($) "(&) " $ ∩ & = "(& ∩ $)

Slide 42

Slide 42 text

CondiEonal probability "($) "(&) "! $ ∩ & = "" (& ∩ $)

Slide 43

Slide 43 text

CondiEonal probability "($) "(&) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,

Slide 44

Slide 44 text

Bayes’ theorem ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , "! $ ∩ & = "" (& ∩ $) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,

Slide 45

Slide 45 text

! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α Bayes’ theorem

Slide 46

Slide 46 text

! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

Slide 47

Slide 47 text

! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

Slide 48

Slide 48 text

! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

Slide 49

Slide 49 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space

Slide 50

Slide 50 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space $# % = ! = $# % = !|1 = $# % = !|4 = < 4: < → α sample space

Slide 51

Slide 51 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|% = 9 $# % = ! = $# % = !|4 = < = = ∀$ $# % = !|4 = α ∗ $$ 4 = α|% = 9 marginaliza7on α ∈ {4, 6, 8, 12, 20}

Slide 52

Slide 52 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} $$ 4 = α = $$ α|9 $# % = ! = $# !|<

Slide 53

Slide 53 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginaliza7on α ∈ {4, 6, 8, 12, 20} likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|<

Slide 54

Slide 54 text

$! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|< = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} likelihood

Slide 55

Slide 55 text

$! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|-

Slide 56

Slide 56 text

Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood

Slide 57

Slide 57 text

$! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -)

Slide 58

Slide 58 text

$! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20!"= 1,024,000,000,000 pa+ern)

Slide 59

Slide 59 text

$! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20$%= 1,024,000,000,000 paHern)

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

$! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) + ≅ +′ approximation $! ) = ∀α + = -& = 1 5 α ∈ {4, 6, 8, 12, 20}

Slide 62

Slide 62 text

$! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀! $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5

Slide 63

Slide 63 text

Hmmm......, so, how many ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≅ $# % = ! 4 = 6 1.7485C − 08 ≈ 94.58%

Slide 64

Slide 64 text

$$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|! ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori (MAP) estimation arg max! $! α ! = 6

Slide 65

Slide 65 text

Hmmm......, so, how many ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob.

Slide 66

Slide 66 text

Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob. Could you predict & II?

Slide 67

Slide 67 text

Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability

Slide 68

Slide 68 text

Could you predict & II? $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}

Slide 69

Slide 69 text

Could you predict & II? $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try !!!!!

Slide 70

Slide 70 text

!!! = 8 Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}

Slide 71

Slide 71 text

$ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try "!!!! !)) = 8 " $ = 6 {,, ,## } = 0%

Slide 72

Slide 72 text

"$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5

Slide 73

Slide 73 text

"$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

Slide 74

Slide 74 text

Slide 75

Slide 75 text

"$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|,) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

Slide 76

Slide 76 text

Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} ́ ! = {!, 8} Non-informa$ve prior distribu$on 20% 20% 20% 20% 20% 0% 94.58% 5.32% 0.09% 0.005% 0% 0% 99.98% 0.02% 0.000004% -! (α|C′) -! (α|.) -! (α| ́ .)

Slide 77

Slide 77 text

$ ) = 8 ́ ! ≈ 99.98% $ !$' ≤ 8 ́ ! ≈ 99.98% Dice with α faces ́ ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4, 8} OK!! Let’s try !!"!! COME OOON

Slide 78

Slide 78 text

No one knows what happened to them......

Slide 79

Slide 79 text

Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with α faces (regular polyhedron) ! = 5 ?

Slide 80

Slide 80 text

Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood ! α = 6 % = & !!??

Slide 81

Slide 81 text

! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f X (unknown)

Slide 87

Slide 87 text

Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X

Slide 88

Slide 88 text

α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem

Slide 89

Slide 89 text

α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem Truth

Slide 90

Slide 90 text

α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ Kullback-Leibler divergence

Slide 91

Slide 91 text

Slide 92

Slide 92 text

/!" (.| # = Q[S $ − S(M)] = Q[(−log $ ) − (−log M )] = Q log ( ) = ∫ M % ∗ log ((#) )(,|#) Y% = ∫ M % ∗ log M(!) Y% − ∫ M % ∗ log $ % ! Y% = −Q S M − ∫ M % ∗ log $ % ! Y% B( C Entropy Generaliza$on error

Slide 93

Slide 93 text

α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribuBon posterior distribution data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy GeneralizaBon error arg min) L&'(M| $ ⟺ arg min) P P ≅ WAIC Watanabe Akaike InformaAon Criterion

Slide 94

Slide 94 text

Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f X (unknown)

Slide 95

Slide 95 text

Anaïs Nin – “Life shrinks or expands in proporRon to one’s courage.” h0ps://images.gr-assets.com