Bayesian statistics Tokyo.R#94

第94回Tokyo.Rでトークした際のスライド資料です。

Transcript

1. #94 @kilometer00 2021.09.11 BeginneR Session -- Bayesian statistics --
2. Who！？ Who?
3. Who！？ ・ @kilometer ・Postdoc Researcher (Ph.D. Eng.) ・Neuroscience ・Computational Behavior ・Functional brain imaging ・R： ~ 10 years
4. 宣伝!!（書籍の翻訳に参加しました。） 絶賛販売中！
5. 宣伝!!（筆頭論⽂が出版されました！！）
6. BeginneR Session
7. -FU`T
   TUBSU
   3 ɾ'SFF ɾ -PX
   JOTUBMMBUJPO
   DPTU
   GPS
   CBTJD
   FOWJSPONFOU ɾ'VMM
   SBOHF
   PG
   GVODUJPOT
   GPS
   EBUB
   TDJFODF ɾ.BOZ
   FYUFOTJPOT
   QBDLBHFT ɾ4USPOH
   DPNNVOJUZ
   ˡ QPTJUJPO
   UBML
   
8. -FU`T
   TUBSU
   3 ɾ'SFF ɾ -PX
   JOTUBMMBUJPO
   DPTU
   GPS
   CBTJD
   FOWJSPONFOU ɾ'VMM
   SBOHF
   PG
   GVODUJPOT
   GPS
   EBUB
   TDJFODF ɾ.BOZ
   FYUFOTJPOT
   QBDLBHFT ɾ4USPOH
   DPNNVOJUZ
   ˡ QPTJUJPO
   UBML
    https://tokyor.connpass.com/
9. -FU`T
   TUBSU
   3 ɾ'SFF ɾ -PX
   JOTUBMMBUJPO
   DPTU
   GPS
   CBTJD
   FOWJSPONFOU ɾ'VMM
   SBOHF
   PG
   GVODUJPOT
   GPS
   EBUB
   TDJFODF ɾ.BOZ
   FYUFOTJPOT
   QBDLBHFT ɾ4USPOH
   DPNNVOJUZ
   ˡ QPTJUJPO
   UBML
    h0ps://tokyor.connpass.com/ SXBLBMBOH TMBDL
   XPSLTQBDF .FNCFS
   ਓ
10. 3Λ࢝ΊΑ͏ 【Step】 1. Install R 2. Install RStudio   
      
    
11. 
    *OTUBMM
    3 ☝
12. 
    *OTUBMM
    34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFE
    EFWFMPQNFOU
    FOWJSPONFOU
    *%& ☝
13. ☝ 
    *OTUBMM
    34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFE
    EFWFMPQNFOU
    FOWJSPONFOU
    *%&
14. )PX
    UP
    VTF
    34UVEJP 4DSJQU
    FEJUPS $POTPMF &OWJSPONFOU QMPU
    FUD 1 write 2 select 3 run(⌘ + ↩) output
15. )PX
    UP
    VTF
    34UVEJP
16. )PX
    UP
    VTF
    34UVEJP
17. 
      
     
    
     > x + y [1] 3 4DSJQU
    FEJUPS $POTPMF
    PVUQVU )PX
    UP
    VTF
    34UVEJP
18. 
      
       > x + y [1] 4 
    ಉ͡ม਺໊ʹ୅ೖ͢Δͱ্ॻ͖͞ΕΔ DPNNFOU
    PVU 4DSJQU
    FEJUPS $POTPMF
    PVUQVU )PX
    UP
    VTF
    34UVEJP
19. QBDLBHFT $3"/
    5IF
    $PNQSFIFOTJWF
    3
    "SDIJWF
    /FUXPSL 0GGJDJBM
    3
    QBDLBHF
    SFQPTJUPSZ h0ps://cran.r-project.org/ 2021.09.04
20.    
    
    
     $dyverse: データサイエンス関連パッケージ群をまとめたパッケージ ・dplyr: テーブルデータの加⼯・集計 ・ggplot2: グラフの描画 ・stringr: ⽂字列加⼯ ・$dyr: データの整形や変形 ・purrrr: 関数型プログラミング⽤ ・magri7r: パイプ演算⼦%>%を提供 *OTUBMM
    QBDLBHF
    GSPN
    $3"/ QBDLBHFT $3"/
    5IF
    $PNQSFIFOTJWF
    3
    "SDIJWF
    /FUXPSL 0⒏DJBM
    3
    QBDLBHF
    SFQPTJUPSZ https://cran.r-project.org/
21. 0367*22(4*,1*/.6&41/6 ) $70-98.56.$' 20+5*59&4*,1*/. ) $70-98.56.$' 20+5*59&70-98.56.'###%# !" "UUBDI
    UIF
    QBDLBHF QBDLBHFT $3"/
    5IF
    $PNQSFIFOTJWF
    3
    "SDIJWF
    /FUXPSL 0GGJDJBM
    3
    QBDLBHF
    SFQPTJUPSZ h0ps://cran.r-project.org/ *OTUBMM
    QBDLBHF
    GSPN
    $3"/
22. Stan A state-of-the-art platform for statistical modeling R A free so4ware environment for sta7s7cal compu7ng and graphics. {rstan} package A pla:orm using stan from R
24. BeginneR
25. Before After BeginneR Session BeginneR BeginneR
26. BeginneR Advanced Hoxo_m If I have seen further it is by standing on the shoulders of Giants. -- Sir Isaac Newton, 1676
27. #94 @kilometer00 BeginneR Session -- Bayesian statistics --
28. Experiment hypothesis observation principle phenotype model data Truth Knowledge f X (unknown)
29. “Hypothesis driven” “Data driven” Experimental design A B Front Back Right Left VerAcal Up A B
30. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X
31. Strong hypothesis obs. principle phenotype f X Weak hypothesis obs. principle phenotype model Complex data f X model Simple data “Hypothesis driven” “Data driven” Experimental design ここが気になる（気になりだす）
32. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with α faces (regular polyhedron) ! = 5 ?
33. Dice with α faces ! = 5 $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6 $ % = ! α = 8 = 1 8 $ % = ! α = 12 = 1 12 $ % = ! α = 20 = 1 20 likelihood   maximum likelihood
34. Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood maximum likelihood  
35.      Could you find α ? Yes, yes, yes. αis 6!! Why do you think so? Because, arg max! - . α = 6 !! Hmmm......, so......, how about ? $(α = 6) Oh, it is " #!"!! ......nnNNNNO!!! WHAT!!????
36.  
         Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood  
       ! α = 6 % = & !!??
37. Probability distribution $(% = !) ! % $(% = !|α = 6) #(% = '|α) parameter data
38. Probability distribution $(%) ! % arg max! -(2|α) 1 6!" α = 6 α = 8 α = 12 $(4) α 4 -(5 = α|2 = .) ! = # α = 20
39. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1 6!" $$(4) α 4 -! (5 = α|2 = .) ! = # α = 6 α = 8 α = 12 α = 20
40. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1 6!" $$(4) α 4 -! (5 = α|2 = .) ! = # ' 5 : α → & ' 6 : & → α α = 6 α = 8 α = 12 α = 20
41. CondiEonal probability "($) "(&) " $ ∩ & = "(& ∩ $)
42. CondiEonal probability "($) "(&) "! $ ∩ & = "" (& ∩ $)
43. CondiEonal probability "($) "(&) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,
44. Bayes’ theorem ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , "! $ ∩ & = "" (& ∩ $) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,
45. ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α Bayes’ theorem
46. ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
47. ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
48. ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
49. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space
50. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space $# % = ! = $# % = !|1 = $# % = !|4 = < 4: < → α sample space
51. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|% = 9 $# % = ! = $# % = !|4 = < = = ∀$ $# % = !|4 = α ∗ $$ 4 = α|% = 9 marginaliza7on α ∈ {4, 6, 8, 12, 20}
52. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} $$ 4 = α = $$ α|9 $# % = ! = $# !|<
53. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginaliza7on α ∈ {4, 6, 8, 12, 20} likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|<
54. $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|< = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} likelihood
55. $! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|-
56. Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood  
57. $! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -)
58. $! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20!"= 1,024,000,000,000 pa+ern)
59. $! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20$%= 1,024,000,000,000 paHern)
61. $! α|! = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) + ≅ +′ approximation $! ) = ∀α + = -& = 1 5 α ∈ {4, 6, 8, 12, 20}
62. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀! $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5
63.      Hmmm......, so, how many ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≅ $# % = ! 4 = 6 1.7485C − 08 ≈ 94.58%
64. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|! ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori (MAP) estimation arg max! $! α ! = 6
65.      Hmmm......, so, how many ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob.
66.      Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob. Could you predict & II?
67. Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability
68.      Could you predict & II? $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}
69.      Could you predict & II? $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try !!!!!
70.      !!! = 8 Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}
71.      $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try "!!!! !)) = 8 " $ = 6 {,, ,## } = 0%
72. "$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5
73. "$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
74. "$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
75. "$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|,) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
76. Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} ́ ! = {!, 8} Non-informa$ve prior distribu$on 20% 20% 20% 20% 20% 0% 94.58% 5.32% 0.09% 0.005% 0% 0% 99.98% 0.02% 0.000004% -! (α|C′) -! (α|.) -! (α| ́ .)
77.      $ ) = 8 ́ ! ≈ 99.98% $ !$' ≤ 8 ́ ! ≈ 99.98% Dice with α faces ́ ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4, 8} OK!! Let’s try !!"!! COME OOON
78. No one knows what happened to them......
79. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with α faces (regular polyhedron) ! = 5 ?
80.  
         Hmmm......, so, how about ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood  
       ! α = 6 % = & !!??
81. ! 7 *|, = ! 8 , * ∗ ! 7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
82. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀! $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5
83. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|! ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori probability (MAP) estimation arg max! $! α ! = 6
84. Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability
85. "$ α|, ≅ "% , α ∗ "$ (α|4′) "% (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
86. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f X (unknown)
87. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X
88. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem
89. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem Truth
90. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ Kullback-Leibler divergence
91. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy Generalization error
92. /!" (.| # = Q[S $ − S(M)] = Q[(−log $ ) − (−log M )] = Q log ( ) = ∫ M % ∗ log ((#) )(,|#) Y% = ∫ M % ∗ log M(!) Y% − ∫ M % ∗ log $ % ! Y% = −Q S M − ∫ M % ∗ log $ % ! Y% B( C Entropy Generaliza$on error
93. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α . prior distribuBon posterior distribution data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy GeneralizaBon error arg min) L&'(M| $ ⟺ arg min) P P ≅ WAIC Watanabe Akaike InformaAon Criterion
94. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f X (unknown)
95. Anaïs Nin – “Life shrinks or expands in proporRon to one’s courage.” h0ps://images.gr-assets.com
96. Before ABer BeginneR Session BeginneR BeginneR
97. Enjoy!!