Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bayesian statistics Tokyo.R#94

8284465a94bbdf1ea82cf1a67d55f447?s=47 kilometer
September 11, 2021

Bayesian statistics Tokyo.R#94

第94回Tokyo.Rでトークした際のスライド資料です。

8284465a94bbdf1ea82cf1a67d55f447?s=128

kilometer

September 11, 2021
Tweet

Transcript

  1. #94 @kilometer00 2021.09.11 BeginneR Session -- Bayesian statistics --

  2. Who!? Who?

  3. Who!? ・ @kilometer ・Postdoc Researcher (Ph.D. Eng.) ・Neuroscience ・Computational Behavior

    ・Functional brain imaging ・R: ~ 10 years
  4. 宣伝!!(書籍の翻訳に参加しました。) 絶賛販売中!

  5. 宣伝!!(筆頭論⽂が出版されました!!)

  6. BeginneR Session

  7. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML

  8. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML https://tokyor.connpass.com/

  9. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML h0ps://tokyor.connpass.com/

    SXBLBMBOH TMBDLXPSLTQBDF .FNCFSਓ
  10. 3Λ࢝ΊΑ͏ 【Step】 1. Install R 2. Install RStudio  

       
  11. *OTUBMM3 ☝

  12. *OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%& ☝

  13. ☝ *OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%&

  14. )PXUPVTF34UVEJP 4DSJQUFEJUPS $POTPMF &OWJSPONFOU QMPU FUD 1 write 2 select

    3 run(⌘ + ↩) output
  15. )PXUPVTF34UVEJP

  16. )PXUPVTF34UVEJP

  17.       > x + y

    [1] 3 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP
  18.        > x +

    y [1] 4 ಉ͡ม਺໊ʹ୅ೖ͢Δͱ্ॻ͖͞ΕΔ DPNNFOUPVU 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP
  19. QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ 2021.09.04

  20.      $dyverse: データサイエンス関連パッケージ群をまとめたパッケージ ・dplyr: テーブルデータの加⼯・集計 ・ggplot2:

    グラフの描画 ・stringr: ⽂字列加⼯ ・$dyr: データの整形や変形 ・purrrr: 関数型プログラミング⽤ ・magri7r: パイプ演算⼦%>%を提供 *OTUBMMQBDLBHFGSPN$3"/ QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0⒏DJBM3QBDLBHFSFQPTJUPSZ https://cran.r-project.org/
  21. 0367*22(4*,1*/.6&41/6 ) $70-98.56.$' 20+5*59&4*,1*/. ) $70-98.56.$' 20+5*59&70-98.56.'###%# !" "UUBDIUIFQBDLBHF QBDLBHFT

    $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ *OTUBMMQBDLBHFGSPN$3"/
  22. Stan A state-of-the-art platform for statistical modeling R A free

    so4ware environment for sta7s7cal compu7ng and graphics. {rstan} package A pla:orm using stan from R
  23. None
  24. BeginneR

  25. Before After BeginneR Session BeginneR BeginneR

  26. BeginneR Advanced Hoxo_m If I have seen further it is

    by standing on the shoulders of Giants. -- Sir Isaac Newton, 1676
  27. #94 @kilometer00 BeginneR Session -- Bayesian statistics --

  28. Experiment hypothesis observation principle phenotype model data Truth Knowledge f

    X (unknown)
  29. “Hypothesis driven” “Data driven” Experimental design A B Front Back

    Right Left VerAcal Up A B
  30. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle

    phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X
  31. Strong hypothesis obs. principle phenotype f X Weak hypothesis obs.

    principle phenotype model Complex data f X model Simple data “Hypothesis driven” “Data driven” Experimental design ここが気になる(気になりだす)
  32. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with

    α faces (regular polyhedron) ! = 5 ?
  33. Dice with α faces ! = 5 $ % =

    ! α = 4 = 0 $ % = ! α = 6 = 1 6 $ % = ! α = 8 = 1 8 $ % = ! α = 12 = 1 12 $ % = ! α = 20 = 1 20 likelihood   maximum likelihood
  34. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood maximum likelihood  
  35.      Could you find α ?

    Yes, yes, yes. αis 6!! Why do you think so? Because, arg max! - . α = 6 !! Hmmm......, so......, how about ? $(α = 6) Oh, it is " #!"!! ......nnNNNNO!!! WHAT!!????
  36.       Hmmm......, so, how about

    ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood     ! α = 6 % = & !!??
  37. Probability distribution $(% = !) ! % $(% = !|α

    = 6) #(% = '|α) parameter data
  38. Probability distribution $(%) ! % arg max! -(2|α) 1 6!"

    α = 6 α = 8 α = 12 $(4) α 4 -(5 = α|2 = .) ! = # α = 20
  39. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1

    6!" $$(4) α 4 -! (5 = α|2 = .) ! = # α = 6 α = 8 α = 12 α = 20
  40. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1

    6!" $$(4) α 4 -! (5 = α|2 = .) ! = # ' 5 : α → & ' 6 : & → α α = 6 α = 8 α = 12 α = 20
  41. CondiEonal probability "($) "(&) " $ ∩ & = "(&

    ∩ $)
  42. CondiEonal probability "($) "(&) "! $ ∩ & = ""

    (& ∩ $)
  43. CondiEonal probability "($) "(&) ! 7 * ∗ ! 8

    , * = ! 7 *|, ∗ ! 8 ,
  44. Bayes’ theorem ! 7 *|, = ! 8 , *

    ∗ ! 7 (*) ! 8 , "! $ ∩ & = "" (& ∩ $) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,
  45. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α Bayes’ theorem
  46. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
  47. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
  48. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
  49. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space
  50. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space $# % = ! = $# % = !|1 = $# % = !|4 = < 4: < → α sample space
  51. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|% = 9 $# % = ! = $# % = !|4 = < = = ∀$ $# % = !|4 = α ∗ $$ 4 = α|% = 9 marginaliza7on α ∈ {4, 6, 8, 12, 20}
  52. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} $$ 4 = α = $$ α|9 $# % = ! = $# !|<
  53. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginaliza7on α ∈ {4, 6, 8, 12, 20} likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|<
  54. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|< = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} likelihood
  55. $! α|! = $" ! α ∗ $! (α) $"

    ! ' 5 : α → & ' 6 : & → α likelihood = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|-
  56. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood  
  57. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -)
  58. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20!"= 1,024,000,000,000 pa+ern)
  59. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20$%= 1,024,000,000,000 paHern)
  60. None
  61. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) + ≅ +′ approximation $! ) = ∀α + = -& = 1 5 α ∈ {4, 6, 8, 12, 20}
  62. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀!

    $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5
  63.      Hmmm......, so, how many ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≅ $# % = ! 4 = 6 1.7485C − 08 ≈ 94.58%
  64. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|!

    ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori (MAP) estimation arg max! $! α ! = 6
  65.      Hmmm......, so, how many ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob.
  66.      Hmmm......, so, how about ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob. Could you predict & II?
  67. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability
  68.      Could you predict & II?

    $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}
  69.      Could you predict & II?

    $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try !!!!!
  70.      !!! = 8 Dice with

    α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}
  71.      $ ) = 6 !

    ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try "!!!! !)) = 8 " $ = 6 {,, ,## } = 0%
  72. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5
  73. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
  74. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
  75. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|,) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
  76. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} ́ ! = {!, 8} Non-informa$ve prior distribu$on 20% 20% 20% 20% 20% 0% 94.58% 5.32% 0.09% 0.005% 0% 0% 99.98% 0.02% 0.000004% -! (α|C′) -! (α|.) -! (α| ́ .)
  77.      $ ) = 8 ́

    ! ≈ 99.98% $ !$' ≤ 8 ́ ! ≈ 99.98% Dice with α faces ́ ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4, 8} OK!! Let’s try !!"!! COME OOON
  78. No one knows what happened to them......

  79. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with

    α faces (regular polyhedron) ! = 5 ?
  80.       Hmmm......, so, how about

    ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood     ! α = 6 % = & !!??
  81. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem
  82. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀!

    $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5
  83. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|!

    ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori probability (MAP) estimation arg max! $! α ! = 6
  84. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability
  85. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}
  86. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f

    X (unknown)
  87. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle

    phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X
  88. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem
  89. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem Truth
  90. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ Kullback-Leibler divergence
  91. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy Generalization error
  92. /!" (.| # = Q[S $ − S(M)] = Q[(−log

    $ ) − (−log M )] = Q log ( ) = ∫ M % ∗ log ((#) )(,|#) Y% = ∫ M % ∗ log M(!) Y% − ∫ M % ∗ log $ % ! Y% = −Q S M − ∫ M % ∗ log $ % ! Y% B( C Entropy Generaliza$on error
  93. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribution data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy GeneralizaBon error arg min) L&'(M| $ ⟺ arg min) P P ≅ WAIC Watanabe Akaike InformaAon Criterion
  94. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f

    X (unknown)
  95. Anaïs Nin – “Life shrinks or expands in proporRon to

    one’s courage.” h0ps://images.gr-assets.com
  96. Before ABer BeginneR Session BeginneR BeginneR

  97. Enjoy!!