前処理勉強会_発表資料_MITTI_20210724.pdf

by mitti1210

Slide 1

Slide 1 text

લॲཧΛRͰ΍ͬͯΈͨʂ   DPCσʔλʹ௅ઓʂ 1 2021/07/24 MITTI

Slide 2

Slide 2 text

ϑΥϧμͷ४උ .*55*ͷൃදͰ͸ҎԼͷߏ଄ͰϑΥϧμΛ࡞͍ͬͯ·͢ 3Ͱ΋Ͱ͖·͕࣌ؒ͢ͷ౎߹্ল͘ͷͰࣄલʹ࡞੒͢ΔͱਐΊ΍͍͢Ͱ͢ 2 ϓϩδΣΫτ ϑΥϧμ JOQVU TVNNBSZ .%$@@ PVUQVU JOQVU .%$@@

Slide 3

Slide 3 text

μ΢ϯϩʔυ 3 ͜ͷ4೥෼ ࢪઃ֓ཁද ʢ̔ʣ࣬ױผखज़ผूܭ MDC07 ˎ2016,2017೥͸2ͭʹ෼͔Ε͍ͯΔ MDC07ɿ੔ܗ֎Պ

Slide 4

Slide 4 text

μ΢ϯϩʔυͱอଘϑΝΠϧ໊ ͜ͷϑΥϧμ໊ͱϑΝΠϧ໊ͰਐΊ·͢ 4 ϓϩδΣΫτ ϑΥϧμ JOQVU TVNNBSZ .%$@@ PVUQVU TVNNBSZ .%$@@ ʢ̔ʣ࣬ױผखज़ผूܭ MDC07 ࢪઃ֓ཁද

Slide 5

Slide 5 text

ࣄલ४උ 5 #ࠓճ͸tidyverseύοέʔδΛ࢖͍·͢ɻ #Πϯετʔϧͨ͜͠ͱͳ͍ํ͸ઌʹΠϯετʔϧ install.packages(“tidyverse") #ͲͪΒͷൃදͰ΋࢖͍·͢ #Πϯετʔϧͨ͠ύοέʔδͷݺͼग़͠ library(tidyverse) #ࠓճ͸ಛʹdplyr,tidyr,stringr,purrrΛ࢖༻ library(readxl) #excelͷಡΈࠐΈʹ࢖༻

Slide 6

Slide 6 text

ୀӃױऀௐࠪ 6 https://www.mhlw.go.jp/stf/shingi2/0000196043_00003.html

Slide 7

Slide 7 text

ख࡞ۀͰͰ͖Δʁ 7 ෳ਺ྻ໊ ࡏӃ೔਺ ͍Βͳ͍ ϋΠϑϯ ͍Βͳ͍ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ʢ༌݂Ҏ֎ͷ࠶ܝʣ Λআ͍ͨ਺͕߹ܭ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸

Slide 8

Slide 8 text

͜ΕͳΒ෼ੳʹ͔͚ͨΓάϥϑ͕࡞ΕΔ 8 ೥౓ͰมΘΔ൪߸Λ ἧ͑ͨ ࢢொଜ໊෇͚ͨ ෳ਺೥Λͭͳ͛ͨ ප໊Ͱ΋ݕࡧͰ͖Δ ਓ਺͚ͩʹͨ͠

Slide 9

Slide 9 text

ࠓճͷલॲཧ ˔UJEZʢ੔વσʔλʣʹ͢Δ ˔จࣈྻॲཧ ˔܁Γฦ͠ॲཧʹରԠ͢Δ ˔ؔ਺Ͱ·ͱΊΔ 9

Slide 10

Slide 10 text

UJEZʢ੔વσʔλʣͱ͸ʁ 8JLJQFEJB༷ΑΓ IUUQTKBXJLJQFEJBPSHXJLJ5JEZ@EBUB 10

Slide 11

Slide 11 text

σʔλͷ੔ܗɿUJEZS 11 ࡶવσʔλ %>% pivot_longer ( col = c("6࣌", "12࣌", “18࣌”) , names_to = “࣌ࠁ” , values_to = “ఱؾ”)

Slide 12

Slide 12 text

UJEZʹ͢ΔͨΊͷઓ͍ 12 5JEZ /PU5JEZ

Slide 13

Slide 13 text

ࠓճͷલॲཧ ˔UJEZʢ੔વσʔλʣʹ͢ΔɿUJEZS ˔จࣈྻॲཧɿTUSJOHS ˔܁Γฦ͠ॲཧʹରԠ͢ΔɿQVSSS ˔ؔ਺Ͱ·ͱΊΔɿGVODUJPO ˔͚ͬͭ͘ΔɿKPJO 13

Slide 14

Slide 14 text

จࣈྻॲཧɿTUSJOHS 14 TUSJOHS ୅දతͳؔ਺ આ໌ ݕࡧ TUS@EFUFDU ͋Δจࣈྻؚ͕·Ε͍ͯͨΒ536& நग़ TUS@FYUSBDU TUS@FYUSBDU@BMM TUS@TVC ͋ΔύλʔϯʹҰக͢ΔจࣈྻΛநग़ ࠷ॳʢ࠷ޙʣ˓จࣈΛநग़ ஔ͖׵͑ TUS@SFQMBDF TUS@SFQMBDF@BMM ύλʔϯʹϚονͨ͠ՕॴΛஔ׵ Ճ޻ TUS@D จࣈྻΛ͚ͬͭ͘Δ https://kazutan.github.io/kazutanR/stringr-intro.html ͕ৄ͍͠Ͱ͢ʂ

Slide 15

Slide 15 text

TUSJOHSͱਖ਼نදݱ 15 ྫʣ“MDC_02_8_07_2019.xlsx"ͱ͍͏ϑΝΠϧ໊͔Β৭ʑऔΓग़͍ͨ͠ str_extract(“จࣈྻ”, “৚݅” ) ৚݅ʹ͸৭ʑͳࢦఆ͕Ͱ͖Δ [ ] ͜ͷதʹ͋Δ਺஋ɾจࣈɾه߸ \\d ਺஋ \\D จࣈ . ͳΜͰ΋O K * 0ճҎ্ͷ܁Γฦ͠ { } ׅހ಺ͷ਺͚ͩ܁Γฦ͠ ^ จࣈྻͷઌ಄ $ จࣈྻͷ࠷ޙ fi MFl.%$@@@@YMTY TUS@FYUSBDU fi MF z\^z TUS@FYUSBDU fi MF z.%$\^z TUS@FYUSBDU fi MF l<>Yz ਺஋4ܻ MDCɹ೚ҙͷ܁Γฦ͠ɹ਺஋4ܻ .(ه߸ͷυοτ)ɹ xɹɹɹ೚ҙͷ܁Γฦ͠

Slide 16

Slide 16 text

˔UJEZʢ੔વσʔλʣʹ͢Δ ˔จࣈྻॲཧ ˔܁Γฦ͠ॲཧʹରԠ͢Δ ˔ؔ਺Ͱ·ͱΊΔ 16

Slide 17

Slide 17 text

17 ྫʣ&YDFMͷෳ਺λϒΛҰؾʹಡΈࠐΉ ܁Γฦ͠QVSSS

Slide 18

Slide 18 text

͚ͬͭ͘ΔɿEQMZSKPJO 18 https://github.com/rstudio/cheatsheets/raw/master/data-transformation.pdf

Slide 19

Slide 19 text

˔UJEZʢ੔વσʔλʣʹ͢Δ ˔܁Γฦ͠ॲཧʹରԠ͢Δ ˔จࣈྻॲཧ ˔ؔ਺Ͱ·ͱΊΔ 19

Slide 20

Slide 20 text

ؔ਺Ͱ·ͱΊΔϝϦοτͱϙΠϯτ ˔ෆཁͳม਺͕ݮΔ ˔ଞͷϑΝΠϧͰ΋ར༻Ͱ͖ΔɻNBQ͕࢖͍΍͘͢ͳΔɻ 20 ؔ਺಺Ͱ࢖ΘΕΔม਺͸ ࢒Βͳ͍ ϑΥϧμ಺ʹ͋ΔϑΝΠϧΛ ·ͱΊͯಡΈࠐΊΔ

Slide 21

Slide 21 text

ؔ਺Ͱ·ͱΊΔϝϦοτͱϙΠϯτ ˔͋Δ೥্͕ख͍͔ͬͨ͘Βͱ͍ͬͯҧ͏೥Ͱ΋্ख͘ߦ͔͘͸֬ೝඞཁ ɹɾYMT͕YMTYʹมΘͬͯ΋ରԠͰ͖Δίʔυ   ɹɾΤϥʔ͕ग़ͨΒͦͷ౎౓௚ؔ͠਺ʹܧ͗଍͍ͯ͘͠ 21

Slide 22

Slide 22 text

۩ମྫ 22

Slide 23

Slide 23 text

ࠂࣔ൪߸Λଗ͑ΔͨΊͷલॲཧ 23

Slide 24

Slide 24 text

σʔλΛ֬ೝ ɾཉ͍͠σʔλ͸͚ͩ͜͜   ɾ˞ͷϚʔΫ͍Βͳ͍   ɾ೥౓ͷྻΛ଍͍ͨ͠   ɾվߦ͍Βͳ͍ 24 ▶︎ ͜Ε͚ͩͳΒ௚઀ίʔυ͔͚ͦ͏

Slide 25

Slide 25 text

೥౓Ͱྻ໊͕มΘ͍ͬͯͳ͍͔νΣοΫ 25 1ʙ3ྻ໨ͱ”ࢪઃ໊”ͷྻ͚ͩཉ͍͠ʂ select(1:3, “ࢪઃ໊”)

Slide 26

Slide 26 text

SFBE@FYDFMͰಡΈࠐΉ 26 read_excel ( path=ϑΝΠϧͷύε , sheet=γʔτͷ໊લ͔˓൪໨ͷγʔτ, #ࢦఆ͠ͳ͚Ε͹1ຕ໨ͷγʔτ skip=ઌ಄ʹ͍Βͳ͍ߦ͕͋Ε͹ࢦఆ , col_names=ྻ໊Λࢦఆ͍ͨ͠৔߹ʢFALSEʹ͢Δͱྻ໊ΛಡΈࠐ·ͳ͍ʣ , n_max=Կߦ໨·ͰಡΈࠐΉ͔ʁ ) ଓ͖͸σϞͰʂ

Slide 27

Slide 27 text

࡞ͬͨ΋ͷΛؔ਺ʹ͢Δ 27 ଓ͖͸σϞͰʂ

Slide 28

Slide 28 text

ࠂࣔ൪߸ͱ௨൪ͷؔ܎ 28 ৽͍͠೥౓ͷ௨൪ͱݹ͍೥౓ͷࠂࣔ൪߸ͷ਺ࣈ͕߹͏Α͏ʹjoin͢Δ ৽͍͠೥౓ %>% left_join(ݹ͍೥౓, by = c(“௨൪” = “ࠂࣔ൪߸”)

Slide 29

Slide 29 text

׬੒ʂ 29 ଓ͖͸σϞͰʂ

Slide 30

Slide 30 text

ࢢொଜ൪߸ ૯຿লͷσʔλ͔ΒࢢொଜίʔυͷϑΝΠϧ͕μ΢ϯϩʔυͰ͖Δ 30 https://www.soumu.go.jp/denshijiti/code.html

Slide 31

Slide 31 text

ୀӃױऀௐࠪ 31 https://www.mhlw.go.jp/stf/shingi2/0000196043_00003.html

Slide 32

Slide 32 text

·ͣ͸ྻ໊͔Β 32 ෳ਺ྻ໊ ࡏӃ೔਺ ͍Βͳ͍ ϋΠϑϯ ͍Βͳ͍ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ʢ༌݂Ҏ֎ͷ࠶ܝʣ Λআ͍ͨ਺͕߹ܭ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸

Slide 33

Slide 33 text

࡞ઓ ˔ྻ໊ͷΈΛಡΈࠐΈॎʹ͢Δ 33

Slide 34

Slide 34 text

࡞ઓ ˔UJEZS fi MM Ͱ/"Λ্ͷ஋ͰຒΊΔ 34

Slide 35

Slide 35 text

࡞ઓ ˔UJEZSVOJUF Ͱෳ਺ྻΛͭʹ߹ମͤ͞Δ 35 ৽͍͠ྻ໊ ܨ͍͛ͨྻ ϕΫτϧʹม׵ QVMM Ͱ΋0, ͜͜Ͱ͸શ෦

Slide 36

Slide 36 text

࡞ઓ ˔TUS@SFQMBSF@BMMͰ/"΍ه߸Λফ͢ 36 ޷ΈͰ͜ΕΒͷه߸΍εϖʔεΛফ͢͜ͱ΋Մೳ ࠓճ͸ɻͱεϖʔεΛফͨ͠

Slide 37

Slide 37 text

ؔ਺ʹ͢Δ 37 ଓ͖͸σϞͰʂ

Slide 38

Slide 38 text

දه༳Εͷ֬ೝ 38 ࡏӃ೔਺ ͍Βͳ͍ ϋΠϑϯ ͍Βͳ͍ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ʢ༌݂Ҏ֎ͷ࠶ܝʣ Λআ͍ͨ਺͕߹ܭ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸

Slide 39

Slide 39 text

දه༳ΕΛ୳͢ ˔֤೥ͷྻ໊Λऔಘ ˔TFUEJ ff Ͱยํʹ͔͠ͳ͍΋ͷΛ୳͢ TFUEJ ff " #   ▶︎ "ʹ͔͠ແ͍΋ͷΛநग़ TFUEJ ff # "   ▶︎ #ʹ͔͠ແ͍΋ͷΛநग़ 39

Slide 40

Slide 40 text

40 ೥ͱ೥ɺ೥ͱ೥͸ಉ͡ ͱʹҧ͍͕͋Ε͹දه༳ΕͷՄೳੑ ΍͸දه༳ΕͰ͸ͳ͘ ͦ΋ͦ΋ͦͷ೥ʹ͔͠ͳ͔ͬͨՄೳੑ

Slide 41

Slide 41 text

͜ΕͰσʔλ͕ಡΈࠐΊΔʢޙͰ௚͢ʣ 41 ଓ͖͸σϞͰʂ

Slide 42

Slide 42 text

͍Βͳ͍ྻ 42 ࡏӃ೔਺ ͍Βͳ͍ ϋΠϑϯ ͍Βͳ͍ ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ʢ༌݂Ҏ֎ͷ࠶ܝʣ Λআ͍ͨ਺͕߹ܭ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ

Slide 43

Slide 43 text

ࡏӃ೔਺ͱʢ࠶ܝʣͷྻ͸͍Βͳ͍ ˔TFMFDU ͱDPOUBJOT ͰߜΓࠐΊΔ 43

Slide 44

Slide 44 text

ϋΠϑϯΛফ͢ ▶︎ ਺஋σʔλʹม׵ 44 ϋΠϑϯ ͍Βͳ͍ ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ

Slide 45

Slide 45 text

SFBESQBSTF@OVNCFS ˔SFBESQBSTF@OVNCFS ͸ࢦఆͨ͠ྻΛ਺஋σʔλʹ͢Δ ˔NVUBUF BDSPTT બͼ͍ͨྻ ؔ਺ Ͱෳ਺ྻಉ࣌ʹॲཧ͢Δ 45

Slide 46

Slide 46 text

/"Λʹม׵͢Δ ˔SFQMBDF@OB ࢦఆͨ͠ྻ ม׵͍ͨ͠஋   ˔ؔ਺ʹҾ਺͕͋Δ৔߹΍ෳ਺ͷؔ਺Λඞཁͱ͢Δ৔߹͸   ɹNVUBUF BDSPTT બͼ͍ͨྻ dؔ਺ Λ࢖͏   46

Slide 47

Slide 47 text

͜͜·ͰΛؔ਺ʹ͢Δ 47 ଓ͖͸σϞͰʂ

Slide 48

Slide 48 text

ࠂࣔ൪߸͕೥౓Ͱҧ͏໰୊ 48 ೥౓͕มΘΔͱ පӃ໊͕มΘΔ͜ͱ͕͋Δ ԣྻɺॎߦ º೥ؒ ೥౓͕มΘΔͱ ൪߸ҧ͏͜ͱ͕͋Δ ௨൪ʹલͷ೥ͷࠂࣔ൪߸ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ

Slide 49

Slide 49 text

ࢪઃ֓ཁදͱKPJO͢ΔʂDTWϑΝΠϧʹ͢Δ 49 ଓ͖͸σϞͰʂ

Slide 50

Slide 50 text

͜͜·ͰͰ͖Δͱ܁Γฦ͢͜ͱ͕Ͱ͖Δ 50 ԣྻɺॎߦ º೥ؒ ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ

Slide 51

Slide 51 text

51 ྫʣ&YDFMͷෳ਺λϒΛҰؾʹಡΈࠐΉ ܁Γฦ͠QVSSS

Slide 52

Slide 52 text

˔ϑΥϧμ಺ʹ͋ΔΤΫηϧϑΝΠϧΛ·ͱΊͯಡΈࠐΉ -JTU fi MFT ϑΥϧμ໊ NBQؔ਺Λ࢖ͬͯ܁Γฦ͢ 52 pattern=Ͱ৚݅ΛߜΔ ଓ͖͸σϞͰʂ

Slide 53

Slide 53 text

ྻ਺ҧ͏໰୊ 53 ೥౓͕มΘΔͱ ૿͑ͨΓݮͬͨΓ͢Δ ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ

Slide 54

Slide 54 text

XJEFσʔλ 54 σʔλ͕ԣʹฒΜͰ͍Δ

Slide 55

Slide 55 text

MPOHσʔλͳΒؔ܎ͳ͍ʂ ˔UJEZͳΒྻ਺͕૿͑Δ͚ͩ 55

Slide 56

Slide 56 text

UJEZSQJWPU@MPOHFSͰॎ࣋ͪσʔλʹม׵ QJWPU@MPOHFS   ɹDPMT·ͱΊ͍ͨྻ   ɹOBNFT@UP·ͱ·ͬͨޙʢྻ໊ʣͷྻ໊   ɹWBMVFT@UP·ͱ·ͬͨޙʢ஋ʣͷྻ໊ 56

Slide 57

Slide 57 text

QJWPU@MPOHFSͷOBNFT@TFQ OBNFT@TFQͰྻΛ෼ׂ͢Δ͜ͱ͕Ͱ͖Δ 57

Slide 58

Slide 58 text

5JEZ 58 ଓ͖͸σϞͰʂ

Slide 59

Slide 59 text

දه༳ΕΛ௚͢ 59 ೥౓͕มΘΔͱ ප໊มΘΔ͜ͱ͕͋Δ

Slide 60

Slide 60 text

දه༳ΕΛ௚͢ ˔UJEZʹͳ͔ͬͨΒNVUBUFͱJGFMTFͰ͙͢ʹ௚ͤΔ 60

Slide 61

Slide 61 text

61 ྫʣ&YDFMͷෳ਺λϒΛҰؾʹಡΈࠐΉ ܁Γฦ͠QVSSS

Slide 62

Slide 62 text

࠷ޙʹશ෦ͷϑΝΠϧΛಡΈࠐΉ 62 ଓ͖͸σϞͰʂ

Slide 63

Slide 63 text

͓ർΕ༷Ͱͨ͠ʂʂʂ ྻߦͷσʔλʂʂ 63

Slide 64

Slide 64 text

՝୊ ࠓճ͸࠷৽ͷ೥౓ͷපӃͷ໊લʹ߹Θ͍ͤͯΔ   ɹɾ࠷৽ͷ೥౓ʹͳ͍පӃ͸ল͔Ε͍ͯΔ   ɹɾ߹ซͨ͠පӃ͸ରԠ͍ͯ͠ͳ͍ ▶︎ ʢࣗ෼ͷ࢓ࣄͷൣғͩͱࠔΒͳ͍͕ʣ   ɹɹݚڀͰ࢖͏ͳΒ΋ͬͱݫີʹ͢Δඞཁ͕͋Δ͔΋͠Εͳ͍ ▶︎ ɹ%1$Λѻ͏ํͷҙݟΛ࢕ͬͯΈ͍ͨͰ͢ɻ 64

Slide 65

Slide 65 text

લॲཧ͕Ͱ͖Ε͹࣍ʹਐΊΔ ूܭɹɹɹɹɹɹɹɹɹɹɹɹɹɹ8ΞϓϦʢTIJOZʣ 65

Slide 66

Slide 66 text

TIJOZͳΒʂʂʂ 66

Slide 67

Slide 67 text

ऴΘΓʹ ˔&YDFMͷख࡞ۀͰ͸Ͱ͖ͳ͍͜ͱ͕3΍QZUIPOͳΒͰ͖ͯ͠·͍·͢ ˔ҩྍ౷ܭ͸&YDFMͷ࡞Γํ͕ྑ͚Ε͹લॲཧ͸΄΅ඞཁ͋Γ·ͤΜ ˔ͨͩ৭ʑͳσʔλΛ෼ੳͨ͘͠ͳΔͱલॲཧ͸ඞཁʹͳΓ·͢   ɹʢ༧ࢉͷׂҎ্͸લॲཧʹ࢖͍ͬͯΔͱ͍͏ӟ΋ʜʣ ˔·ͩ·ͩஓ੿ͳίʔυ΋ଟ͍Ͱ͢ɻ   ɹͲͷఔ౓ͷ࣮ྗ͔ʢશવμϝ ࠷௿ݶ͸ΫϦΞ ଈઓྗʹͳΔPSOPUʣ΍   ɹΞυόΠεɾϑΟʔυόοΫ͍͚ͨͩΔͱ޾͍Ͱ͢ʂ 67