Tokyo.R#93 Data processing

Slide 1

Slide 1 text

#93 @kilometer00 2021.07.03 BeginneR Session -- Data processing --

Slide 2

Slide 2 text

Who！？ Who?

Slide 3

Slide 3 text

Who！？・ @kilometer ・Postdoc Researcher (Ph.D. Eng.) ・Neuroscience ・Computational Behavior ・Functional brain imaging ・R： ~ 10 years

Slide 4

Slide 4 text

宣伝!!（書籍の翻訳に参加しました。）絶賛販売中！

Slide 5

Slide 5 text

BeginneR Session

Slide 6

Slide 6 text

BeginneR

Slide 7

Slide 7 text

Before A'er BeginneR Session BeginneR BeginneR

Slide 8

Slide 8 text

BeginneR Advanced Hoxo_m If I have seen further it is by standing on the shoulders of Giants. -- Sir Isaac Newton, 1676

Slide 9

Slide 9 text

import Tidy Transform Visualise Model Communicate Modiﬁed from “R for Data Science”, H. Wickham, 2017

Slide 10

Slide 10 text

import Tidy Transform Visualise Model Communicate Modiﬁed from “R for Data Science”, H. Wickham, 2017 preprocessing Data processing Data science

Slide 11

Slide 11 text

import Tidy Transform Visualise Model Communicate Modiﬁed from “R for Data Science”, H. Wickham, 2017 preprocessing Data science Data Observa@on Hypothesis feedback Data processing Narra/ve of data

Slide 12

Slide 12 text

import Tidy Transform Visualise Model Communicate Modiﬁed from “R for Data Science”, H. Wickham, 2017 preprocessing Data science Data Observa@on Hypothesis Narra/ve of data feedback Data processing

Slide 13

Slide 13 text

import Tidy Transform Visualise Model Communicate Modiﬁed from “R for Data Science”, H. Wickham, 2017 Data processing

Slide 14

Slide 14 text

raed_csv() write_csv() Data Wide form Long form pivot_longer() Nested form pivot_wider() Figures group_nest() unnest() {ggplot2} {patchwork} Data processing

Slide 15

Slide 15 text

raed_csv() write_csv() Data Wide form Long form pivot_longer() Nested form pivot_wider() Figures group_nest() unnest() {ggplot2} {patchwork} data.frame tibble Data processing

Slide 16

Slide 16 text

data.frame

Slide 17

Slide 17 text

vector in Excel

Slide 18

Slide 18 text

vector in R in Excel pre <- c(1, 2, 3, 4, 5) post <- pre * 5 > pre [1] 1 2 3 4 5 > post [1] 5 10 15 20 25

Slide 19

Slide 19 text

vector vec1 <- c(1, 2, 3, 4, 5) vec2 <- 1:5 vec3 <- seq(from = 1, to = 5, by = 1) > vec1 [1] 1 2 3 4 5 > vec2 [1] 1 2 3 4 5 > vec3 [1] 1 2 3 4 5

Slide 20

Slide 20 text

vector vec1 <- seq(from = 1, to = 5, by = 1) vec2 <- seq(1, 5, 1) > vec1 [1] 1 2 3 4 5 > vec2 [1] 1 2 3 4 5

Slide 21

Slide 21 text

> ?seq vector seq{base} Sequence Generation Description Generate regular sequences. seq is a standard generic with a default method. … Usage seq(...) ## Default S3 method: seq(from = 1, to = 1, by = ((to - from)/(length.out - 1)), length.out = NULL, along.with = NULL, ...)

Slide 22

Slide 22 text

vector vec1 <- rep(1:3, times = 2) vec2 <- rep(1:3, each = 2) vec3 <- rep(1:3, times = 2, each = 2) > vec1 [1] 1 2 3 1 2 3 > vec2 [1] 1 1 2 2 3 3 > vec3 [1] 1 1 2 2 3 3 1 1 2 2 3 3

Slide 23

Slide 23 text

vector vec1 <- 11:15 > vec1 [1] 11 12 13 14 15 > vec1[1] [1] 11 > vec1[3:5] [1] 13 14 15 > vec1[c(1:2, 5)] [1] 11 12 15

Slide 24

Slide 24 text

list list1 <- list(1:6, 11:15, c("a", "b", "c")) > list1 [[1]] [1] 1 2 3 4 5 6 [[2]] [1] 11 12 13 14 15 [[3]] [1] "a" "b" "c"

Slide 25

Slide 25 text

list list1 <- list(1:6, 11:15, c("a", "b", "c")) > list1[[1]] [1] 1 2 3 4 5 6 > list1[[3]][2:3] [1] "b" "c" > list1[[2]] * 3 [1] 33 36 39 42 45

Slide 26

Slide 26 text

named list list2 <- list(A = 1:6, B = 11:15, C = c("a", "b", "c")) > list2 $A [1] 1 2 3 4 5 6 $B [1] 11 12 13 14 15 $C [1] "a" "b" "c"

Slide 27

Slide 27 text

> list2$A [1] 1 2 3 4 5 6 > list2$C[2:3] [1] "b" "c" > list2$B * 3 [1] 33 36 39 42 45 named list list2 <- list(A = 1:6, B = 11:15, C = c("a", "b", "c"))

Slide 28

Slide 28 text

list1 <- list(1:6, 11:15, c("a", "b", "c")) > class(list1) [1] "list" > names(list1) NULL list2 <- list(A = 1:6, B = 11:15, C = c("a", "b", "c")) > class(list2) [1] "list" > names(list2) [1] "A" "B" "C" named list list

Slide 29

Slide 29 text

list3 <- list(A = 1:3, B = 11:13) > class(list3) [1] "list" > names(list3) [1] "A" "B" df1 <- data.frame(A = 1:3, B = 11:13) > class(df1) [1] "data.frame" > names(df1) [1] "A" "B" named list & data.frame

Slide 30

Slide 30 text

> str(list3) List of 2 $ A: int [1:3] 1 2 3 $ B: int [1:3] 11 12 13 > str(df1) 'data.frame': 3 obs. of 2 variables: $ A: int 1 2 3 $ B: int 11 12 13 list3 <- list(A = 1:3, B = 11:13) df1 <- data.frame(A = 1:3, B = 11:13) named list & data.frame

Slide 31

Slide 31 text

> list3 $A [1] 1 2 3 $B [1] 11 12 13 > df1 A B 1 1 11 2 2 12 3 3 13 named list & data.frame

Slide 32

Slide 32 text

> list3 $A [1] 1 2 3 $B [1] 11 12 13 > df1 A B 1 1 11 2 2 12 3 3 13 named list & data.frame observa9on variable

Slide 33

Slide 33 text

data.frame v.s. matrix A B 1 1 11 2 2 12 3 3 13 [,1] [,2] [1,] 1 11 [2,] 2 12 [3,] 3 13 df1 <- data.frame(A = 1:3, B = 11:13) > str(mat1) int [1:3, 1:2] 1 2 3 11 12 13 > str(df1) 'data.frame': 3 obs. of 2 vars.: $ A: int 1 2 3 $ B: int 11 12 13 mat1 <- matrix(c(1:3, 11:13), 3, 2)

Slide 34

Slide 34 text

raed_csv() write_csv() Data Wide form Long form pivot_longer() Nested form pivot_wider() Figures group_nest() unnest() {ggplot2} {patchwork} data.frame tibble Data processing

Slide 35

Slide 35 text

raed_csv() write_csv() Data Wide form Long form pivot_longer() Nested form pivot_wider() Figures group_nest() unnest() {ggplot2} {patchwork} data.frame ,bble Data processing Transform (verb func@ons) {dplyr}

Slide 36

Slide 36 text

vignette("dplyr")

Slide 37

Slide 37 text

It (dplyr) provides simple “verbs” to help you translate your thoughts into code. func?ons that correspond to the most common data manipula?on tasks Introduc>on to dplyr h"ps://cran.r-project.org/web/packages/dplyr/vigne"es/dplyr.html WFSCT {dplyr}

Slide 38

Slide 38 text

dplyrは、あなたの考えをコードに翻訳するための【動詞】を提供する。データ操作における基本のキを、シンプルに実⾏できる関数 (群) Introduc>on to dplyr h"ps://cran.r-project.org/web/packages/dplyr/vigne"es/dplyr.html WFSCT {dplyr} ※ かなり意訳

Slide 39

Slide 39 text

1. mutate() 2. filter() 3. select() 4. group_by() 5. summarize() 6. left_join() 7. arrange() Data.frame manipula9on

Slide 40

Slide 40 text

1. mutate() 2. filter() 3. select() 4. group_by() 5. summarize() 6. left_join() 7. arrange() Data.frame manipula9on 0. %>%

Slide 41

Slide 41 text

1JQFBMHFCSB X %>% f X %>% f(y) X %>% f %>% g X %>% f(y, .) f(X) f(X, y) g(f(X)) f(y, X) %>% {magri8r} 「dplyr再⼊⾨（基本編）」yutanihilaCon h"ps://speakerdeck.com/yutannihila6on/dplyrzai-ru-men-ji-ben-bian

Slide 42

Slide 42 text

① ② ③ ④ lift take pour put Bring milk from the kitchen!

Slide 43

Slide 43 text

① lift Bring milk from the kitchen! lift(Robot, glass, table) -> Robot' take ② take(Robot', fridge, milk) -> Robot''

Slide 44

Slide 44 text

Bring milk from the kitchen! Robot' <- lift(Robot, glass, table) Robot'' <- take(Robot', fridge, milk) Robot''' <- pour(Robot'', milk, glass) result <- put(Robot''', glass, table) result <- Robot %>% lift(glass, table) %>% take(fridge, milk) %>% pour(milk, glass) %>% put(glass, table) by using pipe, # ① # ② # ③ # ④ # ① # ② # ③ # ④

Slide 45

Slide 45 text

The @dyverse style guides h"ps://style.;dyverse.org/syntax.html#object-names "There are only two hard things in Computer Science: cache invalidation and naming things"

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Robot' <- lift(Robot, glass, table) Robot'' <- take(Robot', fridge, milk) Robot''' <- pour(Robot'', milk, glass) result <- put(Robot''', glass, table) result <- Robot %>% lift(glass, table) %>% take(fridge, milk) %>% pour(milk, glass) %>% put(glass, table) by using pipe, # ① # ② # ③ # ④ # ① # ② # ③ # ④ Thinking Reading Bring milk from the kitchen!

Slide 48

Slide 48 text

Programing Write Run Read Think Write Run Read Think Communicate Share

Slide 49

Slide 49 text

1JQFBMHFCSB X %>% f X %>% f(y) X %>% f %>% g X %>% f(y, .) f(X) f(X, y) g(f(X)) f(y, X) %>% {magrittr} 「dplyr再⼊⾨（基本編）」yutanihilation https://speakerdeck.com/yutannihilation/dplyrzai-ru-men-ji-ben-bian

Slide 50

Slide 50 text

1. mutate() 2. filter() 3. select() 4. group_by() 5. summarize() 6. left_join() 7. arrange() Data.frame manipula9on 0. %>% ✔

Slide 51

Slide 51 text

WFSCT {dplyr} mutate # カラムの追加 + mutate(dat, C = fun(A, B))

Slide 52

Slide 52 text

WFSCT {dplyr} mutate # カラムの追加 + dat %>% mutate(C = fun(A, B))

Slide 53

Slide 53 text

WFSCT {dplyr} filter # 行の絞り込み dat %>% filter(tag %in% c(1, 3, 5))

Slide 54

Slide 54 text

ブール演算⼦ Boolean Algebra A == B A != B George Boole 1815 - 1864 A | B A & B A %in% B # equal to # not equal to # or # and # is A in B? wikipedia

Slide 55

Slide 55 text

"a" != "b" # is A in B? ブール演算⼦ Boolean Algebra [1] TRUE 1 %in% 10:100 # is A in B? [1] FALSE

Slide 56

Slide 56 text

George Boole 1815 - 1864 A Class-Room Introduc;on to Logic h"ps://niyamaklogic.wordpress.com/c ategory/laws-of-thoughts/ Mathema=cian Philosopher &

Slide 57

Slide 57 text

WFSCT {dplyr} select # カラムの選択 dat %>% select(tag, B)

Slide 58

Slide 58 text

WFSCT {dplyr} select # カラムの選択 dat %>% select("tag", "B")

Slide 59

Slide 59 text

WFSCT {dplyr} select # カラムの選択 dat %>% select("tag", "B") dat %>% select(tag, B)

Slide 60

Slide 60 text

WFSCT {dplyr} # Select help func?ons starts_with("s") ends_with("s") contains("se") matches("^.e") one_of(c(”tag", ”B")) everything() https://kazutan.github.io/blog/2017/04/dplyr-select-memo/ 「dplyr::selectの活⽤例メモ」kazutan

Slide 61

Slide 61 text

1. mutate() 2. filter() 3. select() 4. group_by() 5. summarize() 6. left_join() 7. arrange() Data.frame manipula9on 0. %>% ✔ ✔ ✔ ✔

Slide 62

Slide 62 text

(SBNNBSPGEBUBNBOJQVMBUJPO By constraining your op9ons, it helps you think about your data manipula9on challenges. Introduc>on to dplyr hLps://cran.r-project.org/web/packages/dplyr/vigneLes/dplyr.html

Slide 63

Slide 63 text

選択肢を制限することで、データ解析のステップをシンプルに考えられますヨ。（めっちゃ意訳） Introduc>on to dplyr hLps://cran.r-project.org/web/packages/dplyr/vigneLes/dplyr.html ※ まさに意訳 (SBNNBSPGEBUBNBOJQVMBUJPO

Slide 64

Slide 64 text

より多くの制約を課す事で、魂の⾜枷から、より⾃由になる。 Igor Stravinsky И8горь Ф Страви́нский The more constraints one imposes, the more one frees one's self of the chains that shackle the spirit. 1882 - 1971 ※ 割と意訳

Slide 65

Slide 65 text

Enjoy!!