煩雑な運用を Goを使って楽にする / Go Conference 2019 Spring

by Konboi

Slide 1

Slide 1 text

൥ࡶͳӡ༻Λ  GoΛ࢖ָͬͯʹ͢Δ GoCon2019 Spring / 2019.05.18

Slide 2

Slide 2 text

About me • ໼ਧྒྷհ a.k.a Konboi • גࣜձࣾmerpay • Coupon Team • Backend / Tech Lead

Slide 3

Slide 3 text

About Talk • ӡ༻Ͱൃੜ͢ΔλεΫΛ  ࣗಈԽ/ޮ཰Խ͢ΔաఔͰͷࢼߦࡨޡͨ͠࿩ • αʔόʔ/ΠϯϑϥدΓͷ࿩͸͋·Γग़·ͤΜ • ஍ຯͰ͢

Slide 4

Slide 4 text

Agenda • GoΛ࢖ͬͯExcelΛѻ͏ • GoͰGoogleDriveΛѻ͏ • GoͰେྔͷϑΝΠϧΛੜ੒͢Δ

Slide 5

Slide 5 text

͸͡Ίʹ • ݱ࣌఺ͰΫʔϙϯσʔλ͸ExcelΛ࢖ͬͯೖߘ • ઐ༻ͷೖߘγεςϜ࡞Δͷ͸ίετେ • Θ͔Γ΍͍͢UI • ηΩϡϦςΟ • ద੾ͳݖݶ؅ཧ • γεςϜͷӡ༻ • etc • ϛχϚϜͰݕূͯ͠Ձ஋͕͋ΔͳΒ͖ͪΜͱ੔උ

Slide 6

Slide 6 text

͸͡Ίʹ • ΋ΒͬͨExcel͔ΒखͰσʔλΛ͍͡Δͱ… • ώϡʔϚϯΤϥʔ • ҙਤ͠ͳ͍σʔλͷࠞೖ • ՄೳͳݶΓਓʹΑΔૢ࡞ΛݮΒ͍ͨ͠

Slide 7

Slide 7 text

͸͡Ίʹ • Execl͸ڞ༗ͷ৔ॴͰ؅ཧ͍ͨ͠ • ʓʓ͞ΜͷPCʹ͔͠ͳ͍ • όʔδϣϯ؅ཧ͍ͨ͠ • 2019/05/18 2.xlsx

Slide 8

Slide 8 text

͸͡Ίʹ • Google DriveͰ؅ཧ • G Suite • όʔδϣϯΛ؅ཧ

Slide 9

Slide 9 text

࣮ݱ͍ͨ͜͠ͱ

Slide 10

Slide 10 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 11

Slide 11 text

GoΛ࢖ͬͯExcelΛѻ͏

Slide 12

Slide 12 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 13

Slide 13 text

GoΛ࢖ͬͯExcelΛѻ͏ • ద੾ʹσʔλΛநग़͍ͨ͠ • ద੾ͳϑΥʔϚοτͰѻ͍͍ͨ • Project͕GoͳͷͰGoͰExcelΛѻ͍͍ͨ

Slide 14

Slide 14 text

GoΛ࢖ͬͯExcelΛѻ͏ • Excel͸ॊೈʹϑΥʔϚοτΛมߋͰ͖Δ • ॊೈͳΏ͑ͷ՝୊ • ηϧ݁߹ • ؔ਺ʹΑΔೖྗ • ॻࣜ • etc

Slide 15

Slide 15 text

ௐࠪ/ݕূ

Slide 16

Slide 16 text

ௐࠪ/ݕূ • ࣮ӡ༻ͷલʹௐࠪ/ݕূΛ࣮ࢪ • ͍͔ͭ͘ͷϥΠϒϥϦΛ࢖༻ͯ͠  ߦͬͨݕূͷҰ෦Λ঺հ͠·͢

Slide 17

Slide 17 text

ௐࠪ/ݕূ

Slide 18

Slide 18 text

ௐࠪ/ݕূ • ηϧ݁߹

Slide 19

Slide 19 text

ௐࠪ/ݕূ • ηϧ݁߹ • ࠨ্ͷηϧͷΈσʔλ͕ೖͬͨঢ়ଶʹ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,

Slide 20

Slide 20 text

ௐࠪ/ݕূ • ؔ਺

Slide 21

Slide 21 text

ௐࠪ/ݕূ • ؔ਺ • දࣔ͞Ε͍ͯΔσʔλ͕औಘͰ͖Δ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,

Slide 22

Slide 22 text

ௐࠪ/ݕূ • ؔ਺

Slide 23

Slide 23 text

ௐࠪ/ݕূ • ؔ਺ • ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ $ date "+%Y/%m/%d %H:%M:%S" 2019/05/18 06:58:14    $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,

Slide 24

Slide 24 text

ௐࠪ/ݕূ • ೖྗܗࣜ • Ex)Ϧετ

Slide 25

Slide 25 text

ௐࠪ/ݕূ • ೖྗܗࣜ - Ϧετ • ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,

Slide 26

Slide 26 text

ௐࠪ/ݕূ • ॻࣜઃఆ • Ex) ೔෇

Slide 27

Slide 27 text

ௐࠪ/ݕূ • ॻࣜઃఆ • ϑΥʔϚοτ͕͓͔͍͠ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,

Slide 28

Slide 28 text

ௐࠪ/ݕূ • ॻࣜઃఆ • ೔෇͸Excel಺෦ͰγϦΞϧ஋ͱͯ͠อଘ • ੔਺஋͕೔෇ • গ਺஋͕࣌ࠁ • ϥΠϒϥϦʹΑͬͯ͸γϦΞϧ஋͕  ͦͷ··ग़ྗ͞ΕΔ΋ͷ΋

Slide 29

Slide 29 text

ௐࠪ/ݕূ • ॻࣜઃఆ • ೔෇͸Excel಺෦ͰγϦΞϧ஋ͱͯ͠อଘ

Slide 30

Slide 30 text

ௐࠪ/ݕূ • ॻࣜઃఆ • ςΩετ

Slide 31

Slide 31 text

ௐࠪ/ݕূ • ॻࣜઃఆ • ςΩετ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,1989/01/08,ఱߖ,ฏ੒:heisei,5/18/19 07:06,A, ฏ੒,heisei,2019/04/30,ఱߖ,ฏ੒:heisei,5/18/19 07:06,, ྩ࿨,reiwa,2019/05/01,্ߖ,ྩ࿨:reiwa,5/18/19 07:06,B,

Slide 32

Slide 32 text

ௐࠪ/ݕূ ݁Ռ • ΍͔͍ͬͳͷ͸ηϧ݁߹ͱॻࣜઃఆ • ͦΕҎ֎͸ҙ֎ͱ໰୊ͳͦ͞͏

Slide 33

Slide 33 text

ௐࠪ/ݕূ ݁Ռ • ͢΂ͯจࣈྻͰ΍Ε͹໰୊͸ͳ͍ • ໰୊͸ͳ͍…

Slide 34

Slide 34 text

ௐࠪ/ݕূ ݁Ռ • ͢΂ͯจࣈྻͩͱ… • શ֯/൒֯ͷѻ͍Λߟྀ • ೔ʹͪͳͲΛϑΥʔϚοτ͕͋Δ΋ͷ͸  ॻࣜઃఆΛઃఆ͠ద੾ͳ஋Λೖྗ/औಘ͍ͨ͠

Slide 35

Slide 35 text

ௐࠪ/ݕূ ݁Ռ • 360EntSecGroup-Skylar/excelize Λ࠾༻ • SetCellStyle method • ॻࣜઃఆΛઃఆͰ͖Δ

Slide 36

Slide 36 text

• Ex) 14: m/d/yy • Ex) 15: d-mmm-yy ௐࠪ/ݕূ ݁Ռ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,8-Jan-89,ఱߖ,ฏ੒:heisei,5/18/19 07:09,A, ฏ੒,heisei,30-Apr-19,ఱߖ,ฏ੒:heisei,5/18/19 07:09,, ྩ࿨,reiwa,1-May-19,্ߖ,ྩ࿨:reiwa,5/18/19 07:09,B, $ go run main.go -f ../example.xlsx ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 07:09,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 07:09,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 07:09,B,

Slide 37

Slide 37 text

ௐࠪ/ݕূ ݁Ռ • ଞʹ΋ExcelΛGo͔Βૢ࡞Մೳ • RemoveRow() • SaveAs() • ExcelʹσʔλΛ௥ه/मਖ਼ͯ͠อଘ͕Մೳ • ೖߘεςʔλεͷ؅ཧ͕ExcelͰՄೳʹ

Slide 38

Slide 38 text

ௐࠪ/ݕূ ݁Ռ • จࣈྻ/୯७ͳσʔλͰ͋Ε͹  ͲͷϥΠϒϥϦͰ΋େମ૝ఆ௨Γʹʹಈ࡞ • ༻్ʹԠͨ͡ϥΠϒϥϦΛ࢖͏ͷ͕Α͍͔ͱ

Slide 39

Slide 39 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 40

Slide 40 text

GoΛ࢖ͬͯ  Google DriveΛѻ͏

Slide 41

Slide 41 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 42

Slide 42 text

GoͰGoogleDriveΛѻ͏ • ެࣜͷSDK • google.golang.org/api/drive/v3 • جຊ͸ެࣜυΩϡϝϯτΛݟΕ͹OK • https://developers.google.com/drive/api/v3/ reference/ • ݸਓతʹቕͬͨͱ͜ΖΛ঺հ 

Slide 43

Slide 43 text

ϑΝΠϧϦετ͸FieldΛࢦఆ • ࢦఆ͠ͳ͍ͱid,name, kind, mimeTypeͷΈ • ˎࢦఆ͢Δͱ͢΂ͯͷσʔλ͕  ฦͬͯ͘Δ͔ύϑΥʔϚϯε͕ྼԽ • srv.Files.List(). Fields(“files(parents, id, name, owners, kind, mimeType)"). Do()    srv.Files.List().Fields(“*”).Do()

Slide 44

Slide 44 text

Update͢Δͱ͖ͷtips • List APIܦ༝ͰFile ObjectΛऔಘ • ಛఆͷFieldΛߋ৽ͯ͠Update͢Δͱ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }  file := list[0]  file.Name = “dummy”  _, err := service.Files.Update(file.Id, file).Context(ctx).Do() if err != nil { return errors.Wrap(err, "error updated") }    > The resource body includes fields which are not directly writable.

Slide 45

Slide 45 text

Update͢Δͱ͖ͷtips • APIͰ͸ߋ৽Ͱ͖ͳ͍Field΋ߋ৽͕͔͔Δ • ߋ৽͍ͨ͠FiledΛࢦఆ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }    file := list[0]  _, err := service.Files.Update(file.Id, &drive.File{ Name: “dummy”, }).Context(ctx).Do()   if err != nil { return errors.Wrap(err, "error updated") }

Slide 46

Slide 46 text

Update͢Δͱ͖ͷtips • ϑΝΠϧͷத਎Λߋ৽ • Media(io.Reader, Content-Type) Λ࢖༻ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }    file := list[0]  content := &bytes.Buffer{} content.WriteString(“hello”)  updatedFile, err := service.Files.Update(file.Id, &drive.File{}). Media(content, googleapi.ContentType(file.MimeType)).Context(ctx).Do()

Slide 47

Slide 47 text

Update͢Δͱ͖ͷtips • όʔδϣϯ؅ཧ͞ΕΔͷͰ  ҆৺(?)࣮ͯ͠ߦՄೳ

Slide 48

Slide 48 text

࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍ • v2ͩͱ Children: list ͱ͍͏API͕͋Δ • FileTypeΛνΣοΫͯࣗ͠લͰ  ࣮૷͢Δඞཁ͕͋Δ • .Q(“xxx in parents”) • ਌ࢠؔ܎Λࢦఆ

Slide 49

Slide 49 text

࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍ • ৄ͘͠͸ Google Drive API v3 Ͱݕࡧ • sample: Konboi/go-google-drive-example  list, err := service.Files.List().Q(fmt.Sprintf("'%s' in parents", driveID)).Context(ctx).Do() if err != nil { return errors.Wrap(err, "list error") }    typeFolder := "application/vnd.google-apps.folder" for _, f := range list.Files { pp.Printf("file id:%s\nname%s\ntype:%v\n", f.Id, f.Name, f.MimeType) if f.MimeType == typeFolder { err := ListRecursive(ctx, service, f.Id) if err != nil { return err } } }

Slide 50

Slide 50 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 51

Slide 51 text

YAML ࣮ݱ͍ͨ͜͠ͱ

Slide 52

Slide 52 text

GoͰେྔͷϑΝΠϧΛ  ੜ੒͢Δ

Slide 53

Slide 53 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ

Slide 54

Slide 54 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ΫʔϙϯʹΑͬͯ͸֤ϢʔβϢχʔΫͳ  ίʔυΛൃߦ͢Δඞཁ͕͋Δ • ༧Ί࣮ࢪళฮ͔ΒίʔυΛൃߦ • ݱঢ়ͷγεςϜͷ౎߹্ɺࣄલʹը૾Λ  ੜ੒͠഑෍͢Δ࢓૊Έ • ࣄલʹେྔͷը૾Λੜ੒͢Δඞཁ͕͋Δ

Slide 55

Slide 55 text

গྔͳΒ໰୊ͳ͍͕

Slide 56

Slide 56 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ̍ຕ 0.1sec • 10ຕ 1sec • 1000ຕ 100sec • 1ສຕ 1000sec (2.5h) • 100ສຕ 100000sec (27h) • 1000ສຕ 1000000sec (270h) • …

Slide 57

Slide 57 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ̍ຕ 100kb • 10ຕ 1000kb • 1000ຕ 100MB • 1ສຕ 1GB • 100ສຕ 100GB • 1000ສຕ 1TB • …

Slide 58

Slide 58 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • goroutineͰ΋ݶք͕… • ڧ͍ΠϯελϯεͰ࡞Δ͜ͱ΋ߟ͕͑ͨ • ౎౓ launch > deploy > shutdown …

Slide 59

Slide 59 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ

Slide 60

Slide 60 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • Google Cloud Functionͱ͸ • ΠϕϯτۦಈܕͷαʔόʔϨε   ίϯϐϡʔςΟϯά ϓϥοτϑΥʔϜ • FaaS • AWS Lambda

Slide 61

Slide 61 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ ?code=abcd123  ?code=efgh456  ?code=abab1212

Slide 62

Slide 62 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ func GenCode(w http.ResponseWriter, r *http.Request) { v := r.URL.Query() code := v.Get("code") img, err := genCode(code, baseImage) if err != nil { return } buf := &bytes.Buffer{} if err := png.Encode(buf, img); err != nil { return } if err := upload(r.Context(), buf, filename); err != nil { return } fmt.Fprintf(w, "OK") w.WriteHeader(http.StatusOK) }

Slide 63

Slide 63 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ $ zip -r function.zip function.go some.png vendor $ gcloud beta functions deploy \ --project=${PROJECT_ID} \ --region=${REGION} \ --memory=${MEMORY} \ --runtime=${RUNTIME} \ --service-account=${SERVICE_ACCOUNT} \ --source=gs://${UPLOAD_BUCKET_PATH}/${FUNCTION}.zip \ ${FUNCTION} \ --entry-point=${FUNCTION} \ —trigger-http

Slide 64

Slide 64 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • func (w http.Writer, r *http.Request) Λ  ΄Μͱͦͷ··ΞοϓϩʔυͰ͖Δ • ґଘpackage΋ҰॹʹzipʹؚΊΕ͹࢖༻Մೳ • assetϑΝΠϧͳͲ΋ಉࠝՄೳ • zipϑΝΠϧ͓Αͼղౚͨ͠ͱ͖ͷ༰ྔ੍ݶ༗

Slide 65

Slide 65 text

GoͰେྔͷϑΝΠϧΛੜ੒͢Δ ?code=abcd123  ?code=efgh456  ?code=abab1212

Slide 66

Slide 66 text

·ͱΊ • ӡ༻Ͱൃੜ͢ΔλεΫΛ  ࣗಈԽ/ޮ཰Խ͢Δաఔͷࢼߦࡨޡͨ͠࿩ • GoΛ࢖ͬͯExcelΛѻ͏ • GoͰGoogleDriveΛѻ͏ • GoͰେྔͷϑΝΠϧΛੜ੒͢Δ