煩雑な運用を Goを使って楽にする / Go Conference 2019 Spring

GoCon 2019 で発表した 煩雑な運用を
Goを使って楽にする の発表資料になります

Transcript

1. ൥ࡶͳӡ༻Λ
 GoΛ࢖ָͬͯʹ͢Δ GoCon2019 Spring / 2019.05.18
2. About me • ໼ਧྒྷհ a.k.a Konboi • גࣜձࣾmerpay • Coupon Team • Backend / Tech Lead
3. About Talk • ӡ༻Ͱൃੜ͢ΔλεΫΛ
 ࣗಈԽ/ޮ཰Խ͢ΔաఔͰͷࢼߦࡨޡͨ͠࿩ • αʔόʔ/ΠϯϑϥدΓͷ࿩͸͋·Γग़·ͤΜ • ஍ຯͰ͢
4. Agenda • GoΛ࢖ͬͯExcelΛѻ͏ • GoͰGoogleDriveΛѻ͏ • GoͰେྔͷϑΝΠϧΛੜ੒͢Δ
5. ͸͡Ίʹ • ݱ࣌఺ͰΫʔϙϯσʔλ͸ExcelΛ࢖ͬͯೖߘ • ઐ༻ͷೖߘγεςϜ࡞Δͷ͸ίετେ • Θ͔Γ΍͍͢UI • ηΩϡϦςΟ • ద੾ͳݖݶ؅ཧ • γεςϜͷӡ༻ • etc • ϛχϚϜͰݕূͯ͠Ձ஋͕͋ΔͳΒ͖ͪΜͱ੔උ
6. ͸͡Ίʹ • ΋ΒͬͨExcel͔ΒखͰσʔλΛ͍͡Δͱ… • ώϡʔϚϯΤϥʔ • ҙਤ͠ͳ͍σʔλͷࠞೖ • ՄೳͳݶΓਓʹΑΔૢ࡞ΛݮΒ͍ͨ͠
7. ͸͡Ίʹ • Execl͸ڞ༗ͷ৔ॴͰ؅ཧ͍ͨ͠ • ʓʓ͞ΜͷPCʹ͔͠ͳ͍ • όʔδϣϯ؅ཧ͍ͨ͠ • 2019/05/18 2.xlsx
8. ͸͡Ίʹ • Google DriveͰ؅ཧ • G Suite • όʔδϣϯΛ؅ཧ
9. ࣮ݱ͍ͨ͜͠ͱ
10. YAML ࣮ݱ͍ͨ͜͠ͱ
11. GoΛ࢖ͬͯExcelΛѻ͏
12. YAML ࣮ݱ͍ͨ͜͠ͱ
13. GoΛ࢖ͬͯExcelΛѻ͏ • ద੾ʹσʔλΛநग़͍ͨ͠ • ద੾ͳϑΥʔϚοτͰѻ͍͍ͨ • Project͕GoͳͷͰGoͰExcelΛѻ͍͍ͨ
14. GoΛ࢖ͬͯExcelΛѻ͏ • Excel͸ॊೈʹϑΥʔϚοτΛมߋͰ͖Δ • ॊೈͳΏ͑ͷ՝୊ • ηϧ݁߹ • ؔ਺ʹΑΔೖྗ • ॻࣜ • etc
15. ௐࠪ/ݕূ
16. ௐࠪ/ݕূ • ࣮ӡ༻ͷલʹௐࠪ/ݕূΛ࣮ࢪ • ͍͔ͭ͘ͷϥΠϒϥϦΛ࢖༻ͯ͠
 ߦͬͨݕূͷҰ෦Λ঺հ͠·͢
17. ௐࠪ/ݕূ
18. ௐࠪ/ݕূ • ηϧ݁߹
19. ௐࠪ/ݕূ • ηϧ݁߹ • ࠨ্ͷηϧͷΈσʔλ͕ೖͬͨঢ়ଶʹ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,
20. ௐࠪ/ݕূ • ؔ਺
21. ௐࠪ/ݕূ • ؔ਺ • දࣔ͞Ε͍ͯΔσʔλ͕औಘͰ͖Δ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,
22. ௐࠪ/ݕূ • ؔ਺
23. ௐࠪ/ݕূ • ؔ਺ • ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ $ date "+%Y/%m/%d %H:%M:%S" 2019/05/18 06:58:14
 
 $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,
24. ௐࠪ/ݕূ • ೖྗܗࣜ • Ex)Ϧετ
25. ௐࠪ/ݕূ • ೖྗܗࣜ - Ϧετ • ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,
26. ௐࠪ/ݕূ • ॻࣜઃఆ • Ex) ೔෇
27. ௐࠪ/ݕূ • ॻࣜઃఆ • ϑΥʔϚοτ͕͓͔͍͠ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 06:53,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 06:53,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 06:53,B,
28. ௐࠪ/ݕূ • ॻࣜઃఆ • ೔෇͸Excel಺෦ͰγϦΞϧ஋ͱͯ͠อଘ • ੔਺஋͕೔෇ • গ਺஋͕࣌ࠁ • ϥΠϒϥϦʹΑͬͯ͸γϦΞϧ஋͕
 ͦͷ··ग़ྗ͞ΕΔ΋ͷ΋
29. ௐࠪ/ݕূ • ॻࣜઃఆ • ೔෇͸Excel಺෦ͰγϦΞϧ஋ͱͯ͠อଘ
30. ௐࠪ/ݕূ • ॻࣜઃఆ • ςΩετ
31. ௐࠪ/ݕূ • ॻࣜઃఆ • ςΩετ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,1989/01/08,ఱߖ,ฏ੒:heisei,5/18/19 07:06,A, ฏ੒,heisei,2019/04/30,ఱߖ,ฏ੒:heisei,5/18/19 07:06,, ྩ࿨,reiwa,2019/05/01,্ߖ,ྩ࿨:reiwa,5/18/19 07:06,B,
32. ௐࠪ/ݕূ ݁Ռ • ΍͔͍ͬͳͷ͸ηϧ݁߹ͱॻࣜઃఆ • ͦΕҎ֎͸ҙ֎ͱ໰୊ͳͦ͞͏
33. ௐࠪ/ݕূ ݁Ռ • ͢΂ͯจࣈྻͰ΍Ε͹໰୊͸ͳ͍ • ໰୊͸ͳ͍…
34. ௐࠪ/ݕূ ݁Ռ • ͢΂ͯจࣈྻͩͱ… • શ֯/൒֯ͷѻ͍Λߟྀ • ೔ʹͪͳͲΛϑΥʔϚοτ͕͋Δ΋ͷ͸
 ॻࣜઃఆΛઃఆ͠ద੾ͳ஋Λೖྗ/औಘ͍ͨ͠
35. ௐࠪ/ݕূ ݁Ռ • 360EntSecGroup-Skylar/excelize Λ࠾༻ • SetCellStyle method • ॻࣜઃఆΛઃఆͰ͖Δ
36. • Ex) 14: m/d/yy • Ex) 15: d-mmm-yy ௐࠪ/ݕূ ݁Ռ $ go run main.go -f ../example.xlsx name,name_en,date,status,id,created_at,category, ฏ੒,heisei,8-Jan-89,ఱߖ,ฏ੒:heisei,5/18/19 07:09,A, ฏ੒,heisei,30-Apr-19,ఱߖ,ฏ੒:heisei,5/18/19 07:09,, ྩ࿨,reiwa,1-May-19,্ߖ,ྩ࿨:reiwa,5/18/19 07:09,B, $ go run main.go -f ../example.xlsx ฏ੒,heisei,01-08-89,ఱߖ,ฏ੒:heisei,5/18/19 07:09,A, ฏ੒,heisei,04-30-19,ఱߖ,ฏ੒:heisei,5/18/19 07:09,, ྩ࿨,reiwa,05-01-19,্ߖ,ྩ࿨:reiwa,5/18/19 07:09,B,
37. ௐࠪ/ݕূ ݁Ռ • ଞʹ΋ExcelΛGo͔Βૢ࡞Մೳ • RemoveRow() • SaveAs() • ExcelʹσʔλΛ௥ه/मਖ਼ͯ͠อଘ͕Մೳ • ೖߘεςʔλεͷ؅ཧ͕ExcelͰՄೳʹ
38. ௐࠪ/ݕূ ݁Ռ • จࣈྻ/୯७ͳσʔλͰ͋Ε͹
 ͲͷϥΠϒϥϦͰ΋େମ૝ఆ௨Γʹʹಈ࡞ • ༻్ʹԠͨ͡ϥΠϒϥϦΛ࢖͏ͷ͕Α͍͔ͱ
39. YAML ࣮ݱ͍ͨ͜͠ͱ
40. GoΛ࢖ͬͯ
 Google DriveΛѻ͏
41. YAML ࣮ݱ͍ͨ͜͠ͱ
42. GoͰGoogleDriveΛѻ͏ • ެࣜͷSDK • google.golang.org/api/drive/v3 • جຊ͸ެࣜυΩϡϝϯτΛݟΕ͹OK • https://developers.google.com/drive/api/v3/ reference/ • ݸਓతʹቕͬͨͱ͜ΖΛ঺հ

43. ϑΝΠϧϦετ͸FieldΛࢦఆ • ࢦఆ͠ͳ͍ͱid,name, kind, mimeTypeͷΈ • ˎࢦఆ͢Δͱ͢΂ͯͷσʔλ͕
 ฦͬͯ͘Δ͔ύϑΥʔϚϯε͕ྼԽ • srv.Files.List(). Fields(“files(parents, id, name, owners, kind, mimeType)"). Do()
 
 srv.Files.List().Fields(“*”).Do()
44. Update͢Δͱ͖ͷtips • List APIܦ༝ͰFile ObjectΛऔಘ • ಛఆͷFieldΛߋ৽ͯ͠Update͢Δͱ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }
 file := list[0]
 file.Name = “dummy”
 _, err := service.Files.Update(file.Id, file).Context(ctx).Do() if err != nil { return errors.Wrap(err, "error updated") }
 
 > The resource body includes fields which are not directly writable.
45. Update͢Δͱ͖ͷtips • APIͰ͸ߋ৽Ͱ͖ͳ͍Field΋ߋ৽͕͔͔Δ • ߋ৽͍ͨ͠FiledΛࢦఆ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }
 
 file := list[0]
 _, err := service.Files.Update(file.Id, &drive.File{ Name: “dummy”, }).Context(ctx).Do() 
 if err != nil { return errors.Wrap(err, "error updated") }
46. Update͢Δͱ͖ͷtips • ϑΝΠϧͷத਎Λߋ৽ • Media(io.Reader, Content-Type) Λ࢖༻ list, err := service.Files.List().Context(ctx).Do() if err != nil { return err }
 
 file := list[0]
 content := &bytes.Buffer{} content.WriteString(“hello”)
 updatedFile, err := service.Files.Update(file.Id, &drive.File{}). Media(content, googleapi.ContentType(file.MimeType)).Context(ctx).Do()
47. Update͢Δͱ͖ͷtips • όʔδϣϯ؅ཧ͞ΕΔͷͰ
 ҆৺(?)࣮ͯ͠ߦՄೳ
48. ࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍ • v2ͩͱ Children: list ͱ͍͏API͕͋Δ • FileTypeΛνΣοΫͯࣗ͠લͰ
 ࣮૷͢Δඞཁ͕͋Δ • .Q(“xxx in parents”) • ਌ࢠؔ܎Λࢦఆ
49. ࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍ • ৄ͘͠͸ Google Drive API v3 Ͱݕࡧ • sample: Konboi/go-google-drive-example
 list, err := service.Files.List().Q(fmt.Sprintf("'%s' in parents", driveID)).Context(ctx).Do() if err != nil { return errors.Wrap(err, "list error") }
 
 typeFolder := "application/vnd.google-apps.folder" for _, f := range list.Files { pp.Printf("file id:%s\nname%s\ntype:%v\n", f.Id, f.Name, f.MimeType) if f.MimeType == typeFolder { err := ListRecursive(ctx, service, f.Id) if err != nil { return err } } }
50. YAML ࣮ݱ͍ͨ͜͠ͱ
51. YAML ࣮ݱ͍ͨ͜͠ͱ
52. GoͰେྔͷϑΝΠϧΛ
 ੜ੒͢Δ
53. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ
54. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ΫʔϙϯʹΑͬͯ͸֤ϢʔβϢχʔΫͳ
 ίʔυΛൃߦ͢Δඞཁ͕͋Δ • ༧Ί࣮ࢪళฮ͔ΒίʔυΛൃߦ • ݱঢ়ͷγεςϜͷ౎߹্ɺࣄલʹը૾Λ
 ੜ੒͠഑෍͢Δ࢓૊Έ • ࣄલʹେྔͷը૾Λੜ੒͢Δඞཁ͕͋Δ
55. গྔͳΒ໰୊ͳ͍͕
56. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ̍ຕ 0.1sec • 10ຕ 1sec • 1000ຕ 100sec • 1ສຕ 1000sec (2.5h) • 100ສຕ 100000sec (27h) • 1000ສຕ 1000000sec (270h) • …
57. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • ̍ຕ 100kb • 10ຕ 1000kb • 1000ຕ 100MB • 1ສຕ 1GB • 100ສຕ 100GB • 1000ສຕ 1TB • …
58. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • goroutineͰ΋ݶք͕… • ڧ͍ΠϯελϯεͰ࡞Δ͜ͱ΋ߟ͕͑ͨ • ౎౓ launch > deploy > shutdown …
59. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ
60. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • Google Cloud Functionͱ͸ • ΠϕϯτۦಈܕͷαʔόʔϨε 
 ίϯϐϡʔςΟϯά ϓϥοτϑΥʔϜ • FaaS • AWS Lambda
61. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ ?code=abcd123
 ?code=efgh456
 ?code=abab1212
62. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ func GenCode(w http.ResponseWriter, r *http.Request) { v := r.URL.Query() code := v.Get("code") img, err := genCode(code, baseImage) if err != nil { return } buf := &bytes.Buffer{} if err := png.Encode(buf, img); err != nil { return } if err := upload(r.Context(), buf, filename); err != nil { return } fmt.Fprintf(w, "OK") w.WriteHeader(http.StatusOK) }
63. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ $ zip -r function.zip function.go some.png vendor $ gcloud beta functions deploy \ --project=${PROJECT_ID} \ --region=${REGION} \ --memory=${MEMORY} \ --runtime=${RUNTIME} \ --service-account=${SERVICE_ACCOUNT} \ --source=gs://${UPLOAD_BUCKET_PATH}/${FUNCTION}.zip \ ${FUNCTION} \ --entry-point=${FUNCTION} \ —trigger-http
64. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ • func (w http.Writer, r *http.Request) Λ
 ΄Μͱͦͷ··ΞοϓϩʔυͰ͖Δ • ґଘpackage΋ҰॹʹzipʹؚΊΕ͹࢖༻Մೳ • assetϑΝΠϧͳͲ΋ಉࠝՄೳ • zipϑΝΠϧ͓Αͼղౚͨ͠ͱ͖ͷ༰ྔ੍ݶ༗
65. GoͰେྔͷϑΝΠϧΛੜ੒͢Δ ?code=abcd123
 ?code=efgh456
 ?code=abab1212
66. ·ͱΊ • ӡ༻Ͱൃੜ͢ΔλεΫΛ
 ࣗಈԽ/ޮ཰Խ͢Δաఔͷࢼߦࡨޡͨ͠࿩ • GoΛ࢖ͬͯExcelΛѻ͏ • GoͰGoogleDriveΛѻ͏ • GoͰେྔͷϑΝΠϧΛੜ੒͢Δ