GoCon 2019 で発表した 煩雑な運用を Goを使って楽にする の発表資料になります
ࡶͳӡ༻Λ GoΛָͬͯʹ͢ΔGoCon2019 Spring / 2019.05.18
View Slide
About me• ਧྒྷհ a.k.a Konboi• גࣜձࣾmerpay• Coupon Team• Backend / Tech Lead
About Talk• ӡ༻Ͱൃੜ͢ΔλεΫΛ ࣗಈԽ/ޮԽ͢ΔաఔͰͷࢼߦࡨޡͨ͠• αʔόʔ/ΠϯϑϥدΓͷ͋·Γग़·ͤΜ• ຯͰ͢
Agenda• GoΛͬͯExcelΛѻ͏• GoͰGoogleDriveΛѻ͏• GoͰେྔͷϑΝΠϧΛੜ͢Δ
͡Ίʹ• ݱ࣌ͰΫʔϙϯσʔλExcelΛͬͯೖߘ• ઐ༻ͷೖߘγεςϜ࡞Δͷίετେ• Θ͔Γ͍͢UI• ηΩϡϦςΟ• దͳݖݶཧ• γεςϜͷӡ༻• etc• ϛχϚϜͰݕূͯ͠Ձ͕͋ΔͳΒ͖ͪΜͱඋ
͡Ίʹ• ΒͬͨExcel͔ΒखͰσʔλΛ͍͡Δͱ…• ώϡʔϚϯΤϥʔ• ҙਤ͠ͳ͍σʔλͷࠞೖ• ՄೳͳݶΓਓʹΑΔૢ࡞ΛݮΒ͍ͨ͠
͡Ίʹ• Execlڞ༗ͷॴͰཧ͍ͨ͠• ʓʓ͞ΜͷPCʹ͔͠ͳ͍• όʔδϣϯཧ͍ͨ͠• 2019/05/18 2.xlsx
͡Ίʹ• Google DriveͰཧ• G Suite• όʔδϣϯΛཧ
࣮ݱ͍ͨ͜͠ͱ
YAML࣮ݱ͍ͨ͜͠ͱ
GoΛͬͯExcelΛѻ͏
GoΛͬͯExcelΛѻ͏• దʹσʔλΛநग़͍ͨ͠• దͳϑΥʔϚοτͰѻ͍͍ͨ• Project͕GoͳͷͰGoͰExcelΛѻ͍͍ͨ
GoΛͬͯExcelΛѻ͏• ExcelॊೈʹϑΥʔϚοτΛมߋͰ͖Δ• ॊೈͳΏ͑ͷ՝• ηϧ݁߹• ؔʹΑΔೖྗ• ॻࣜ• etc
ௐࠪ/ݕূ
ௐࠪ/ݕূ• ࣮ӡ༻ͷલʹௐࠪ/ݕূΛ࣮ࢪ• ͍͔ͭ͘ͷϥΠϒϥϦΛ༻ͯ͠ ߦͬͨݕূͷҰ෦Λհ͠·͢
ௐࠪ/ݕূ• ηϧ݁߹
ௐࠪ/ݕূ• ηϧ݁߹• ࠨ্ͷηϧͷΈσʔλ͕ೖͬͨঢ়ଶʹ$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 06:53,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 06:53,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 06:53,B,
ௐࠪ/ݕূ• ؔ
ௐࠪ/ݕূ• ؔ• දࣔ͞Ε͍ͯΔσʔλ͕औಘͰ͖Δ$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 06:53,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 06:53,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 06:53,B,
ௐࠪ/ݕূ• ؔ• ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ$ date "+%Y/%m/%d %H:%M:%S"2019/05/18 06:58:14 $ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 06:53,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 06:53,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 06:53,B,
ௐࠪ/ݕূ• ೖྗܗࣜ• Ex)Ϧετ
ௐࠪ/ݕূ• ೖྗܗࣜ - Ϧετ• ݱࡏηϧʹೖྗ͞Ε͍ͯΔσʔλΛऔಘ$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 06:53,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 06:53,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 06:53,B,
ௐࠪ/ݕূ• ॻࣜઃఆ• Ex)
ௐࠪ/ݕূ• ॻࣜઃఆ• ϑΥʔϚοτ͕͓͔͍͠$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 06:53,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 06:53,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 06:53,B,
ௐࠪ/ݕূ• ॻࣜઃఆ• Excel෦ͰγϦΞϧͱͯ͠อଘ• ͕• গ͕࣌ࠁ• ϥΠϒϥϦʹΑͬͯγϦΞϧ͕ ͦͷ··ग़ྗ͞ΕΔͷ
ௐࠪ/ݕূ• ॻࣜઃఆ• Excel෦ͰγϦΞϧͱͯ͠อଘ
ௐࠪ/ݕূ• ॻࣜઃఆ• ςΩετ
ௐࠪ/ݕূ• ॻࣜઃఆ• ςΩετ$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,1989/01/08,ఱߖ,ฏ:heisei,5/18/19 07:06,A,ฏ,heisei,2019/04/30,ఱߖ,ฏ:heisei,5/18/19 07:06,,ྩ,reiwa,2019/05/01,্ߖ,ྩ:reiwa,5/18/19 07:06,B,
ௐࠪ/ݕূ ݁Ռ• ͔͍ͬͳͷηϧ݁߹ͱॻࣜઃఆ• ͦΕҎ֎ҙ֎ͱͳͦ͞͏
ௐࠪ/ݕূ ݁Ռ• ͯ͢จࣈྻͰΕͳ͍• ͳ͍…
ௐࠪ/ݕূ ݁Ռ• ͯ͢จࣈྻͩͱ…• શ֯/֯ͷѻ͍Λߟྀ• ʹͪͳͲΛϑΥʔϚοτ͕͋Δͷ ॻࣜઃఆΛઃఆ͠దͳΛೖྗ/औಘ͍ͨ͠
ௐࠪ/ݕূ ݁Ռ• 360EntSecGroup-Skylar/excelize Λ࠾༻• SetCellStyle method• ॻࣜઃఆΛઃఆͰ͖Δ
• Ex) 14: m/d/yy• Ex) 15: d-mmm-yyௐࠪ/ݕূ ݁Ռ$ go run main.go -f ../example.xlsxname,name_en,date,status,id,created_at,category,ฏ,heisei,8-Jan-89,ఱߖ,ฏ:heisei,5/18/19 07:09,A,ฏ,heisei,30-Apr-19,ఱߖ,ฏ:heisei,5/18/19 07:09,,ྩ,reiwa,1-May-19,্ߖ,ྩ:reiwa,5/18/19 07:09,B,$ go run main.go -f ../example.xlsxฏ,heisei,01-08-89,ఱߖ,ฏ:heisei,5/18/19 07:09,A,ฏ,heisei,04-30-19,ఱߖ,ฏ:heisei,5/18/19 07:09,,ྩ,reiwa,05-01-19,্ߖ,ྩ:reiwa,5/18/19 07:09,B,
ௐࠪ/ݕূ ݁Ռ• ଞʹExcelΛGo͔Βૢ࡞Մೳ• RemoveRow()• SaveAs()• ExcelʹσʔλΛه/मਖ਼ͯ͠อଘ͕Մೳ• ೖߘεςʔλεͷཧ͕ExcelͰՄೳʹ
ௐࠪ/ݕূ ݁Ռ• จࣈྻ/୯७ͳσʔλͰ͋Ε ͲͷϥΠϒϥϦͰେମఆ௨Γʹʹಈ࡞• ༻్ʹԠͨ͡ϥΠϒϥϦΛ͏ͷ͕Α͍͔ͱ
GoΛͬͯ Google DriveΛѻ͏
GoͰGoogleDriveΛѻ͏• ެࣜͷSDK• google.golang.org/api/drive/v3• جຊެࣜυΩϡϝϯτΛݟΕOK• https://developers.google.com/drive/api/v3/reference/• ݸਓతʹቕͬͨͱ͜ΖΛհ
ϑΝΠϧϦετFieldΛࢦఆ• ࢦఆ͠ͳ͍ͱid,name, kind, mimeTypeͷΈ• ˎࢦఆ͢Δͱͯ͢ͷσʔλ͕ ฦͬͯ͘Δ͔ύϑΥʔϚϯε͕ྼԽ•srv.Files.List().Fields(“files(parents, id, name, owners, kind, mimeType)").Do() srv.Files.List().Fields(“*”).Do()
Update͢Δͱ͖ͷtips• List APIܦ༝ͰFile ObjectΛऔಘ• ಛఆͷFieldΛߋ৽ͯ͠Update͢Δͱlist, err := service.Files.List().Context(ctx).Do()if err != nil {return err} file := list[0] file.Name = “dummy” _, err := service.Files.Update(file.Id, file).Context(ctx).Do()if err != nil {return errors.Wrap(err, "error updated")} > The resource body includes fields which are not directly writable.
Update͢Δͱ͖ͷtips• APIͰߋ৽Ͱ͖ͳ͍Fieldߋ৽͕͔͔Δ• ߋ৽͍ͨ͠FiledΛࢦఆlist, err := service.Files.List().Context(ctx).Do()if err != nil {return err} file := list[0] _, err := service.Files.Update(file.Id, &drive.File{Name: “dummy”,}).Context(ctx).Do() if err != nil {return errors.Wrap(err, "error updated")}
Update͢Δͱ͖ͷtips• ϑΝΠϧͷதΛߋ৽• Media(io.Reader, Content-Type) Λ༻list, err := service.Files.List().Context(ctx).Do()if err != nil {return err} file := list[0] content := &bytes.Buffer{}content.WriteString(“hello”) updatedFile, err := service.Files.Update(file.Id, &drive.File{}).Media(content, googleapi.ContentType(file.MimeType)).Context(ctx).Do()
Update͢Δͱ͖ͷtips• όʔδϣϯཧ͞ΕΔͷͰ ҆৺(?)࣮ͯ͠ߦՄೳ
࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍• v2ͩͱ Children: list ͱ͍͏API͕͋Δ• FileTypeΛνΣοΫͯࣗ͠લͰ ࣮͢Δඞཁ͕͋Δ• .Q(“xxx in parents”)• ࢠؔΛࢦఆ
࠶ؼతʹϑΝΠϧΛऔಘͰ͖ͳ͍• ৄ͘͠ Google Drive API v3 Ͱݕࡧ• sample: Konboi/go-google-drive-example list, err := service.Files.List().Q(fmt.Sprintf("'%s' in parents", driveID)).Context(ctx).Do()if err != nil {return errors.Wrap(err, "list error")} typeFolder := "application/vnd.google-apps.folder"for _, f := range list.Files {pp.Printf("file id:%s\nname%s\ntype:%v\n", f.Id, f.Name, f.MimeType)if f.MimeType == typeFolder {err := ListRecursive(ctx, service, f.Id)if err != nil {return err}}}
GoͰେྔͷϑΝΠϧΛ ੜ͢Δ
GoͰେྔͷϑΝΠϧΛੜ͢Δ
GoͰେྔͷϑΝΠϧΛੜ͢Δ• ΫʔϙϯʹΑ֤ͬͯϢʔβϢχʔΫͳ ίʔυΛൃߦ͢Δඞཁ͕͋Δ• ༧Ί࣮ࢪళฮ͔ΒίʔυΛൃߦ• ݱঢ়ͷγεςϜͷ߹্ɺࣄલʹը૾Λ ੜ͢͠ΔΈ• ࣄલʹେྔͷը૾Λੜ͢Δඞཁ͕͋Δ
গྔͳΒͳ͍͕
GoͰେྔͷϑΝΠϧΛੜ͢Δ• ̍ຕ 0.1sec• 10ຕ 1sec• 1000ຕ 100sec• 1ສຕ 1000sec (2.5h)• 100ສຕ 100000sec (27h)• 1000ສຕ 1000000sec (270h)• …
GoͰେྔͷϑΝΠϧΛੜ͢Δ• ̍ຕ 100kb• 10ຕ 1000kb• 1000ຕ 100MB• 1ສຕ 1GB• 100ສຕ 100GB• 1000ສຕ 1TB• …
GoͰେྔͷϑΝΠϧΛੜ͢Δ• goroutineͰݶք͕…• ڧ͍ΠϯελϯεͰ࡞Δ͜ͱߟ͕͑ͨ• launch > deploy > shutdown …
GoͰେྔͷϑΝΠϧΛੜ͢Δ• Google Cloud Functionͱ• ΠϕϯτۦಈܕͷαʔόʔϨε ίϯϐϡʔςΟϯά ϓϥοτϑΥʔϜ• FaaS• AWS Lambda
GoͰେྔͷϑΝΠϧΛੜ͢Δ?code=abcd123 ?code=efgh456 ?code=abab1212
GoͰେྔͷϑΝΠϧΛੜ͢Δfunc GenCode(w http.ResponseWriter, r *http.Request) {v := r.URL.Query()code := v.Get("code")img, err := genCode(code, baseImage)if err != nil {return}buf := &bytes.Buffer{}if err := png.Encode(buf, img); err != nil {return}if err := upload(r.Context(), buf, filename); err != nil {return}fmt.Fprintf(w, "OK")w.WriteHeader(http.StatusOK)}
GoͰେྔͷϑΝΠϧΛੜ͢Δ$ zip -r function.zip function.go some.png vendor$ gcloud beta functions deploy \--project=${PROJECT_ID} \--region=${REGION} \--memory=${MEMORY} \--runtime=${RUNTIME} \--service-account=${SERVICE_ACCOUNT} \--source=gs://${UPLOAD_BUCKET_PATH}/${FUNCTION}.zip \${FUNCTION} \--entry-point=${FUNCTION} \—trigger-http
GoͰେྔͷϑΝΠϧΛੜ͢Δ• func (w http.Writer, r *http.Request) Λ ΄Μͱͦͷ··ΞοϓϩʔυͰ͖Δ• ґଘpackageҰॹʹzipʹؚΊΕ༻Մೳ• assetϑΝΠϧͳͲಉࠝՄೳ• zipϑΝΠϧ͓Αͼղౚͨ͠ͱ͖ͷ༰ྔ੍ݶ༗
·ͱΊ• ӡ༻Ͱൃੜ͢ΔλεΫΛ ࣗಈԽ/ޮԽ͢Δաఔͷࢼߦࡨޡͨ͠• GoΛͬͯExcelΛѻ͏• GoͰGoogleDriveΛѻ͏• GoͰେྔͷϑΝΠϧΛੜ͢Δ
konboi
coe
.jpg){kind=avatar}
oshanqq
shomaekawa
andpad
sugawani
datarian
toshi0383
okamototakuyasr2
chloe463
hokaccha
wakamsha
rynsuke
geeforr
chrisshort
gr2m
jacobian
jakevdp
wjessup
kevinliebholz
pedronauck
jasonvnalue
keithpitt
sferik
sugarenia