Upgrade to Pro — share decks privately, control downloads, hide ads and more …

トルテリリースまでの Go Tips 16/torte-go-tips-16

Take
September 06, 2017

トルテリリースまでの Go Tips 16/torte-go-tips-16

Take

September 06, 2017
Tweet

More Decks by Take

Other Decks in Technology

Transcript

  1. • Go on Docker 1. timezone 2. static build •

    Upload Images 3. Exif আڈ 4. Orientation ໰୊ 5. ύϑΥʔϚϯε • Terminate 6. Signal ϋϯυϦϯά 7. context ͱ cancel ͱ goroutine GO TIPS 16 • JSON 8. json.Marshaler 9. struct embed 10. omitempty • Credentials 11. ؀ڥม਺ 12. IAMೝূ • Logging 13. zap 14. access log • Test & Error handling 15. gomock 16. errors
  2. • time.LoadLocation Go on Docker // LoadLocation returns the Location

    with the given name. // // If the name is "" or "UTC", LoadLocation returns UTC. // If the name is "Local", LoadLocation returns Local. // // Otherwise, the name is taken to be a location name corresponding to a file // in the IANA Time Zone database, such as "America/New_York". // // The time zone database needed by LoadLocation may not be // present on all systems, especially non-Unix systems. // LoadLocation looks in the directory or uncompressed zip file // named by the ZONEINFO environment variable, if any, then looks in // known installation locations on Unix systems, // and finally looks in $GOROOT/lib/time/zoneinfo.zip. func LoadLocation(name string) (*Location, error) {
  3. • static build
 Go on Docker CGO_ENABLED=0 go build -a

    -tags netgo \
 -installsuffix netgo -ldflags '-extldflags "-static"' ढจԽʜ
  4. • JPEG decode → encode
 ͢Δ͚ͩͰ Exif ͸औΓআ͚Δ͕… • Orientation

    (࣮ࡍͷը૾Λදࣔ͢Δํ޲)
 ͜Ε͕ܽଛ͢Δͱճసͨ͠Γ൓సͨ͠ঢ়ଶ Ͱදࣔ͞Εͯ͠·͏ Upload Images
  5. • Go Ͱ Exif Λ parse ͢Δํ๏ • https://github.com/golang/go/issues/4341
 ͔ͳΓલ͔Βٞ࿦͞Ε͍ͯΔ͕ਐḿ͕ͳ͍

    • ౰෼͸αʔυύʔςΟ੡ͷϥΠϒϥϦΛ
 ࢖͏͔͠ແ͍ (or ࣗ࡞) Upload Images
  6. Upload Images IUUQXXXDJQBKQTUEEPDVNFOUTK%$+QEG ▪ ը૾ํ޲ Orientation ߦͱྻͷ؍఺͔Βݟͨɺը૾ͷํ޲ɻ Tag = 274

    (112.H) Type = SHORT Count = 1 Default = 1 1 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷ্(visual top)ɺ0 ൪໨ͷྻ͕ࠨଆ(visual left-hand side)ͱͳΔɻ 2 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷ্ɺ0 ൪໨ͷྻ͕ӈଆ(visual right-hand side)ͱͳΔɻ 3 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷԼ(visual bottom)ɺ0 ൪໨ͷྻ͕ӈଆͱͳΔɻ 4 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷԼɺ0 ൪໨ͷྻ͕ࠨଆͱͳΔɻ 5 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪໨ͷྻ্͕ͱͳΔɻ 6 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪໨ͷྻ্͕ͱͳΔɻ 7 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪໨ͷྻ͕ԼͱͳΔɻ 8 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪໨ͷྻ͕ԼͱͳΔɻ ͦͷଞ = ༧໿ 1ΑΓҾ༻ • Exif ͷن֨ॻΛಡ΋͏
  7. • Exif ͷن֨ॻΛಡ΋͏ ▪ ը૾ํ޲ Orientation ߦͱྻͷ؍఺͔Βݟͨɺը૾ͷํ޲ɻ Tag = 274

    (112.H) Type = SHORT Count = 1 Default = 1 1 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷ্(visual top)ɺ0 ൪໨ͷྻ͕ࠨଆ(visual left-hand side)ͱͳΔɻ 2 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷ্ɺ0 ൪໨ͷྻ͕ӈଆ(visual right-hand side)ͱͳΔɻ 3 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷԼ(visual bottom)ɺ0 ൪໨ͷྻ͕ӈଆͱͳΔɻ 4 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷԼɺ0 ൪໨ͷྻ͕ࠨଆͱͳΔɻ 5 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪໨ͷྻ্͕ͱͳΔɻ 6 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪໨ͷྻ্͕ͱͳΔɻ 7 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷӈଆɺ0 ൪໨ͷྻ͕ԼͱͳΔɻ 8 = 0 ൪໨ͷߦ͕໨Ͱݟͨͱ͖ͷը૾ͷࠨଆɺ0 ൪໨ͷྻ͕ԼͱͳΔɻ ͦͷଞ = ༧໿ Upload Images IUUQXXXDJQBKQTUEEPDVNFOUTK%$+QEG 1ΑΓҾ༻ ೔ຊޠͳͷͰಡΈ΍͍͢ʂʂ
 ࣮૷͍ͨ͠ͱ͸ݴͬͯͳ͍
  8. • αʔυύʔςΟ੡ϥΠϒϥϦɺπʔϧ • ImageMagick • GraphicsMagick • GD Graphics Library

    (libgd) • libvips • Pure Go • github.com/BurntSushi/graphics-go • github.com/anthonynsimon/bild • github.com/disintegration/imaging • github.com/disintegration/gift Upload Images
  9. • ϕϯνϚʔΫ݁Ռ Upload Images BenchmarkGraphicsgo_LargeImage-8 1 4686575195 ns/op BenchmarkBild_LargeImage-8 3

    460527153 ns/op BenchmarkImaging_LargeImage-8 10 115713079 ns/op BenchmarkGift_LargeImage-8 5 237302274 ns/op BenchmarkGraphicsgo-8 20 88962194 ns/op BenchmarkBild-8 200 7481979 ns/op BenchmarkImaging-8 500 2507617 ns/op BenchmarkGift-8 300 4335670 ns/op ࣌ܭճΓʹ౓ճసͯ͠ࠨӈ൓స͢Δͱ͍͏ॲཧΛɺ
 Ұ؟ͰࡱӨ͞Εͨେ͖͍ࣸਅ .#௒͑ ͱɺ
 খ͍͞ը૾ ,#ఔ౓ ͷ௨ΓͰࢼߦɻ
  10. • ϕϯνϚʔΫ݁Ռ Upload Images BenchmarkGraphicsgo_LargeImage-8 1 4686575195 ns/op BenchmarkBild_LargeImage-8 3

    460527153 ns/op BenchmarkImaging_LargeImage-8 10 115713079 ns/op BenchmarkGift_LargeImage-8 5 237302274 ns/op BenchmarkGraphicsgo-8 20 88962194 ns/op BenchmarkBild-8 200 7481979 ns/op BenchmarkImaging-8 500 2507617 ns/op BenchmarkGift-8 300 4335670 ns/op ࣌ܭճΓʹ౓ճసͯ͠ࠨӈ൓స͢Δͱ͍͏ॲཧΛɺ
 Ұ؟ͰࡱӨ͞Εͨେ͖͍ࣸਅ .#௒͑ ͱɺ
 খ͍͞ը૾ ,#ఔ౓ ͷ௨ΓͰࢼߦɻ github.com/disintegration/imaging
 ࠾༻
  11. Upload Images func applyOrientation(img image.Image, meta *exif.Exif) (image.Image, error) {

    tag, err := meta.Get(exif.Orientation) if err != nil { return nil, errors.WithStack(err) } orientation, err := tag.Int(0) if err != nil { return nil, errors.WithStack(err) } switch orientation { case 1: return img, nil case 2: img = imaging.FlipH(img) case 3: img = imaging.FlipH(img) img = imaging.FlipV(img) case 4: img = imaging.FlipV(img) case 5: img = imaging.Rotate270(img) img = imaging.FlipH(img) case 6: img = imaging.Rotate270(img) case 7: img = imaging.Rotate90(img) img = imaging.FlipH(img) case 8: img = imaging.Rotate90(img) default: return nil, errors.New("invalid orientation") } }
  12. • Signal ͕ඈΜͰ͖ͨ࣌ʹऴྃॲཧΛߦ͏ Terminate func terminateHandler(cancel func()) { sigCh :=

    make(chan os.Signal) signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGINT, syscall.SIGQUIT, ) go func() { sig := <-sigCh log.Printf("Terminated by SIGNAL: %v\n", sig) cancel() }() } ˞syscall.SIGKILL͸ϋϯυϦϯάͰ͖ͳ͍
  13. Terminate func main() { ctx, cancel := context.WithCancel(context.Background()) terminateHandler(cancel) }

    func exec(ctx context.Context) { ctx, cancel := context.WithTimeout(ctx, execTimeout) defer cancel() ch := make(chan *Record, channelBufferSize) go fetchRecords(ctx, ch) for { select { case <-ctx.Done(): return case record := <-ch: // 1Ϩίʔυͣͭߦ͏ԿΒ͔ͷॲཧ } } }
  14. Terminate func fetchRecords(ctx context.Context, ch chan<- *Record) (err error) {

    defer func() { if cause := recover(); cause != nil { if e, ok := cause.(error); ok { err = e } else { // Τϥʔϩάͱ͔ } } }() // ࣮ࡍʹϨίʔυΛऔͬͯ͘Δॲཧ }
  15. • json.Marshaler Λ࣮૷ • MarshalJSON ͷதͰ json.Marshal ΛݺͿ ͱ stack

    overflow ͢Δ࣌ͷରॲ๏ JSON  IUUQTQMBZHPMBOHPSHQ6X"&-R
  16. JSON  IUUQTQMBZHPMBOHPSHQ6X"&-R • json.Marshaler Λ࣮૷ͨ͠৔߹ʹ
 stack overflow ͢Δ࣌ͷରॲ๏ package

    main import ( "encoding/json" "fmt" ) type Obj struct { Text string } func (o *Obj) MarshalJSON() ([]byte, error) { // ԿΒ͔ͷॲཧ type alias Obj return json.Marshal((*alias)(o)) } func main() { obj := &Obj{ Text: "json.Marshaler", } d, _ := json.Marshal(obj) fmt.Println(string(d)) }
  17. JSON  IUUQTQMBZHPMBOHPSHQ6X"&-R • json.Marshaler Λ࣮૷ͨ͠৔߹ʹ
 stack overflow ͢Δ࣌ͷରॲ๏ package

    main import ( "encoding/json" "fmt" ) type Obj struct { Text string } func (o *Obj) MarshalJSON() ([]byte, error) { // ԿΒ͔ͷॲཧ type alias Obj return json.Marshal((*alias)(o)) } func main() { obj := &Obj{ Text: "json.Marshaler", } d, _ := json.Marshal(obj) fmt.Println(string(d)) }
  18. • struct ͷຒΊࠐΈ • struct in struct ͸ flat •

    interface in struct ͸ nest IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' JSON
  19. • struct ͷຒΊࠐΈ • struct in struct ͸ flat •

    interface in struct ͸ nest package main import ( "encoding/json" "fmt" ) type Interface interface{} type OuterA struct { Interface Property string } type InnerA struct { InnerProperty string } func main() { objA := &OuterA{ Interface: &InnerA{InnerProperty: "inner"}, Property: "outer", } d, _ := json.Marshal(objA) fmt.Println(string(d)) } IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' JSON
  20. • struct in struct ͸ flatten ͞ΕΔ • interface in

    struct ͸ nest ͞ΕΔ • ͑͑ͬʂʁ package main type Interface interface{} type OuterA struct { Interface Property string } type InnerA struct { InnerProperty string } func main() { objA := &OuterA{ Interface: &InnerA{InnerProperty: "inner"}, Property: "outer", } d, _ := json.Marshal(objA) fmt.Println(string(d)) } IUUQTQMBZHPMBOHPSHQ[Q'(*XS(X' { "Interface": { "InnerProperty": "inner" }, "Property": "outer" } JSON ͑͑ͬʂʁ
  21. • EC2 System Manager Parameter Store
 → AWS ͳΒ͜ΕҰ୒ʁ
 →

    KMS ͷ伴Ͱ҉߸Խͯ͘͠ΕΔ • ଞͳΒ etcd, consul ͱ͍͏બ୒ࢶ΋ • github.com/vrischmann/envconfig Λ wrap ͯ͠ೝূ৘ใ͚ͩ Parameter Store ࢀর Credentials 
  22. • ೝূϦΫΤετ • Amazon Elasticsearch Service
 ͳͲΛ࢖͏࣌ʹ Signing AWS API

    Request ʹ ै͏ඞཁ͕͋Δ • ରԠ͍ͯ͠ͳ͍αʔυύʔςΟͷΫϥΠΞϯτ Λͦͷ··࢖͏ͳΒ http.RoundTripper Λ
 ࣮૷͢Δ Credentials 
  23. Credentials  // Transport implement http.RoundTripper. type Transport struct {

    Base http.RoundTripper } func (t *Transport) base() http.RoundTripper { if t.Base != nil { return t.Base } return http.DefaultTransport } // RoundTrip wraps http.DefaultTransport.RoundTrip for AWS signed request. func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error) { // clone request body var body io.ReadSeeker buff := &bytes.Buffer{} if req.Body != nil { buff.ReadFrom(req.Body) req.Body = ioutil.NopCloser(buff) body = bytes.NewReader(buff.Bytes()) } signer := v4.NewSigner(credentials.AWSSession.Config.Credentials) _, err := signer.Sign(req, body, awsServiceName, awsRegion, time.Now()) if err != nil { return nil, err } return t.base().RoundTrip(req) }
  24. • zap • JSON ܗࣜͷϩά • ͍Ζ͍ΖͭΒ͔ͬͨ json package Λ࢖͏

    ඞཁ͕ͳ͍ • ύϑΥʔϚϯεʹ༏Ε͍ͯΔ Logging  HJUIVCDPNVCFSHP[BQ
  25. Logging  logger, _ := zap.NewProduction() defer logger.Sync() logger.Info("failed to

    fetch URL", zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), )
  26. • access/app log • RequestID Λੜ੒ͯ͠෇༩͢Δ • ϨεϙϯεͰ X-Request-ID ϔομΛฦ͢

    • 1ͭͷϦΫΤετͷྲྀΕ͕௥͑ΔͷͰɺ
 σόοάָ͕ʹʂ Logging 
  27. • gomock • interface Λݩʹ mock Λੜ੒ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢Δ΋ͷΛ
 mock

    Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚ਺ͳͲ • ςετͰ༗༻ Test & Error handling  HJUIVCDPNHPMBOHNPDL
  28. • gomock • interface Λݩʹ mock Λੜ੒ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢Δ΋ͷΛ
 mock

    Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚ਺ͳͲ • ςετͰ༗༻ Test & Error handling  HJUIVCDPNHPMBOHNPDL package s3 import "io" type Client interface { Upload(bucket string, key string, file io.ReadSeeker) error } type implement struct{} func (s *implement) Upload(bucket string, key string, file io.ReadSeeker) error { // implement return nil } func New() Client { return &implement{} }
  29. • gomock • interface Λݩʹ mock Λੜ੒ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢Δ΋ͷΛ
 mock

    Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚ਺ͳͲ • ςετͰ༗༻ Test & Error handling  HJUIVCDPNHPMBOHNPDL package main import // தུ const bucketName = "{bucket_name}" var s3Client = s3.New() func UploadImage(fileName string, file io.ReadSeeker) error { fileHeader := make([]byte, 512) file.Read(fileHeader) mimeType := http.DetectContentType(fileHeader) _, err := file.Seek(0, io.SeekStart) if err != nil { return err } var ext string switch mimeType { case "image/png": ext = ".png" case "image/jpeg": ext = ".jpg" default: return errors.New("unsupported file type") } return s3Client.Upload(bucketName, fileName+ext, file) }
  30. • gomock • interface Λݩʹ mock Λੜ੒ • ಛʹ࣍ͷΑ͏ͳ֎෦ͷঢ়ଶʹґଘ͢Δ΋ͷΛ
 mock

    Խ͢Δͱศར • σʔλετΞपΓ • ֎෦ API पΓ • ཚ਺ͳͲ • ςετͰ༗༻ Test & Error handling  HJUIVCDPNHPMBOHNPDL package main import ( "os" "path/filepath" "testing" // தུ "github.com/golang/mock/gomock" ) func TestUploadImage(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() mockS3Client := s3.NewMockClient(ctrl) mockS3Client.EXPECT().Upload( bucketName, "test.jpg", gomock.Any(), ).Return(nil) s3Client = mockS3Client filePath, _ := filepath.Abs("testdata/test.jpg") file, _ := os.Open(filePath) err := UploadImage("test", file) if err != nil { t.Error(err) } }
  31. • Error handling • ඪ४ͷ errors package ͷ୅ΘΓʹ࢖͏ • ಛʹ

    WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ෇༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕෼͔ΓσόοάḿΔ Test & Error handling  HJUIVCDPNQLHFSSPST
  32. • Error handling • ඪ४ͷ errors package ͷ୅ΘΓʹ࢖͏ • ಛʹ

    WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ෇༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕෼͔ΓσόοάḿΔ Test & Error handling  HJUIVCDPNQLHFSSPST package main import ( "strconv" "github.com/pkg/errors" ) func main() { } func errorMethod() error { i, err := strconv.Atoi("") if err != nil { return errors.WithStack(err) } return nil }
  33. • Error handling • ඪ४ͷ errors package ͷ୅ΘΓʹ࢖͏ • ಛʹ

    WithStack ͱ͍͏ϝιου͕ศར • ඪ४ͷ error Λड͚औΔՕॴͰ WithStack Ͱ stack trace Λ෇༩͢Δ͜ͱͰϩά͔ΒΤϥʔ ൃੜՕॴ͕෼͔ΓσόοάḿΔ Test & Error handling  HJUIVCDPNQLHFSSPST package main import ( "strconv" "github.com/pkg/errors" ) func main() { } func errorMethod() error { i, err := strconv.Atoi("") if err != nil { return errors.WithStack(err) } return nil } TUSDPOW"UPJQBSTJOHJOWBMJETZOUBY NBJOFSSPS.FUIPE 6TFSTXXHPTSDFSSPSTNBJOHP NBJONBJO 6TFSTXXHPTSDFSSPSTNBJOHP SVOUJNFNBJO VTSMPDBM$FMMBSHPMJCFYFDTSDSVOUJNFQSPDHP SVOUJNFHPFYJU VTSMPDBM$FMMBSHPMJCFYFDTSDSVOUJNFBTN@BNET
  34. ͜͜·Ͱొ৔͠·ͤΜͰ͕ͨ͠ɺ • ϑϨʔϜϫʔΫ • echo (github.com/labstack/echo) • package ؅ཧ •

    glide (github.com/Masterminds/glide) • ORM • xorm (github.com/go-xorm/xorm) ࢖ͬͯ·͢ɻ ·ͱΊ