Upgrade to Pro — share decks privately, control downloads, hide ads and more …

go.mercari.io/datastore はいいぞ! / go.mercari.io/d...

go.mercari.io/datastore はいいぞ! / go.mercari.io/datastore is pretty good!

go.mercari.io/datastore の紹介です。
https://gaeja.connpass.com/event/90133/

Masahiro Wakame

June 14, 2018
Tweet

More Decks by Masahiro Wakame

Other Decks in Technology

Transcript

  1. ͦΕԿʁ • (AppEngine|Cloud) Datastore Wrapper • In Go! Seamless! •

    Middleware • Retry, Caching, Logging… etc • Batch operation • Single-Op Batch-Op aggregation
  2. Datastore Storage AppEngine RPC Cloud RPC google.golang.org/ appengine/datastore cloud.google.com/ go/datastore

    GAE Instance Any Server RPC Endpoint Package Runtime Env ଘࡏ͢Δػೳʹ ݁ߏ͕ࠩ͋Δ ৽ چ
  3. flatten? • datastore.Entity ʹ͍ͭͯ • AE → Entity͸Entity࣋ͯͳ͍ • Cloud

    → Entity͸Entity࣋ͯΔ • struct͸Entityʹม׵͞ΕΔ • nest ͨ͠struct͸EntityͷೖΕࢠʹͳΔ
  4. flatten? type AEEntity struct { "Slice.A" []string "Slice.B" []string }

    ͜ͷίʔυྫ͸ ΠϝʔδͰ͢ type Inner struct { A string B string } type Data struct { Slice []Inner } type CloudEntity struct { Slice []struct{ A string B string } }
  5. flatten? type Inner struct { A string B string }

    type Data struct { Slice []Inner } type Data struct { Slice []Inner ↩ `datastore:",flatten"` } flattenͰAEͱಉ͡ʹ
  6. ☠ ATTENTION ☠ • Cloud DatastoreͰNestͨ͠Entity࡞Δͱ AppEngine Datastore͔ΒಡΊͳ͍ • ͸ͣ(ࢼͯ͠ͳ͍)

    • ؾܰͳؾ࣋ͪͰAppEngine͔ΒGetͯ͠ CloudʹPut͢Δͱࢮ͵Մೳੑ͕͋Δ ςετ͸͔ͬ͠ΓͶʂ
  7. ࠓ͋ΔMiddleware • AE Memcache • In-Memory Cache • Automatic RPC

    Retry • Redis(Memorystore) Cache • Logger, Chaos RPC, Fishbone … • ↑ओʹςετ΍ݕূ༻ ຊ൪ར༻࣮੷ͷน
  8. ͜͏͍͏ίʔυ͕ॻ͚Δ type Comment struct { Message string } type Post

    struct { Content string CommentIDs []int64 `json:"-"` Comments []*Comment `datastore:"-"` } // Entityͷ४උ(ׂѪ) ……… // start fetching... posts := make([]*Post, 0) _, err = client.GetAll(ctx, client.NewQuery("Post").Order("Content"), &posts) if err != nil { panic(err) } // Let's batch get! bt := client.Batch() for _, post := range posts { comments := make([]*Comment, 0) for _, id := range post.CommentIDs { comment := &Comment{} bt.Get(client.IDKey("Comment", id, nil), comment, nil) comments = append(comments, comment) } post.Comments = comments } err = bt.Exec(ctx) if err != nil { panic(err) }
  9. ΋͠΋Batch͕ͳ͔ͬͨΒ // start fetching... posts := make([]*Post, 0) _, err

    = client.GetAll(ctx, client.NewQuery("Post").Order("Content"), &posts) if err != nil { panic(err) } // Let's batch get! var commentKeys []datastore.Key for _, post := range posts { for _, commentID := range post.CommentIDs { commentKeys = append(commentKeys, client.IDKey("Comment", commentID, nil)) } } comments := make([]*Comment, len(commentKeys)) err = client.GetMulti(ctx, commentKeys, comments) if err != nil { panic(err) } for _, post := range posts { for _, commentID := range post.CommentIDs { for idx, comment := range comments { key := commentKeys[idx] if commentID == key.ID() { post.Comments = append(post.Comments, comment) break } } } } ErrNoSuchEntity͕ ࠞͬͨ͟Β͞ΒʹͭΒ͍…ʂ
  10. ErrNoSuchEntityରԠ bt.Get(key, comment, func(err error) error { if err ==

    datastore.ErrNoSuchEntity { // ignore ErrNoSuchEntity return nil } else if err != nil { return err } post.Comments = append(post.Comments, comment) return nil }) Entity୯ҐͷΤϥʔϋϯυϥ༗Γ
  11. ར༻࣮੷༗ • ϝϧΧϦ Ξος (AppEngine) • ϝϧΧϦ KYC (Cloud) •

    ٕज़ॻయ (AppEngine) • timakin͞Μ (AppEngine) ར༻ใࠂ଴ͬͯ·͢❤
  12. godocݟͯ͘Εʂ How to migrate to this library Here's an overview

    of what you need to do to migrate your existing code. replace *datastore.Key to datastore.Key. replace *datastore.Query to datastore.Query. replace *datastore.Iterator to datastore.Iterator. from AE Datastore import go.mercari.io/datastore and go.mercari.io/datastore/aedatastore both. rewrite those using functions of datastore package to FromContext function and Client method calls. replace err.(appengine.MultiError) to err.(datastore.MultiError) . Stop using appengine.BlobKey and replace with string. replace google.golang.org/appengine/datastore.Done to google.golang.org/api/iterator.Done . replace key.IntID() to key.ID() . replace key.StringID() to key.Name() . When nesting a struct, apply `datastore:", flatten "` to the corresponding field. Delete datastore.TransactionOptions, it is not supported. If using google.golang.org/appengine/datastore , replace to go.mercari.io/datastore . from Cloud Datastore import go.mercari.io/datastore and go.mercari.io/datastore/clouddatastore . rewrite those using functions of datastore package to FromContext function and Client method calls. replace *datastore.Commit to datastore.Commit . If using cloud.google.com/go/datastore , replace to go.mercari.io/datastore . from goon to boom replace *goon.Goon to *boom.Boom . replace goon.FromContext(ctx) to ds, _ := aedatastore.FromContext(ctx); boom.FromClient(ctx, ds) . ଍Γͳ͍ͷݟ͚ͭͨΒIssue΁ʂ