ௐ΂ͳ͕ΒGCP΍ͬͯ Έͨ࿩ GCPUG In Osaka #3 @plan9user (2017-07-19) 

ࣗݾ঺հ • ໳ଟګฏ • Favorites • OS: Plan 9 • Editor: acme • Shell: rc • ϑΣϯϦϧͰಇ͍͍ͯ·͢ 

No content

Ϟνϕʔγϣϯ • GoΛॻ͍͍ͯͨͷͰڵຯ͸ͣͬͱ͋ͬͨ • ࣮͸Compute EngineͰPlan 9͕ಈ͘ • ಈ͚ͩ͘͡Όͳͯ͘ίϯιʔϧ͕࢖͑Δ • Plan 9ͷ؅ཧʹ͸ίϯιʔϧඞਢ 

৘ใऩू • Google Cloud Platform Japan ެࣜϒϩά • Publickey • Google Cloud Platform Advent Calendar • ໘നͦ͏ͳαʔϏε͸υΩϡϝϯτಡΉ 

Կ͔࿩͠·ͤΜ͔? 

• ཉ͔ͬͨ͠αʔϏεΛ(్த·Ͱ)࡞ͬͨ࿩ • ॳΊͯͷਓ͸ • Ͳ͏͍͏αʔϏε͕͋Δͷ͔ • ࢖͍ͬͯΔਓ͸ • ڭ͍͑ͯͩ͘͞ ࠓ೔࿩͢͜ͱ 

͜Μͳ͜ͱ͋Γ·ͤΜ͔? • ಡΜͩ͜ͱ͸͋Δ͚ͲݕࡧͰ͖ͳ͍ • ϒοΫϚʔΫ͢ΔͷΛ๨Ε͍ͯͨ • هࣄͦͷ΋ͷ͕ͳ͘ͳͬͯ͠·ͬͨ ޙഐʮͳΜͰ͜Ε೩͑ͯΔΜͰ͔͢ʯʮͦΕ͸͔֬ݩهࣄ͕...ʯ 

ಡΜͩهࣄશ෦࢒͍ͨ͠ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

༻ҙ͢Δ΋ͷ • GAE/Go SDK • Go (͋Δͱศར) $ nix-env -i go-1.8.3 $ nix-env -i google-app-engine-go-sdk 

؀ڥΛબͿ 

Computing Option • App Engine Standard Environment (GAE SE) • App Engine Flexible Environment (GAE FE) • Container Engine (GKE) • Compute Engine (GCE) GCP ͷίϯϐϡʔτ Φϓγϣϯ ΨΠυΛެ։ https://cloudplatform-jp.googleblog.com/2016/06/gcp_17.html 

GAE/Go SEΛ࢖͏ • ғ͍ࠐΈͷϦεΫ͸͋Δ͚Ͳ... • ӡ༻ʹखؒΛ͔͚ͨ͘ͳ͔ͬͨ • ແྉ࿮͕͋Δͷخ͍͠ • RDB࢖͑ͳͯ͘΋ࠔΒͳ͍ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

Ϣʔβೝূ(1) • Users API (google.golang.org/appengine/user) • ͱͯ΋؆୯ʹ࢖͑Δ • Current(); CurrentOAuth(); LoginURL() • GoogleΞΧ΢ϯτ͔͠ରԠ͍ͯ͠ͳ͍ • GitHubΞΧ΢ϯτͰ΋ϩάΠϯ͍ͨ͠... • ࣗલ࣮૷ • ໘౗͍͘͞ 

Ϣʔβೝূ(2) • Cloud Identity-Aware Proxy (IAP) • Cloud Endpoints 

Cloud Identity-Aware Proxy • ೝূ͞ΕͨϢʔβʹΞϓϦΛެ։͢Δ • GoogleΞΧ΢ϯτલఏͷΑ͏ͳͷͰະ࢖༻ 

Cloud Endpoints • REST APIΛ࡞ΔͨΊͷαʔϏε • ϩΪϯάɺϞχλϦϯάɺอޢͳͲߦ͏ • GAE SE: Cloud Endpoints Framework • ͦΕҎ֎: Extensible Service Proxy Extensible Service Proxy ͱ Endpoints Frameworks ͷൺֱ https://cloud.google.com/endpoints/docs/frameworks-extensible-service-proxy?hl=ja 

Cloud Endpoints • Cloud Endpoints Framework
 github.com/GoogleCloudPlatform/go-endpoints/endpoints • ඪ४Ͱ͸Issuer = accounts.google.com • AuthenticatorΛ࣮૷͢Ε͹৭ʑ΍Εͦ͏ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

go-endpointsΛ࢖͏ • app.yamlʹhandlerΛ௥Ճ application: [app-name] version: 1 runtime: go api_version: go1 handlers: - url: /_ah/spi/.* script: _go_app 

Go 1.8Λ࢖͏ • api_versionΛมߋ͢Δ(1.9.55Ҏ্) application: [app-name] version: 1 runtime: go api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app Google App Engine for Go͕Go1.8ʹରԠͨ͠ͷͰࢼͯ͠Έͨ http://qiita.com/tenntenn/items/0b92fc089f8826fabaf1 

go-endpointsͰAPI࣮૷ • APIͱͯ͠ѻ͏ϝιου͸ಛఆͷϧʔϧΛຬͨ͢ • JSON΁ͷม׵౳͸উखʹ΍ͬͯ͘ΕΔ type Service struct{} type ListOptions struct { Limit int `json:"limit" endpoints:"d=10"` } type ArticleList struct { Items []*Article `json:"items"` } type Article struct { URL string `json:"url"` } func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) { return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil } 

go-endpointsͷURL Router • ࡞ͬͨAPIΛURLʹϚοϓ͢Δ func init() { srv := new(Service) api, err := endpoints.RegisterService(srv, "logs", "v1", "Reading Log API", true) if err != nil { log.Fatal(err) } m := api.MethodByName("ArticleList") if m == nil { log.Fatal("Missing ArticleList method") } info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." endpoints.HandleHTTP() } APIαʔϏεొ࿥ URLͷઃఆ ͜ͷྫ͸ https://[app-name].appspot.com/_ah/api/logs/v1/articles ͱͳΔ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

go-endpointsͰೝূ • infoʹɺೝূ͢ΔͨΊʹඞཁͳ஋Λઃఆ͢Δ • API Manager / ೝূ৘ใ / ΫϥΠΞϯτIDͷ࡞੒ ͰWeb Application Λબ୒ • ੜ੒͞ΕͨΫϥΠΞϯτIDͱγʔΫϨοτΛอଘ const ClientID = "xxxx" func init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} endpoints.HandleHTTP() } 

go-endpointsͰೝূ • infoͱಉ͡஋Λ࢖ͬͯCurrentUser()Λݺͼग़͢ func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) { scopes := []string{endpoints.EmailScope} audiences := []string{"[app-name].appspot.com"} clientIDs := []string{ClientID, endpoints.APIExplorerClientID} user, err := endpoints.CurrentUser(c, scopes, audiences, clientIDs) if err != nil { return nil, err } return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil } 

API ExplorerͰςετ • https://[app-name].appspot.com/_ah/api/explorer • ೝূ͕ඞཁͱѻΘΕΔΑ͏ʹͳ͍ͬͯΔ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

DatastoreΛ࢖͏ • NoSQLܕσʔλϕʔε • Keyͷઃܭ͕೉͍͠ • Cloud Storage΍SQLܕDB΋͋Δ͚Ͳ... 

Datastore΁อଘ • ϢʔβIDͱهࣄͷURLͰ֊૚ʹ͍ͯ͠Δ type Page struct { URL string LastMod time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) } 

Datastore΁อଘ • datastore.RunInTransaction()Ͱߋ৽͢Δ type Page struct { URL string LastMod time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) } ࣦഊͷ৔߹͸ϦτϥΠ͞ΕΔͷͰɺ ࠷৽ͷ஋ΛಡΈ௚͔ͯ͠Βߋ৽ 

هࣄ಺༰ͷऔಘ • Goඪ४ͷhttp.Get()͸੍ݶ͞Ε͍ͯΔ • ୅ΘΓʹurlfetch.ClientΛ࢖͏ • API࣮ߦͷཪͰऔಘ౳Λ͍ͨ͠ • GAE/Go SEͷϦΫΤετ͸60ඵͰλΠϜΞ΢τ • Task QueueΛ࢖͏ • Cloud Pub/Sub͸Subscribe͢Δଆ͕ෳ਺ͷ৔߹ʹศརͦ͏ 

Task Queue • Push QueueͱPull Queueͷ2छྨ͋Δ • GAE/Go SE͸Push Queue • ಛఆͷΤϯυϙΠϯτʹϦΫΤετ͞ΕΔ • GAE͔ΒͷΈ࣮ߦ͍ͤͨ͞ 

Task Queue༻URL௥Ճ • Task Queue͕ϦΫΤετ͢ΔURLΛ௥Ճ application: [app-name] version: 1 runtime: go api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app - url: /_ah/queue/.* script: _go_app login: admin ؅ཧऀͷΈΞΫηεՄೳ 

Ωϡʔͷઃఆ • queue.yamlΛ࡞੒ͯ͠ΩϡʔΛઃఆ • ৭ʑͰ͖ΔͷͰެࣜυΩϡϝϯτΛಡ΋͏ queue: - name: snap rate: 1/s bucket_size: 5 

Task Queue࣮૷ • ΩϡʔʹೖΕ࣮ͯߦɺΤϥʔͳΒϦτϥΠ func AddTask(c context.Context, url string) error { v := url.Values{} v.Set("url", url) t := taskqueue.NewPOSTTask("/_ah/queue/snap", v) _, err := taskqueue.Add(c, t, "snap") return err } func SnapQueueHandler(w http.ResponseWriter, r *http.Request) { c := appengine.NewContext(r) url := r.FormValue("url") cli := urlfetch.Client(c) resp, err := cli.Get(uri) if err != nil { w.WriteHeader(http.StatusInternalServerError) return } ... } 

Task Queueϋϯυϥ௥Ճ • go-endpoints͕endpoints.HandleHTTP()͍ͯ͠Δ͕ɺ
 ී௨ʹͦͷ··Goඪ४net/httpͰ௥Ճ͢Ε͹͍͍ const ClientID = "xxxx" func init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} http.HandleFunc("/_ah/queue/snap", SnapQueueHandler) endpoints.HandleHTTP() } 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

Χʔιϧͷར༻ • Datastore͸೚ҙͷϖʔδʹҠಈ͢Δͷ͸޲͔ͳ͍
 ࢓༷͔Β֎ͨ͠΄͏͕͍͍ • Query.Offset()͸ಡΈඈ͹͕͢ಡΈࠐΈ͸͍ͯ͠Δ
 ಡΈࠐΈճ਺Ͱ՝ۚ͞ΕΔ... • ΧʔιϧΛ࢖ͬͯલޙͷ஋Λऔಘ͢Δͱ͍͍ • ͨͩ͠Χʔιϧ͕࡞ΒΕͨΫΤϦͱಉ͡৚͕݅ඞཁ 

Χʔιϧͷ࢖͍ํ func FetchEntries(c context.Context, u *User, cursor string) ([]*Page, string, error) { key := datastore.NewKey(c, "User", u.ID, 0, nil) q := datastore.NewQuery("Page").Ancestor(key).Limit(20) if cursor != "" { p, err := datastore.DecodeCursor(cursor) if err != nil { return nil, "", err } q = q.Start(p) } var a []*Page t := q.Run(c) for { var page Page _, err := t.Next(&page) ... লུ ... } p, err := t.Cursor() if err != nil { return nil, "", err } return a, p.String(), nil } ಉ౳ͳΫΤϦͰऔಘͨ͠Χʔιϧ 

࡞ΔαʔϏεͷ࢓༷ • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ • APIͱ࣮ͯ͠૷͢Δ • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ 

Search API • Datastore͸Ωʔϫʔυݕࡧ͕ۤख • Search APIͰ୅༻͢Δ • ͨͩ͠Datastoreͱ͸ผͷ؅ཧʹͳΔ 

• ΠϯσοΫεʹɺෳ਺ͷυΩϡϝϯτΛొ࿥ Search APIͷ࢖͍ํ import "google.golang.org/appengine/search" type Document struct { Content search.HTML LastMod time.Time } func AddContent(c context.Context, uid string, body []byte) error { doc := Document{ Content: search.HTML(body), LastMod: time.Now(), } x, err := search.Open("article_" + uid) if err != nil { return err } if _, err := x.Put(c, uri, &doc); err != nil { return err } return nil } υΩϡϝϯτΛఆٛ ΠϯσοΫε΁௥Ճ 

ݕࡧͯ͠ΈΔ Query String https://cloud.google.com/appengine/docs/standard/go/search/query_strings 

• ΠϯσοΫε͔ΒΩʔϫʔυݕࡧ...ͷ͸͕ͣ... Search APIͷ࢖͍ํ func Query(c context.Context, uid, expr string, cursor Cursor) ([]*Document, string, error) { x, err := search.Open("article_" + uid) if err != nil { return nil, "", err } var a []*Document if cursor != "" { options.Cursor = search.Cursor(cursor) } t := x.Search(c, expr, nil) for { var doc Document _, err := t.Next(&doc) if err == search.Done { return a, string(t.Cursor()), nil } if err != nil { return nil, "", err } a = append(a, &doc) } } 

·ͱΊ • ϚωʔδυαʔϏε͓खܰ • झຯͰॻ͘ίʔυͳΒ໎Θͣ࢖͏ • ϕϯμʔϩοΫΠϯ͸ؾΛ͚ͭͯ... 

͋Γ͕ͱ͏͍͟͝·ͨ͠