GCPUG In Osaka #3 で発表した資料。 色々サービス使ってみました。 https://gcpug-osaka.connpass.com/event/61297/
ௐͳ͕ΒGCPͬͯΈͨGCPUG In Osaka #3@plan9user (2017-07-19)
View Slide
ࣗݾհ• ଟګฏ• Favorites• OS: Plan 9• Editor: acme• Shell: rc• ϑΣϯϦϧͰಇ͍͍ͯ·͢
Ϟνϕʔγϣϯ• GoΛॻ͍͍ͯͨͷͰڵຯͣͬͱ͋ͬͨ• ࣮Compute EngineͰPlan 9͕ಈ͘• ಈ͚ͩ͘͡Όͳͯ͘ίϯιʔϧ͕͑Δ• Plan 9ͷཧʹίϯιʔϧඞਢ
ใऩू• Google Cloud Platform Japan ެࣜϒϩά• Publickey• Google Cloud Platform Advent Calendar• ໘നͦ͏ͳαʔϏευΩϡϝϯτಡΉ
Կ͔͠·ͤΜ͔?
• ཉ͔ͬͨ͠αʔϏεΛ(్த·Ͱ)࡞ͬͨ• ॳΊͯͷਓ• Ͳ͏͍͏αʔϏε͕͋Δͷ͔• ͍ͬͯΔਓ• ڭ͍͑ͯͩ͘͞ࠓ͢͜ͱ
͜Μͳ͜ͱ͋Γ·ͤΜ͔?• ಡΜͩ͜ͱ͋Δ͚ͲݕࡧͰ͖ͳ͍• ϒοΫϚʔΫ͢ΔͷΛΕ͍ͯͨ• هࣄͦͷͷ͕ͳ͘ͳͬͯ͠·ͬͨޙഐʮͳΜͰ͜Ε೩͑ͯΔΜͰ͔͢ʯʮͦΕ͔֬ݩهࣄ͕...ʯ
ಡΜͩهࣄશ෦͍ͨ͠
࡞ΔαʔϏεͷ༷• Ϣʔβຖʹ͚ͯཧ͢Δ• APIͱ࣮ͯ͢͠Δ• ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ• ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ• อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
༻ҙ͢Δͷ• GAE/Go SDK• Go (͋Δͱศར)$ nix-env -i go-1.8.3$ nix-env -i google-app-engine-go-sdk
ڥΛબͿ
Computing Option• App Engine Standard Environment (GAE SE)• App Engine Flexible Environment (GAE FE)• Container Engine (GKE)• Compute Engine (GCE)GCP ͷίϯϐϡʔτ Φϓγϣϯ ΨΠυΛެ։https://cloudplatform-jp.googleblog.com/2016/06/gcp_17.html
GAE/Go SEΛ͏• ғ͍ࠐΈͷϦεΫ͋Δ͚Ͳ...• ӡ༻ʹखؒΛ͔͚ͨ͘ͳ͔ͬͨ• ແྉ͕͋Δͷخ͍͠• RDB͑ͳͯ͘ࠔΒͳ͍
Ϣʔβೝূ(1)• Users API (google.golang.org/appengine/user)• ͱͯ؆୯ʹ͑Δ• Current(); CurrentOAuth(); LoginURL()• GoogleΞΧϯτ͔͠ରԠ͍ͯ͠ͳ͍• GitHubΞΧϯτͰϩάΠϯ͍ͨ͠...• ࣗલ࣮• ໘͍͘͞
Ϣʔβೝূ(2)• Cloud Identity-Aware Proxy (IAP)• Cloud Endpoints
Cloud Identity-Aware Proxy• ೝূ͞ΕͨϢʔβʹΞϓϦΛެ։͢Δ• GoogleΞΧϯτલఏͷΑ͏ͳͷͰະ༻
Cloud Endpoints• REST APIΛ࡞ΔͨΊͷαʔϏε• ϩΪϯάɺϞχλϦϯάɺอޢͳͲߦ͏• GAE SE: Cloud Endpoints Framework• ͦΕҎ֎: Extensible Service ProxyExtensible Service Proxy ͱ Endpoints Frameworks ͷൺֱhttps://cloud.google.com/endpoints/docs/frameworks-extensible-service-proxy?hl=ja
Cloud Endpoints• Cloud Endpoints Framework github.com/GoogleCloudPlatform/go-endpoints/endpoints• ඪ४ͰIssuer = accounts.google.com• AuthenticatorΛ࣮͢Ε৭ʑΕͦ͏
go-endpointsΛ͏• app.yamlʹhandlerΛՃapplication: [app-name]version: 1runtime: goapi_version: go1handlers:- url: /_ah/spi/.*script: _go_app
Go 1.8Λ͏• api_versionΛมߋ͢Δ(1.9.55Ҏ্)application: [app-name]version: 1runtime: goapi_version: go1.8handlers:- url: /_ah/spi/.*script: _go_appGoogle App Engine for Go͕Go1.8ʹରԠͨ͠ͷͰࢼͯ͠Έͨhttp://qiita.com/tenntenn/items/0b92fc089f8826fabaf1
go-endpointsͰAPI࣮• APIͱͯ͠ѻ͏ϝιουಛఆͷϧʔϧΛຬͨ͢• JSONͷมউखʹͬͯ͘ΕΔtype Service struct{}type ListOptions struct {Limit int `json:"limit" endpoints:"d=10"`}type ArticleList struct {Items []*Article `json:"items"`}type Article struct {URL string `json:"url"`}func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) {return &ArticleList{Items: []*Article{&Article{URL: "http://example.com/"},},}, nil}
go-endpointsͷURL Router• ࡞ͬͨAPIΛURLʹϚοϓ͢Δfunc init() {srv := new(Service)api, err := endpoints.RegisterService(srv, "logs", "v1", "Reading Log API", true)if err != nil {log.Fatal(err)}m := api.MethodByName("ArticleList")if m == nil {log.Fatal("Missing ArticleList method")}info := m.Info()info.Name = "article.list"info.Path = "articles"info.HTTPMethod = "GET"info.Desc = "List most recent reading articles."endpoints.HandleHTTP()}APIαʔϏεొURLͷઃఆ͜ͷྫ https://[app-name].appspot.com/_ah/api/logs/v1/articles ͱͳΔ
go-endpointsͰೝূ• infoʹɺೝূ͢ΔͨΊʹඞཁͳΛઃఆ͢Δ• API Manager / ೝূใ / ΫϥΠΞϯτIDͷ࡞ ͰWeb Application Λબ• ੜ͞ΕͨΫϥΠΞϯτIDͱγʔΫϨοτΛอଘconst ClientID = "xxxx"func init() {info := m.Info()info.Name = "article.list"info.Path = "articles"info.HTTPMethod = "GET"info.Desc = "List most recent reading articles."info.Scopes = []string{endpoints.EmailScope}info.Audiences = []string{"[app-name].appspot.com"}info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID}endpoints.HandleHTTP()}
go-endpointsͰೝূ• infoͱಉ͡ΛͬͯCurrentUser()Λݺͼग़͢func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) {scopes := []string{endpoints.EmailScope}audiences := []string{"[app-name].appspot.com"}clientIDs := []string{ClientID, endpoints.APIExplorerClientID}user, err := endpoints.CurrentUser(c, scopes, audiences, clientIDs)if err != nil {return nil, err}return &ArticleList{Items: []*Article{&Article{URL: "http://example.com/"},},}, nil}
API ExplorerͰςετ• https://[app-name].appspot.com/_ah/api/explorer• ೝূ͕ඞཁͱѻΘΕΔΑ͏ʹͳ͍ͬͯΔ
DatastoreΛ͏• NoSQLܕσʔλϕʔε• Keyͷઃܭ͕͍͠• Cloud StorageSQLܕDB͋Δ͚Ͳ...
Datastoreอଘ• ϢʔβIDͱهࣄͷURLͰ֊ʹ͍ͯ͠Δtype Page struct {URL stringLastMod time.Time}func AddEntry(c context.Context, u *User, url string) error {p := datastore.NewKey(c, "User", u.ID, 0, nil)key := datastore.NewKey(c, "Page", url, 0, p)return datastore.RunInTransaction(c, func(c context.Context) error {var page Pageerr := datastore.Get(c, key, &page)if err != nil && err != datastore.ErrNoSuchEntity {return err}page.URL = urlpage.LastMod = time.Now()_, err = datastore.Put(c, key, &page)return err}, nil)}
Datastoreอଘ• datastore.RunInTransaction()Ͱߋ৽͢Δtype Page struct {URL stringLastMod time.Time}func AddEntry(c context.Context, u *User, url string) error {p := datastore.NewKey(c, "User", u.ID, 0, nil)key := datastore.NewKey(c, "Page", url, 0, p)return datastore.RunInTransaction(c, func(c context.Context) error {var page Pageerr := datastore.Get(c, key, &page)if err != nil && err != datastore.ErrNoSuchEntity {return err}page.URL = urlpage.LastMod = time.Now()_, err = datastore.Put(c, key, &page)return err}, nil)}ࣦഊͷ߹ϦτϥΠ͞ΕΔͷͰɺ࠷৽ͷΛಡΈ͔ͯ͠Βߋ৽
هࣄ༰ͷऔಘ• Goඪ४ͷhttp.Get()੍ݶ͞Ε͍ͯΔ• ΘΓʹurlfetch.ClientΛ͏• API࣮ߦͷཪͰऔಘΛ͍ͨ͠• GAE/Go SEͷϦΫΤετ60ඵͰλΠϜΞτ• Task QueueΛ͏• Cloud Pub/SubSubscribe͢Δଆ͕ෳͷ߹ʹศརͦ͏
Task Queue• Push QueueͱPull Queueͷ2छྨ͋Δ• GAE/Go SEPush Queue• ಛఆͷΤϯυϙΠϯτʹϦΫΤετ͞ΕΔ• GAE͔ΒͷΈ࣮ߦ͍ͤͨ͞
Task Queue༻URLՃ• Task Queue͕ϦΫΤετ͢ΔURLΛՃapplication: [app-name]version: 1runtime: goapi_version: go1.8handlers:- url: /_ah/spi/.*script: _go_app- url: /_ah/queue/.*script: _go_applogin: adminཧऀͷΈΞΫηεՄೳ
Ωϡʔͷઃఆ• queue.yamlΛ࡞ͯ͠ΩϡʔΛઃఆ• ৭ʑͰ͖ΔͷͰެࣜυΩϡϝϯτΛಡ͏queue:- name: snaprate: 1/sbucket_size: 5
Task Queue࣮• ΩϡʔʹೖΕ࣮ͯߦɺΤϥʔͳΒϦτϥΠfunc AddTask(c context.Context, url string) error {v := url.Values{}v.Set("url", url)t := taskqueue.NewPOSTTask("/_ah/queue/snap", v)_, err := taskqueue.Add(c, t, "snap")return err}func SnapQueueHandler(w http.ResponseWriter, r *http.Request) {c := appengine.NewContext(r)url := r.FormValue("url")cli := urlfetch.Client(c)resp, err := cli.Get(uri)if err != nil {w.WriteHeader(http.StatusInternalServerError)return}...}
Task QueueϋϯυϥՃ• go-endpoints͕endpoints.HandleHTTP()͍ͯ͠Δ͕ɺ ී௨ʹͦͷ··Goඪ४net/httpͰՃ͢Ε͍͍const ClientID = "xxxx"func init() {info := m.Info()info.Name = "article.list"info.Path = "articles"info.HTTPMethod = "GET"info.Desc = "List most recent reading articles."info.Scopes = []string{endpoints.EmailScope}info.Audiences = []string{"[app-name].appspot.com"}info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID}http.HandleFunc("/_ah/queue/snap", SnapQueueHandler)endpoints.HandleHTTP()}
Χʔιϧͷར༻• DatastoreҙͷϖʔδʹҠಈ͢Δͷ͔ͳ͍ ༷͔Β֎ͨ͠΄͏͕͍͍• Query.Offset()ಡΈඈ͕͢ಡΈࠐΈ͍ͯ͠Δ ಡΈࠐΈճͰ՝ۚ͞ΕΔ...• ΧʔιϧΛͬͯલޙͷΛऔಘ͢Δͱ͍͍• ͨͩ͠Χʔιϧ͕࡞ΒΕͨΫΤϦͱಉ͕݅͡ඞཁ
Χʔιϧͷ͍ํfunc FetchEntries(c context.Context, u *User, cursor string) ([]*Page, string, error) {key := datastore.NewKey(c, "User", u.ID, 0, nil)q := datastore.NewQuery("Page").Ancestor(key).Limit(20)if cursor != "" {p, err := datastore.DecodeCursor(cursor)if err != nil {return nil, "", err}q = q.Start(p)}var a []*Paget := q.Run(c)for {var page Page_, err := t.Next(&page)... লུ ...}p, err := t.Cursor()if err != nil {return nil, "", err}return a, p.String(), nil}ಉͳΫΤϦͰऔಘͨ͠Χʔιϧ
Search API• DatastoreΩʔϫʔυݕࡧ͕ۤख• Search APIͰ༻͢Δ• ͨͩ͠DatastoreͱผͷཧʹͳΔ
• ΠϯσοΫεʹɺෳͷυΩϡϝϯτΛొSearch APIͷ͍ํimport "google.golang.org/appengine/search"type Document struct {Content search.HTMLLastMod time.Time}func AddContent(c context.Context, uid string, body []byte) error {doc := Document{Content: search.HTML(body),LastMod: time.Now(),}x, err := search.Open("article_" + uid)if err != nil {return err}if _, err := x.Put(c, uri, &doc); err != nil {return err}return nil}υΩϡϝϯτΛఆٛΠϯσοΫεՃ
ݕࡧͯ͠ΈΔQuery Stringhttps://cloud.google.com/appengine/docs/standard/go/search/query_strings
• ΠϯσοΫε͔ΒΩʔϫʔυݕࡧ...ͷ͕ͣ...Search APIͷ͍ํfunc Query(c context.Context, uid, expr string, cursor Cursor) ([]*Document, string, error) {x, err := search.Open("article_" + uid)if err != nil {return nil, "", err}var a []*Documentif cursor != "" {options.Cursor = search.Cursor(cursor)}t := x.Search(c, expr, nil)for {var doc Document_, err := t.Next(&doc)if err == search.Done {return a, string(t.Cursor()), nil}if err != nil {return nil, "", err}a = append(a, &doc)}}
·ͱΊ• ϚωʔδυαʔϏε͓खܰ• झຯͰॻ͘ίʔυͳΒ໎Θͣ͏• ϕϯμʔϩοΫΠϯؾΛ͚ͭͯ...
͋Γ͕ͱ͏͍͟͝·ͨ͠
lufia
tsukushi
shibe23
tomokusaba
hschwentner
hsawaji
ivargrimstad
lnit
heyitsmohit
fornewid
iimokoy
ogiwarat
shohata
roundedbygravity
keithpitt
kneath
cromwellryan
tammielis
jakevdp
bryan
rmw
revolveconf
marcduiker
pedronauck
bkeepers