Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
調べながらGCPやってみた話/gcpug-osaka-3
kadota kyohei
July 20, 2017
Programming
1
310
調べながらGCPやってみた話/gcpug-osaka-3
GCPUG In Osaka #3 で発表した資料。
色々サービス使ってみました。
https://gcpug-osaka.connpass.com/event/61297/
kadota kyohei
July 20, 2017
Tweet
Share
More Decks by kadota kyohei
See All by kadota kyohei
lufia
0
440
lufia
0
410
lufia
1
550
lufia
19
6.7k
Other Decks in Programming
See All in Programming
progate
3
1.2k
rukiadia
3
970
dnskimo
8
1.5k
thatjeffsmith
0
1.6k
cc4966
1
110
yattom
33
11k
qsona
21
5.3k
shiba6v
0
210
chimerast
0
360
sgeengineer
1
210
kidokidofire
1
120
fancyweb
1
100
Featured
See All Featured
philhawksworth
192
17k
jponch
103
5.1k
reverentgeek
28
2.1k
brad_frost
157
6.5k
addyosmani
1346
200k
malarkey
119
16k
jeffersonlam
330
15k
aarron
258
36k
jasonvnalue
81
8.2k
cassininazir
347
20k
philhawksworth
193
8.9k
maltzj
502
36k
Transcript
ௐͳ͕ΒGCPͬͯ Έͨ GCPUG In Osaka #3 @plan9user (2017-07-19)
ࣗݾհ • ଟګฏ • Favorites • OS: Plan 9 •
Editor: acme • Shell: rc • ϑΣϯϦϧͰಇ͍͍ͯ·͢
None
Ϟνϕʔγϣϯ • GoΛॻ͍͍ͯͨͷͰڵຯͣͬͱ͋ͬͨ • ࣮Compute EngineͰPlan 9͕ಈ͘ • ಈ͚ͩ͘͡Όͳͯ͘ίϯιʔϧ͕͑Δ •
Plan 9ͷཧʹίϯιʔϧඞਢ
ใऩू • Google Cloud Platform Japan ެࣜϒϩά • Publickey •
Google Cloud Platform Advent Calendar • ໘നͦ͏ͳαʔϏευΩϡϝϯτಡΉ
Կ͔͠·ͤΜ͔?
• ཉ͔ͬͨ͠αʔϏεΛ(్த·Ͱ)࡞ͬͨ • ॳΊͯͷਓ • Ͳ͏͍͏αʔϏε͕͋Δͷ͔ • ͍ͬͯΔਓ • ڭ͍͑ͯͩ͘͞
ࠓ͢͜ͱ
͜Μͳ͜ͱ͋Γ·ͤΜ͔? • ಡΜͩ͜ͱ͋Δ͚ͲݕࡧͰ͖ͳ͍ • ϒοΫϚʔΫ͢ΔͷΛΕ͍ͯͨ • هࣄͦͷͷ͕ͳ͘ͳͬͯ͠·ͬͨ ޙഐʮͳΜͰ͜Ε೩͑ͯΔΜͰ͔͢ʯʮͦΕ͔֬ݩهࣄ͕...ʯ
ಡΜͩهࣄશ෦͍ͨ͠
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
༻ҙ͢Δͷ • GAE/Go SDK • Go (͋Δͱศར) $ nix-env -i
go-1.8.3 $ nix-env -i google-app-engine-go-sdk
ڥΛબͿ
Computing Option • App Engine Standard Environment (GAE SE) •
App Engine Flexible Environment (GAE FE) • Container Engine (GKE) • Compute Engine (GCE) GCP ͷίϯϐϡʔτ Φϓγϣϯ ΨΠυΛެ։ https://cloudplatform-jp.googleblog.com/2016/06/gcp_17.html
GAE/Go SEΛ͏ • ғ͍ࠐΈͷϦεΫ͋Δ͚Ͳ... • ӡ༻ʹखؒΛ͔͚ͨ͘ͳ͔ͬͨ • ແྉ͕͋Δͷخ͍͠ • RDB͑ͳͯ͘ࠔΒͳ͍
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Ϣʔβೝূ(1) • Users API (google.golang.org/appengine/user) • ͱͯ؆୯ʹ͑Δ • Current(); CurrentOAuth();
LoginURL() • GoogleΞΧϯτ͔͠ରԠ͍ͯ͠ͳ͍ • GitHubΞΧϯτͰϩάΠϯ͍ͨ͠... • ࣗલ࣮ • ໘͍͘͞
Ϣʔβೝূ(2) • Cloud Identity-Aware Proxy (IAP) • Cloud Endpoints
Cloud Identity-Aware Proxy • ೝূ͞ΕͨϢʔβʹΞϓϦΛެ։͢Δ • GoogleΞΧϯτલఏͷΑ͏ͳͷͰະ༻
Cloud Endpoints • REST APIΛ࡞ΔͨΊͷαʔϏε • ϩΪϯάɺϞχλϦϯάɺอޢͳͲߦ͏ • GAE SE:
Cloud Endpoints Framework • ͦΕҎ֎: Extensible Service Proxy Extensible Service Proxy ͱ Endpoints Frameworks ͷൺֱ https://cloud.google.com/endpoints/docs/frameworks-extensible-service-proxy?hl=ja
Cloud Endpoints • Cloud Endpoints Framework github.com/GoogleCloudPlatform/go-endpoints/endpoints • ඪ४ͰIssuer =
accounts.google.com • AuthenticatorΛ࣮͢Ε৭ʑΕͦ͏
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
go-endpointsΛ͏ • app.yamlʹhandlerΛՃ application: [app-name] version: 1 runtime: go api_version:
go1 handlers: - url: /_ah/spi/.* script: _go_app
Go 1.8Λ͏ • api_versionΛมߋ͢Δ(1.9.55Ҏ্) application: [app-name] version: 1 runtime: go
api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app Google App Engine for Go͕Go1.8ʹରԠͨ͠ͷͰࢼͯ͠Έͨ http://qiita.com/tenntenn/items/0b92fc089f8826fabaf1
go-endpointsͰAPI࣮ • APIͱͯ͠ѻ͏ϝιουಛఆͷϧʔϧΛຬͨ͢ • JSONͷมউखʹͬͯ͘ΕΔ type Service struct{} type ListOptions
struct { Limit int `json:"limit" endpoints:"d=10"` } type ArticleList struct { Items []*Article `json:"items"` } type Article struct { URL string `json:"url"` } func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) { return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil }
go-endpointsͷURL Router • ࡞ͬͨAPIΛURLʹϚοϓ͢Δ func init() { srv := new(Service)
api, err := endpoints.RegisterService(srv, "logs", "v1", "Reading Log API", true) if err != nil { log.Fatal(err) } m := api.MethodByName("ArticleList") if m == nil { log.Fatal("Missing ArticleList method") } info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." endpoints.HandleHTTP() } APIαʔϏεొ URLͷઃఆ ͜ͷྫ https://[app-name].appspot.com/_ah/api/logs/v1/articles ͱͳΔ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
go-endpointsͰೝূ • infoʹɺೝূ͢ΔͨΊʹඞཁͳΛઃఆ͢Δ • API Manager / ೝূใ / ΫϥΠΞϯτIDͷ࡞
ͰWeb Application Λબ • ੜ͞ΕͨΫϥΠΞϯτIDͱγʔΫϨοτΛอଘ const ClientID = "xxxx" func init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} endpoints.HandleHTTP() }
go-endpointsͰೝূ • infoͱಉ͡ΛͬͯCurrentUser()Λݺͼग़͢ func (s *Service) ArticleList(c context.Context, r *ListOptions)
(*ArticleList, error) { scopes := []string{endpoints.EmailScope} audiences := []string{"[app-name].appspot.com"} clientIDs := []string{ClientID, endpoints.APIExplorerClientID} user, err := endpoints.CurrentUser(c, scopes, audiences, clientIDs) if err != nil { return nil, err } return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil }
API ExplorerͰςετ • https://[app-name].appspot.com/_ah/api/explorer • ೝূ͕ඞཁͱѻΘΕΔΑ͏ʹͳ͍ͬͯΔ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
DatastoreΛ͏ • NoSQLܕσʔλϕʔε • Keyͷઃܭ͕͍͠ • Cloud StorageSQLܕDB͋Δ͚Ͳ...
Datastoreอଘ • ϢʔβIDͱهࣄͷURLͰ֊ʹ͍ͯ͠Δ type Page struct { URL string LastMod
time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) }
Datastoreอଘ • datastore.RunInTransaction()Ͱߋ৽͢Δ type Page struct { URL string LastMod
time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) } ࣦഊͷ߹ϦτϥΠ͞ΕΔͷͰɺ ࠷৽ͷΛಡΈ͔ͯ͠Βߋ৽
هࣄ༰ͷऔಘ • Goඪ४ͷhttp.Get()੍ݶ͞Ε͍ͯΔ • ΘΓʹurlfetch.ClientΛ͏ • API࣮ߦͷཪͰऔಘΛ͍ͨ͠ • GAE/Go SEͷϦΫΤετ60ඵͰλΠϜΞτ
• Task QueueΛ͏ • Cloud Pub/SubSubscribe͢Δଆ͕ෳͷ߹ʹศརͦ͏
Task Queue • Push QueueͱPull Queueͷ2छྨ͋Δ • GAE/Go SEPush Queue
• ಛఆͷΤϯυϙΠϯτʹϦΫΤετ͞ΕΔ • GAE͔ΒͷΈ࣮ߦ͍ͤͨ͞
Task Queue༻URLՃ • Task Queue͕ϦΫΤετ͢ΔURLΛՃ application: [app-name] version: 1 runtime:
go api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app - url: /_ah/queue/.* script: _go_app login: admin ཧऀͷΈΞΫηεՄೳ
Ωϡʔͷઃఆ • queue.yamlΛ࡞ͯ͠ΩϡʔΛઃఆ • ৭ʑͰ͖ΔͷͰެࣜυΩϡϝϯτΛಡ͏ queue: - name: snap rate:
1/s bucket_size: 5
Task Queue࣮ • ΩϡʔʹೖΕ࣮ͯߦɺΤϥʔͳΒϦτϥΠ func AddTask(c context.Context, url string) error
{ v := url.Values{} v.Set("url", url) t := taskqueue.NewPOSTTask("/_ah/queue/snap", v) _, err := taskqueue.Add(c, t, "snap") return err } func SnapQueueHandler(w http.ResponseWriter, r *http.Request) { c := appengine.NewContext(r) url := r.FormValue("url") cli := urlfetch.Client(c) resp, err := cli.Get(uri) if err != nil { w.WriteHeader(http.StatusInternalServerError) return } ... }
Task QueueϋϯυϥՃ • go-endpoints͕endpoints.HandleHTTP()͍ͯ͠Δ͕ɺ ී௨ʹͦͷ··Goඪ४net/httpͰՃ͢Ε͍͍ const ClientID = "xxxx" func
init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} http.HandleFunc("/_ah/queue/snap", SnapQueueHandler) endpoints.HandleHTTP() }
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Χʔιϧͷར༻ • DatastoreҙͷϖʔδʹҠಈ͢Δͷ͔ͳ͍ ༷͔Β֎ͨ͠΄͏͕͍͍ • Query.Offset()ಡΈඈ͕͢ಡΈࠐΈ͍ͯ͠Δ ಡΈࠐΈճͰ՝ۚ͞ΕΔ... • ΧʔιϧΛͬͯલޙͷΛऔಘ͢Δͱ͍͍ •
ͨͩ͠Χʔιϧ͕࡞ΒΕͨΫΤϦͱಉ͕݅͡ඞཁ
Χʔιϧͷ͍ํ func FetchEntries(c context.Context, u *User, cursor string) ([]*Page, string,
error) { key := datastore.NewKey(c, "User", u.ID, 0, nil) q := datastore.NewQuery("Page").Ancestor(key).Limit(20) if cursor != "" { p, err := datastore.DecodeCursor(cursor) if err != nil { return nil, "", err } q = q.Start(p) } var a []*Page t := q.Run(c) for { var page Page _, err := t.Next(&page) ... লུ ... } p, err := t.Cursor() if err != nil { return nil, "", err } return a, p.String(), nil } ಉͳΫΤϦͰऔಘͨ͠Χʔιϧ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Search API • DatastoreΩʔϫʔυݕࡧ͕ۤख • Search APIͰ༻͢Δ • ͨͩ͠DatastoreͱผͷཧʹͳΔ
• ΠϯσοΫεʹɺෳͷυΩϡϝϯτΛొ Search APIͷ͍ํ import "google.golang.org/appengine/search" type Document struct {
Content search.HTML LastMod time.Time } func AddContent(c context.Context, uid string, body []byte) error { doc := Document{ Content: search.HTML(body), LastMod: time.Now(), } x, err := search.Open("article_" + uid) if err != nil { return err } if _, err := x.Put(c, uri, &doc); err != nil { return err } return nil } υΩϡϝϯτΛఆٛ ΠϯσοΫεՃ
ݕࡧͯ͠ΈΔ Query String https://cloud.google.com/appengine/docs/standard/go/search/query_strings
• ΠϯσοΫε͔ΒΩʔϫʔυݕࡧ...ͷ͕ͣ... Search APIͷ͍ํ func Query(c context.Context, uid, expr string,
cursor Cursor) ([]*Document, string, error) { x, err := search.Open("article_" + uid) if err != nil { return nil, "", err } var a []*Document if cursor != "" { options.Cursor = search.Cursor(cursor) } t := x.Search(c, expr, nil) for { var doc Document _, err := t.Next(&doc) if err == search.Done { return a, string(t.Cursor()), nil } if err != nil { return nil, "", err } a = append(a, &doc) } }
·ͱΊ • ϚωʔδυαʔϏε͓खܰ • झຯͰॻ͘ίʔυͳΒ໎Θͣ͏ • ϕϯμʔϩοΫΠϯؾΛ͚ͭͯ...
͋Γ͕ͱ͏͍͟͝·ͨ͠