Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
調べながらGCPやってみた話/gcpug-osaka-3
kadota kyohei
July 20, 2017
Programming
1
380
調べながらGCPやってみた話/gcpug-osaka-3
GCPUG In Osaka #3 で発表した資料。
色々サービス使ってみました。
https://gcpug-osaka.connpass.com/event/61297/
kadota kyohei
July 20, 2017
Tweet
Share
More Decks by kadota kyohei
See All by kadota kyohei
最近変わった開発時のあれこれ/features-of-recent-go
lufia
0
600
GCPとGoの話/gcpug-osaka-6
lufia
0
450
REST is not only (web) API interface
lufia
1
730
Go駆動開発で超速Pushエンジンを作った話
lufia
19
7k
Other Decks in Programming
See All in Programming
Circuit⚡
monaapk
0
200
レガシーフレームワークからの移行
ug
0
120
ペパカレで入社した私が感じた2つのギャップと向き合い方
kosuke_ito
0
310
What's new in Shopware 6.5
shyim
0
110
フロントエンドで 良いコードを書くために
t_keshi
3
1.6k
子育てとEMと転職と
_atsushisakai
1
420
コンピュータビジョンセミナー2 / computer_vision_seminar_libSGM
fixstars
0
320
量子コンピュータ時代のプログラミングセミナー / 20221222_Amplify_seminar _route_optimization
fixstars
0
250
(新米)エンジニアリングマネージャーのしごと #RSGT2023
murabayashi
9
5.9k
Listかもしれない
irof
1
280
Writing Greener Java Applications
hollycummins
0
350
Prácticas de Seguridad en Kubernetes
pablokbs
0
130
Featured
See All Featured
Build The Right Thing And Hit Your Dates
maggiecrowley
22
1.4k
Imperfection Machines: The Place of Print at Facebook
scottboms
254
12k
5 minutes of I Can Smell Your CMS
philhawksworth
198
18k
Statistics for Hackers
jakevdp
785
210k
Git: the NoSQL Database
bkeepers
PRO
419
60k
Art, The Web, and Tiny UX
lynnandtonic
284
18k
Building Flexible Design Systems
yeseniaperezcruz
314
35k
BBQ
matthewcrist
75
8.1k
Visualization
eitanlees
128
12k
Faster Mobile Websites
deanohume
295
29k
Robots, Beer and Maslow
schacon
154
7.3k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
24
4.6k
Transcript
ௐͳ͕ΒGCPͬͯ Έͨ GCPUG In Osaka #3 @plan9user (2017-07-19)
ࣗݾհ • ଟګฏ • Favorites • OS: Plan 9 •
Editor: acme • Shell: rc • ϑΣϯϦϧͰಇ͍͍ͯ·͢
None
Ϟνϕʔγϣϯ • GoΛॻ͍͍ͯͨͷͰڵຯͣͬͱ͋ͬͨ • ࣮Compute EngineͰPlan 9͕ಈ͘ • ಈ͚ͩ͘͡Όͳͯ͘ίϯιʔϧ͕͑Δ •
Plan 9ͷཧʹίϯιʔϧඞਢ
ใऩू • Google Cloud Platform Japan ެࣜϒϩά • Publickey •
Google Cloud Platform Advent Calendar • ໘നͦ͏ͳαʔϏευΩϡϝϯτಡΉ
Կ͔͠·ͤΜ͔?
• ཉ͔ͬͨ͠αʔϏεΛ(్த·Ͱ)࡞ͬͨ • ॳΊͯͷਓ • Ͳ͏͍͏αʔϏε͕͋Δͷ͔ • ͍ͬͯΔਓ • ڭ͍͑ͯͩ͘͞
ࠓ͢͜ͱ
͜Μͳ͜ͱ͋Γ·ͤΜ͔? • ಡΜͩ͜ͱ͋Δ͚ͲݕࡧͰ͖ͳ͍ • ϒοΫϚʔΫ͢ΔͷΛΕ͍ͯͨ • هࣄͦͷͷ͕ͳ͘ͳͬͯ͠·ͬͨ ޙഐʮͳΜͰ͜Ε೩͑ͯΔΜͰ͔͢ʯʮͦΕ͔֬ݩهࣄ͕...ʯ
ಡΜͩهࣄશ෦͍ͨ͠
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
༻ҙ͢Δͷ • GAE/Go SDK • Go (͋Δͱศར) $ nix-env -i
go-1.8.3 $ nix-env -i google-app-engine-go-sdk
ڥΛબͿ
Computing Option • App Engine Standard Environment (GAE SE) •
App Engine Flexible Environment (GAE FE) • Container Engine (GKE) • Compute Engine (GCE) GCP ͷίϯϐϡʔτ Φϓγϣϯ ΨΠυΛެ։ https://cloudplatform-jp.googleblog.com/2016/06/gcp_17.html
GAE/Go SEΛ͏ • ғ͍ࠐΈͷϦεΫ͋Δ͚Ͳ... • ӡ༻ʹखؒΛ͔͚ͨ͘ͳ͔ͬͨ • ແྉ͕͋Δͷخ͍͠ • RDB͑ͳͯ͘ࠔΒͳ͍
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Ϣʔβೝূ(1) • Users API (google.golang.org/appengine/user) • ͱͯ؆୯ʹ͑Δ • Current(); CurrentOAuth();
LoginURL() • GoogleΞΧϯτ͔͠ରԠ͍ͯ͠ͳ͍ • GitHubΞΧϯτͰϩάΠϯ͍ͨ͠... • ࣗલ࣮ • ໘͍͘͞
Ϣʔβೝূ(2) • Cloud Identity-Aware Proxy (IAP) • Cloud Endpoints
Cloud Identity-Aware Proxy • ೝূ͞ΕͨϢʔβʹΞϓϦΛެ։͢Δ • GoogleΞΧϯτલఏͷΑ͏ͳͷͰະ༻
Cloud Endpoints • REST APIΛ࡞ΔͨΊͷαʔϏε • ϩΪϯάɺϞχλϦϯάɺอޢͳͲߦ͏ • GAE SE:
Cloud Endpoints Framework • ͦΕҎ֎: Extensible Service Proxy Extensible Service Proxy ͱ Endpoints Frameworks ͷൺֱ https://cloud.google.com/endpoints/docs/frameworks-extensible-service-proxy?hl=ja
Cloud Endpoints • Cloud Endpoints Framework github.com/GoogleCloudPlatform/go-endpoints/endpoints • ඪ४ͰIssuer =
accounts.google.com • AuthenticatorΛ࣮͢Ε৭ʑΕͦ͏
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
go-endpointsΛ͏ • app.yamlʹhandlerΛՃ application: [app-name] version: 1 runtime: go api_version:
go1 handlers: - url: /_ah/spi/.* script: _go_app
Go 1.8Λ͏ • api_versionΛมߋ͢Δ(1.9.55Ҏ্) application: [app-name] version: 1 runtime: go
api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app Google App Engine for Go͕Go1.8ʹରԠͨ͠ͷͰࢼͯ͠Έͨ http://qiita.com/tenntenn/items/0b92fc089f8826fabaf1
go-endpointsͰAPI࣮ • APIͱͯ͠ѻ͏ϝιουಛఆͷϧʔϧΛຬͨ͢ • JSONͷมউखʹͬͯ͘ΕΔ type Service struct{} type ListOptions
struct { Limit int `json:"limit" endpoints:"d=10"` } type ArticleList struct { Items []*Article `json:"items"` } type Article struct { URL string `json:"url"` } func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) { return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil }
go-endpointsͷURL Router • ࡞ͬͨAPIΛURLʹϚοϓ͢Δ func init() { srv := new(Service)
api, err := endpoints.RegisterService(srv, "logs", "v1", "Reading Log API", true) if err != nil { log.Fatal(err) } m := api.MethodByName("ArticleList") if m == nil { log.Fatal("Missing ArticleList method") } info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." endpoints.HandleHTTP() } APIαʔϏεొ URLͷઃఆ ͜ͷྫ https://[app-name].appspot.com/_ah/api/logs/v1/articles ͱͳΔ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
go-endpointsͰೝূ • infoʹɺೝূ͢ΔͨΊʹඞཁͳΛઃఆ͢Δ • API Manager / ೝূใ / ΫϥΠΞϯτIDͷ࡞
ͰWeb Application Λબ • ੜ͞ΕͨΫϥΠΞϯτIDͱγʔΫϨοτΛอଘ const ClientID = "xxxx" func init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} endpoints.HandleHTTP() }
go-endpointsͰೝূ • infoͱಉ͡ΛͬͯCurrentUser()Λݺͼग़͢ func (s *Service) ArticleList(c context.Context, r *ListOptions)
(*ArticleList, error) { scopes := []string{endpoints.EmailScope} audiences := []string{"[app-name].appspot.com"} clientIDs := []string{ClientID, endpoints.APIExplorerClientID} user, err := endpoints.CurrentUser(c, scopes, audiences, clientIDs) if err != nil { return nil, err } return &ArticleList{ Items: []*Article{ &Article{URL: "http://example.com/"}, }, }, nil }
API ExplorerͰςετ • https://[app-name].appspot.com/_ah/api/explorer • ೝূ͕ඞཁͱѻΘΕΔΑ͏ʹͳ͍ͬͯΔ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
DatastoreΛ͏ • NoSQLܕσʔλϕʔε • Keyͷઃܭ͕͍͠ • Cloud StorageSQLܕDB͋Δ͚Ͳ...
Datastoreอଘ • ϢʔβIDͱهࣄͷURLͰ֊ʹ͍ͯ͠Δ type Page struct { URL string LastMod
time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) }
Datastoreอଘ • datastore.RunInTransaction()Ͱߋ৽͢Δ type Page struct { URL string LastMod
time.Time } func AddEntry(c context.Context, u *User, url string) error { p := datastore.NewKey(c, "User", u.ID, 0, nil) key := datastore.NewKey(c, "Page", url, 0, p) return datastore.RunInTransaction(c, func(c context.Context) error { var page Page err := datastore.Get(c, key, &page) if err != nil && err != datastore.ErrNoSuchEntity { return err } page.URL = url page.LastMod = time.Now() _, err = datastore.Put(c, key, &page) return err }, nil) } ࣦഊͷ߹ϦτϥΠ͞ΕΔͷͰɺ ࠷৽ͷΛಡΈ͔ͯ͠Βߋ৽
هࣄ༰ͷऔಘ • Goඪ४ͷhttp.Get()੍ݶ͞Ε͍ͯΔ • ΘΓʹurlfetch.ClientΛ͏ • API࣮ߦͷཪͰऔಘΛ͍ͨ͠ • GAE/Go SEͷϦΫΤετ60ඵͰλΠϜΞτ
• Task QueueΛ͏ • Cloud Pub/SubSubscribe͢Δଆ͕ෳͷ߹ʹศརͦ͏
Task Queue • Push QueueͱPull Queueͷ2छྨ͋Δ • GAE/Go SEPush Queue
• ಛఆͷΤϯυϙΠϯτʹϦΫΤετ͞ΕΔ • GAE͔ΒͷΈ࣮ߦ͍ͤͨ͞
Task Queue༻URLՃ • Task Queue͕ϦΫΤετ͢ΔURLΛՃ application: [app-name] version: 1 runtime:
go api_version: go1.8 handlers: - url: /_ah/spi/.* script: _go_app - url: /_ah/queue/.* script: _go_app login: admin ཧऀͷΈΞΫηεՄೳ
Ωϡʔͷઃఆ • queue.yamlΛ࡞ͯ͠ΩϡʔΛઃఆ • ৭ʑͰ͖ΔͷͰެࣜυΩϡϝϯτΛಡ͏ queue: - name: snap rate:
1/s bucket_size: 5
Task Queue࣮ • ΩϡʔʹೖΕ࣮ͯߦɺΤϥʔͳΒϦτϥΠ func AddTask(c context.Context, url string) error
{ v := url.Values{} v.Set("url", url) t := taskqueue.NewPOSTTask("/_ah/queue/snap", v) _, err := taskqueue.Add(c, t, "snap") return err } func SnapQueueHandler(w http.ResponseWriter, r *http.Request) { c := appengine.NewContext(r) url := r.FormValue("url") cli := urlfetch.Client(c) resp, err := cli.Get(uri) if err != nil { w.WriteHeader(http.StatusInternalServerError) return } ... }
Task QueueϋϯυϥՃ • go-endpoints͕endpoints.HandleHTTP()͍ͯ͠Δ͕ɺ ී௨ʹͦͷ··Goඪ४net/httpͰՃ͢Ε͍͍ const ClientID = "xxxx" func
init() { info := m.Info() info.Name = "article.list" info.Path = "articles" info.HTTPMethod = "GET" info.Desc = "List most recent reading articles." info.Scopes = []string{endpoints.EmailScope} info.Audiences = []string{"[app-name].appspot.com"} info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID} http.HandleFunc("/_ah/queue/snap", SnapQueueHandler) endpoints.HandleHTTP() }
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Χʔιϧͷར༻ • DatastoreҙͷϖʔδʹҠಈ͢Δͷ͔ͳ͍ ༷͔Β֎ͨ͠΄͏͕͍͍ • Query.Offset()ಡΈඈ͕͢ಡΈࠐΈ͍ͯ͠Δ ಡΈࠐΈճͰ՝ۚ͞ΕΔ... • ΧʔιϧΛͬͯલޙͷΛऔಘ͢Δͱ͍͍ •
ͨͩ͠Χʔιϧ͕࡞ΒΕͨΫΤϦͱಉ͕݅͡ඞཁ
Χʔιϧͷ͍ํ func FetchEntries(c context.Context, u *User, cursor string) ([]*Page, string,
error) { key := datastore.NewKey(c, "User", u.ID, 0, nil) q := datastore.NewQuery("Page").Ancestor(key).Limit(20) if cursor != "" { p, err := datastore.DecodeCursor(cursor) if err != nil { return nil, "", err } q = q.Start(p) } var a []*Page t := q.Run(c) for { var page Page _, err := t.Next(&page) ... লུ ... } p, err := t.Cursor() if err != nil { return nil, "", err } return a, p.String(), nil } ಉͳΫΤϦͰऔಘͨ͠Χʔιϧ
࡞ΔαʔϏεͷ༷ • Ϣʔβຖʹ͚ͯཧ͢Δ • APIͱ࣮ͯ͢͠Δ • ಡΜͩهࣄΛऔࣺબͤͣશ෦อଘ͢Δ • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ •
อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ
Search API • DatastoreΩʔϫʔυݕࡧ͕ۤख • Search APIͰ༻͢Δ • ͨͩ͠DatastoreͱผͷཧʹͳΔ
• ΠϯσοΫεʹɺෳͷυΩϡϝϯτΛొ Search APIͷ͍ํ import "google.golang.org/appengine/search" type Document struct {
Content search.HTML LastMod time.Time } func AddContent(c context.Context, uid string, body []byte) error { doc := Document{ Content: search.HTML(body), LastMod: time.Now(), } x, err := search.Open("article_" + uid) if err != nil { return err } if _, err := x.Put(c, uri, &doc); err != nil { return err } return nil } υΩϡϝϯτΛఆٛ ΠϯσοΫεՃ
ݕࡧͯ͠ΈΔ Query String https://cloud.google.com/appengine/docs/standard/go/search/query_strings
• ΠϯσοΫε͔ΒΩʔϫʔυݕࡧ...ͷ͕ͣ... Search APIͷ͍ํ func Query(c context.Context, uid, expr string,
cursor Cursor) ([]*Document, string, error) { x, err := search.Open("article_" + uid) if err != nil { return nil, "", err } var a []*Document if cursor != "" { options.Cursor = search.Cursor(cursor) } t := x.Search(c, expr, nil) for { var doc Document _, err := t.Next(&doc) if err == search.Done { return a, string(t.Cursor()), nil } if err != nil { return nil, "", err } a = append(a, &doc) } }
·ͱΊ • ϚωʔδυαʔϏε͓खܰ • झຯͰॻ͘ίʔυͳΒ໎Θͣ͏ • ϕϯμʔϩοΫΠϯؾΛ͚ͭͯ...
͋Γ͕ͱ͏͍͟͝·ͨ͠