Upgrade to Pro — share decks privately, control downloads, hide ads and more …

調べながらGCPやってみた話/gcpug-osaka-3

 調べながらGCPやってみた話/gcpug-osaka-3

GCPUG In Osaka #3 で発表した資料。
色々サービス使ってみました。
https://gcpug-osaka.connpass.com/event/61297/

kadota kyohei

July 20, 2017
Tweet

More Decks by kadota kyohei

Other Decks in Programming

Transcript

  1. ௐ΂ͳ͕ΒGCP΍ͬͯ
    Έͨ࿩
    GCPUG In Osaka #3
    @plan9user (2017-07-19)

    View Slide

  2. ࣗݾ঺հ
    • ໳ଟګฏ
    • Favorites
    • OS: Plan 9
    • Editor: acme
    • Shell: rc
    • ϑΣϯϦϧͰಇ͍͍ͯ·͢

    View Slide

  3. View Slide

  4. Ϟνϕʔγϣϯ
    • GoΛॻ͍͍ͯͨͷͰڵຯ͸ͣͬͱ͋ͬͨ
    • ࣮͸Compute EngineͰPlan 9͕ಈ͘
    • ಈ͚ͩ͘͡Όͳͯ͘ίϯιʔϧ͕࢖͑Δ
    • Plan 9ͷ؅ཧʹ͸ίϯιʔϧඞਢ

    View Slide

  5. ৘ใऩू
    • Google Cloud Platform Japan ެࣜϒϩά
    • Publickey
    • Google Cloud Platform Advent Calendar
    • ໘നͦ͏ͳαʔϏε͸υΩϡϝϯτಡΉ

    View Slide

  6. Կ͔࿩͠·ͤΜ͔?

    View Slide

  7. • ཉ͔ͬͨ͠αʔϏεΛ(్த·Ͱ)࡞ͬͨ࿩
    • ॳΊͯͷਓ͸
    • Ͳ͏͍͏αʔϏε͕͋Δͷ͔
    • ࢖͍ͬͯΔਓ͸
    • ڭ͍͑ͯͩ͘͞
    ࠓ೔࿩͢͜ͱ

    View Slide

  8. ͜Μͳ͜ͱ͋Γ·ͤΜ͔?
    • ಡΜͩ͜ͱ͸͋Δ͚ͲݕࡧͰ͖ͳ͍
    • ϒοΫϚʔΫ͢ΔͷΛ๨Ε͍ͯͨ
    • هࣄͦͷ΋ͷ͕ͳ͘ͳͬͯ͠·ͬͨ
    ޙഐʮͳΜͰ͜Ε೩͑ͯΔΜͰ͔͢ʯʮͦΕ͸͔֬ݩهࣄ͕...ʯ

    View Slide

  9. ಡΜͩهࣄશ෦࢒͍ͨ͠

    View Slide

  10. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  11. ༻ҙ͢Δ΋ͷ
    • GAE/Go SDK
    • Go (͋Δͱศར)
    $ nix-env -i go-1.8.3
    $ nix-env -i google-app-engine-go-sdk

    View Slide

  12. ؀ڥΛબͿ

    View Slide

  13. Computing Option
    • App Engine Standard Environment (GAE SE)
    • App Engine Flexible Environment (GAE FE)
    • Container Engine (GKE)
    • Compute Engine (GCE)
    GCP ͷίϯϐϡʔτ Φϓγϣϯ ΨΠυΛެ։
    https://cloudplatform-jp.googleblog.com/2016/06/gcp_17.html

    View Slide

  14. GAE/Go SEΛ࢖͏
    • ғ͍ࠐΈͷϦεΫ͸͋Δ͚Ͳ...
    • ӡ༻ʹखؒΛ͔͚ͨ͘ͳ͔ͬͨ
    • ແྉ࿮͕͋Δͷخ͍͠
    • RDB࢖͑ͳͯ͘΋ࠔΒͳ͍

    View Slide

  15. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  16. Ϣʔβೝূ(1)
    • Users API (google.golang.org/appengine/user)
    • ͱͯ΋؆୯ʹ࢖͑Δ
    • Current(); CurrentOAuth(); LoginURL()
    • GoogleΞΧ΢ϯτ͔͠ରԠ͍ͯ͠ͳ͍
    • GitHubΞΧ΢ϯτͰ΋ϩάΠϯ͍ͨ͠...
    • ࣗલ࣮૷
    • ໘౗͍͘͞

    View Slide

  17. Ϣʔβೝূ(2)
    • Cloud Identity-Aware Proxy (IAP)
    • Cloud Endpoints

    View Slide

  18. Cloud Identity-Aware Proxy
    • ೝূ͞ΕͨϢʔβʹΞϓϦΛެ։͢Δ
    • GoogleΞΧ΢ϯτલఏͷΑ͏ͳͷͰະ࢖༻

    View Slide

  19. Cloud Endpoints
    • REST APIΛ࡞ΔͨΊͷαʔϏε
    • ϩΪϯάɺϞχλϦϯάɺอޢͳͲߦ͏
    • GAE SE: Cloud Endpoints Framework
    • ͦΕҎ֎: Extensible Service Proxy
    Extensible Service Proxy ͱ Endpoints Frameworks ͷൺֱ
    https://cloud.google.com/endpoints/docs/frameworks-extensible-service-proxy?hl=ja

    View Slide

  20. Cloud Endpoints
    • Cloud Endpoints Framework

    github.com/GoogleCloudPlatform/go-endpoints/endpoints
    • ඪ४Ͱ͸Issuer = accounts.google.com
    • AuthenticatorΛ࣮૷͢Ε͹৭ʑ΍Εͦ͏

    View Slide

  21. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  22. go-endpointsΛ࢖͏
    • app.yamlʹhandlerΛ௥Ճ
    application: [app-name]
    version: 1
    runtime: go
    api_version: go1
    handlers:
    - url: /_ah/spi/.*
    script: _go_app

    View Slide

  23. Go 1.8Λ࢖͏
    • api_versionΛมߋ͢Δ(1.9.55Ҏ্)
    application: [app-name]
    version: 1
    runtime: go
    api_version: go1.8
    handlers:
    - url: /_ah/spi/.*
    script: _go_app
    Google App Engine for Go͕Go1.8ʹରԠͨ͠ͷͰࢼͯ͠Έͨ
    http://qiita.com/tenntenn/items/0b92fc089f8826fabaf1

    View Slide

  24. go-endpointsͰAPI࣮૷
    • APIͱͯ͠ѻ͏ϝιου͸ಛఆͷϧʔϧΛຬͨ͢
    • JSON΁ͷม׵౳͸উखʹ΍ͬͯ͘ΕΔ
    type Service struct{}
    type ListOptions struct {
    Limit int `json:"limit" endpoints:"d=10"`
    }
    type ArticleList struct {
    Items []*Article `json:"items"`
    }
    type Article struct {
    URL string `json:"url"`
    }
    func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) {
    return &ArticleList{
    Items: []*Article{
    &Article{URL: "http://example.com/"},
    },
    }, nil
    }

    View Slide

  25. go-endpointsͷURL Router
    • ࡞ͬͨAPIΛURLʹϚοϓ͢Δ
    func init() {
    srv := new(Service)
    api, err := endpoints.RegisterService(srv, "logs", "v1", "Reading Log API", true)
    if err != nil {
    log.Fatal(err)
    }
    m := api.MethodByName("ArticleList")
    if m == nil {
    log.Fatal("Missing ArticleList method")
    }
    info := m.Info()
    info.Name = "article.list"
    info.Path = "articles"
    info.HTTPMethod = "GET"
    info.Desc = "List most recent reading articles."
    endpoints.HandleHTTP()
    }
    APIαʔϏεొ࿥
    URLͷઃఆ
    ͜ͷྫ͸ https://[app-name].appspot.com/_ah/api/logs/v1/articles ͱͳΔ

    View Slide

  26. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  27. go-endpointsͰೝূ
    • infoʹɺೝূ͢ΔͨΊʹඞཁͳ஋Λઃఆ͢Δ
    • API Manager / ೝূ৘ใ / ΫϥΠΞϯτIDͷ࡞੒ ͰWeb Application Λબ୒
    • ੜ੒͞ΕͨΫϥΠΞϯτIDͱγʔΫϨοτΛอଘ
    const ClientID = "xxxx"
    func init() {
    info := m.Info()
    info.Name = "article.list"
    info.Path = "articles"
    info.HTTPMethod = "GET"
    info.Desc = "List most recent reading articles."
    info.Scopes = []string{endpoints.EmailScope}
    info.Audiences = []string{"[app-name].appspot.com"}
    info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID}
    endpoints.HandleHTTP()
    }

    View Slide

  28. go-endpointsͰೝূ
    • infoͱಉ͡஋Λ࢖ͬͯCurrentUser()Λݺͼग़͢
    func (s *Service) ArticleList(c context.Context, r *ListOptions) (*ArticleList, error) {
    scopes := []string{endpoints.EmailScope}
    audiences := []string{"[app-name].appspot.com"}
    clientIDs := []string{ClientID, endpoints.APIExplorerClientID}
    user, err := endpoints.CurrentUser(c, scopes, audiences, clientIDs)
    if err != nil {
    return nil, err
    }
    return &ArticleList{
    Items: []*Article{
    &Article{URL: "http://example.com/"},
    },
    }, nil
    }

    View Slide

  29. API ExplorerͰςετ
    • https://[app-name].appspot.com/_ah/api/explorer
    • ೝূ͕ඞཁͱѻΘΕΔΑ͏ʹͳ͍ͬͯΔ

    View Slide

  30. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  31. DatastoreΛ࢖͏
    • NoSQLܕσʔλϕʔε
    • Keyͷઃܭ͕೉͍͠
    • Cloud Storage΍SQLܕDB΋͋Δ͚Ͳ...

    View Slide

  32. Datastore΁อଘ
    • ϢʔβIDͱهࣄͷURLͰ֊૚ʹ͍ͯ͠Δ
    type Page struct {
    URL string
    LastMod time.Time
    }
    func AddEntry(c context.Context, u *User, url string) error {
    p := datastore.NewKey(c, "User", u.ID, 0, nil)
    key := datastore.NewKey(c, "Page", url, 0, p)
    return datastore.RunInTransaction(c, func(c context.Context) error {
    var page Page
    err := datastore.Get(c, key, &page)
    if err != nil && err != datastore.ErrNoSuchEntity {
    return err
    }
    page.URL = url
    page.LastMod = time.Now()
    _, err = datastore.Put(c, key, &page)
    return err
    }, nil)
    }

    View Slide

  33. Datastore΁อଘ
    • datastore.RunInTransaction()Ͱߋ৽͢Δ
    type Page struct {
    URL string
    LastMod time.Time
    }
    func AddEntry(c context.Context, u *User, url string) error {
    p := datastore.NewKey(c, "User", u.ID, 0, nil)
    key := datastore.NewKey(c, "Page", url, 0, p)
    return datastore.RunInTransaction(c, func(c context.Context) error {
    var page Page
    err := datastore.Get(c, key, &page)
    if err != nil && err != datastore.ErrNoSuchEntity {
    return err
    }
    page.URL = url
    page.LastMod = time.Now()
    _, err = datastore.Put(c, key, &page)
    return err
    }, nil)
    }
    ࣦഊͷ৔߹͸ϦτϥΠ͞ΕΔͷͰɺ
    ࠷৽ͷ஋ΛಡΈ௚͔ͯ͠Βߋ৽

    View Slide

  34. هࣄ಺༰ͷऔಘ
    • Goඪ४ͷhttp.Get()͸੍ݶ͞Ε͍ͯΔ
    • ୅ΘΓʹurlfetch.ClientΛ࢖͏
    • API࣮ߦͷཪͰऔಘ౳Λ͍ͨ͠
    • GAE/Go SEͷϦΫΤετ͸60ඵͰλΠϜΞ΢τ
    • Task QueueΛ࢖͏
    • Cloud Pub/Sub͸Subscribe͢Δଆ͕ෳ਺ͷ৔߹ʹศརͦ͏

    View Slide

  35. Task Queue
    • Push QueueͱPull Queueͷ2छྨ͋Δ
    • GAE/Go SE͸Push Queue
    • ಛఆͷΤϯυϙΠϯτʹϦΫΤετ͞ΕΔ
    • GAE͔ΒͷΈ࣮ߦ͍ͤͨ͞

    View Slide

  36. Task Queue༻URL௥Ճ
    • Task Queue͕ϦΫΤετ͢ΔURLΛ௥Ճ
    application: [app-name]
    version: 1
    runtime: go
    api_version: go1.8
    handlers:
    - url: /_ah/spi/.*
    script: _go_app
    - url: /_ah/queue/.*
    script: _go_app
    login: admin
    ؅ཧऀͷΈΞΫηεՄೳ

    View Slide

  37. Ωϡʔͷઃఆ
    • queue.yamlΛ࡞੒ͯ͠ΩϡʔΛઃఆ
    • ৭ʑͰ͖ΔͷͰެࣜυΩϡϝϯτΛಡ΋͏
    queue:
    - name: snap
    rate: 1/s
    bucket_size: 5

    View Slide

  38. Task Queue࣮૷
    • ΩϡʔʹೖΕ࣮ͯߦɺΤϥʔͳΒϦτϥΠ
    func AddTask(c context.Context, url string) error {
    v := url.Values{}
    v.Set("url", url)
    t := taskqueue.NewPOSTTask("/_ah/queue/snap", v)
    _, err := taskqueue.Add(c, t, "snap")
    return err
    }
    func SnapQueueHandler(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    url := r.FormValue("url")
    cli := urlfetch.Client(c)
    resp, err := cli.Get(uri)
    if err != nil {
    w.WriteHeader(http.StatusInternalServerError)
    return
    }
    ...
    }

    View Slide

  39. Task Queueϋϯυϥ௥Ճ
    • go-endpoints͕endpoints.HandleHTTP()͍ͯ͠Δ͕ɺ

    ී௨ʹͦͷ··Goඪ४net/httpͰ௥Ճ͢Ε͹͍͍
    const ClientID = "xxxx"
    func init() {
    info := m.Info()
    info.Name = "article.list"
    info.Path = "articles"
    info.HTTPMethod = "GET"
    info.Desc = "List most recent reading articles."
    info.Scopes = []string{endpoints.EmailScope}
    info.Audiences = []string{"[app-name].appspot.com"}
    info.ClientIds = []string{ClientID, endpoints.APIExplorerClientID}
    http.HandleFunc("/_ah/queue/snap", SnapQueueHandler)
    endpoints.HandleHTTP()
    }

    View Slide

  40. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  41. Χʔιϧͷར༻
    • Datastore͸೚ҙͷϖʔδʹҠಈ͢Δͷ͸޲͔ͳ͍

    ࢓༷͔Β֎ͨ͠΄͏͕͍͍
    • Query.Offset()͸ಡΈඈ͹͕͢ಡΈࠐΈ͸͍ͯ͠Δ

    ಡΈࠐΈճ਺Ͱ՝ۚ͞ΕΔ...
    • ΧʔιϧΛ࢖ͬͯલޙͷ஋Λऔಘ͢Δͱ͍͍
    • ͨͩ͠Χʔιϧ͕࡞ΒΕͨΫΤϦͱಉ͡৚͕݅ඞཁ

    View Slide

  42. Χʔιϧͷ࢖͍ํ
    func FetchEntries(c context.Context, u *User, cursor string) ([]*Page, string, error) {
    key := datastore.NewKey(c, "User", u.ID, 0, nil)
    q := datastore.NewQuery("Page").Ancestor(key).Limit(20)
    if cursor != "" {
    p, err := datastore.DecodeCursor(cursor)
    if err != nil {
    return nil, "", err
    }
    q = q.Start(p)
    }
    var a []*Page
    t := q.Run(c)
    for {
    var page Page
    _, err := t.Next(&page)
    ... লུ ...
    }
    p, err := t.Cursor()
    if err != nil {
    return nil, "", err
    }
    return a, p.String(), nil
    }
    ಉ౳ͳΫΤϦͰऔಘͨ͠Χʔιϧ

    View Slide

  43. ࡞ΔαʔϏεͷ࢓༷
    • Ϣʔβຖʹ෼͚ͯ؅ཧ͢Δ
    • APIͱ࣮ͯ͠૷͢Δ
    • ಡΜͩهࣄΛऔࣺબ୒ͤͣશ෦อଘ͢Δ
    • ࣌ܥྻʹಡΜͩϦετΛऔಘ͢Δ
    • อଘͨ͠هࣄ͔Βશจݕࡧ͢Δ

    View Slide

  44. Search API
    • Datastore͸Ωʔϫʔυݕࡧ͕ۤख
    • Search APIͰ୅༻͢Δ
    • ͨͩ͠Datastoreͱ͸ผͷ؅ཧʹͳΔ

    View Slide

  45. • ΠϯσοΫεʹɺෳ਺ͷυΩϡϝϯτΛొ࿥
    Search APIͷ࢖͍ํ
    import "google.golang.org/appengine/search"
    type Document struct {
    Content search.HTML
    LastMod time.Time
    }
    func AddContent(c context.Context, uid string, body []byte) error {
    doc := Document{
    Content: search.HTML(body),
    LastMod: time.Now(),
    }
    x, err := search.Open("article_" + uid)
    if err != nil {
    return err
    }
    if _, err := x.Put(c, uri, &doc); err != nil {
    return err
    }
    return nil
    }
    υΩϡϝϯτΛఆٛ
    ΠϯσοΫε΁௥Ճ

    View Slide

  46. ݕࡧͯ͠ΈΔ
    Query String
    https://cloud.google.com/appengine/docs/standard/go/search/query_strings

    View Slide

  47. • ΠϯσοΫε͔ΒΩʔϫʔυݕࡧ...ͷ͸͕ͣ...
    Search APIͷ࢖͍ํ
    func Query(c context.Context, uid, expr string, cursor Cursor) ([]*Document, string, error) {
    x, err := search.Open("article_" + uid)
    if err != nil {
    return nil, "", err
    }
    var a []*Document
    if cursor != "" {
    options.Cursor = search.Cursor(cursor)
    }
    t := x.Search(c, expr, nil)
    for {
    var doc Document
    _, err := t.Next(&doc)
    if err == search.Done {
    return a, string(t.Cursor()), nil
    }
    if err != nil {
    return nil, "", err
    }
    a = append(a, &doc)
    }
    }

    View Slide

  48. ·ͱΊ
    • ϚωʔδυαʔϏε͓खܰ
    • झຯͰॻ͘ίʔυͳΒ໎Θͣ࢖͏
    • ϕϯμʔϩοΫΠϯ͸ؾΛ͚ͭͯ...

    View Slide

  49. ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View Slide