Upgrade to Pro — share decks privately, control downloads, hide ads and more …

とあるサービスにおける、とあるAPIの、 サーバーレス化を目指した検証のアレコレ

とあるサービスにおける、とあるAPIの、 サーバーレス化を目指した検証のアレコレ

JAWS-UG青森支部 [第5回] (https://jaws-tohoku.doorkeeper.jp/events/45162) でAPI Gateway, AWS Lambda(Apex), Elasticsearch Serviceについて話しました

Retu Fukui

June 25, 2016
Tweet

More Decks by Retu Fukui

Other Decks in Technology

Transcript

  1. ࣗݾ঺հ ෱Ҫ ྽ (Takeshi Fukui) Engineer @ Piece of Cake,

    inc. https://github.com/fukuiretu https://twitter.com/fukuiretu https://www.facebook.com/fukuiretu http://fukuiretu.com/ since 2015. 3 ~
  2. ݕ౼ϙΠϯτͱͨ͠ϫέ • ϚωʔδυαʔϏε΁ͷ੾Γସ͑ • ਓతϦιʔεͷίετμ΢ϯ • ΞϓϦέʔγϣϯαʔό͔Βͷಠཱ • ෛՙͷܰݮ •

    Մ༻ੑ • อकੑ (Մಡੑ, ϘτϧωοΫ΍ো֐ͷ੾Γ෼͚) • (͋ΘΑ͘͹) αʔόඅͷίετμ΢ϯ
  3. AWS Lambda ͱ͸ • ԿΒ͔ͷΠϕϯτΛτϦΨʔʹॲཧΛ࣮ߦͰ͖Δ • S3ͷಛఆόέοτ΁ͷϑΝΠϧΞοϓϩʔυ • KinesisͷετϦʔϜ΁ͷσʔλૠೖ •

    DynamoDBͷςʔϒϧߋ৽ • εέδϡʔϥʔ • Πϯϑϥͷ؅ཧ͕ෆཁ • ΦʔτεέʔϦϯά • ࡉ͔͍ྉۚମܥ
  4. Amazon API Gatewayͱ͸ • RESTFulͳAPIͷΤϯυϙΠϯτͷఏڙ͕Ͱ͖Δ • AWSͷ֤छαʔϏε • طଘͷΞϓϦέʔγϣϯ •

    ηΩϡϦςΟ(ೝূ) • IAM • Cognito • Custom Authorizer • Πϯϑϥͷ؅ཧ͕ෆཁ • ΦʔτεέʔϦϯά
  5. Amazon Elasticsearch Serviceͱ͸ • ElasticsearchͷϚωʔδυαʔϏε • Elasticsearchͱ͸Elastic͔ࣾΒఏڙ͞Ε͍ͯΔݕࡧΤϯδϯ • Luceneͱ͍͏શจݕࡧΤϯδϯΛόοΫΤϯυʹ࣋ͭ •

    RESTFulͳAPIͰσʔλͷૢ࡞͕Մೳ • σϓϩΠ͕؆୯ • Πϯετʔϧ࡞ۀͳͲ͕ෆཁ • ؅ཧ͕؆୯ • ো֐ݕग़ • ϊʔυͷަ׵ • σʔλόοΫΞοϓ
  6. Lambda, API GatewayपΓͷΤίγεςϜ • Serverless (http://serverless.com/) • Apex (http://apex.run/) •

    Zappa (https://zappa.gun.io/) • Fluct (https://github.com/fluct/fluct) • Lamvery (https://github.com/marcy-terui/lamvery)
  7. Lambda, API GatewayपΓͷΤίγεςϜ • Serverless (http://serverless.com/) • Apex (http://apex.run/) •

    Zappa (https://zappa.gun.io/) • Fluct (https://github.com/fluct/fluct) • Lamvery (https://github.com/marcy-terui/lamvery)
  8. Goͱ͸ • ಛ௃ • ੩తܕ෇͚ • ฒߦॲཧ͕खܰʹॻ͚Δ • ΫϩείϯύΠϧ͕؆୯Ͱ1ͭͷόΠφϦϑΝΠ ϧͰಈ࡞͢Δ

    • ඪ४Ͱॆ࣮ͨ͠πʔϧ܈ • ϚείοτΩϟϥͷGopher͘Μ͕͖΋͔Θ͍͍ https://golang.org/ GoogleʹΑͬͯ։ൃ͞Εͨϓϩάϥϛϯάݴޠ
  9. ApexͰHello World 2. ϓϩδΣΫτͷ࡞੒ NLEJSBQFYTBNQMF DEBQFYTBNQMF BQFYJOJU &OUFSUIFOBNFPGZPVSQSPKFDU*UTIPVMECFNBDIJOFGSJFOEMZ BTUIJT JTVTFEUPQSFpYZPVSGVODUJPOTJO-BNCEB

     1SPKFDUOBNFBQFYTBNQMF  &OUFSBOPQUJPOBMEFTDSJQUJPOPGZPVSQSPKFDU  1SPKFDUEFTDSJQUJPOBQFYTBNQMF ※ࣄલʹIAMͷ४උ͕ඞཁ
  10. ApexͰHello World package main import ( "encoding/json" "github.com/apex/go-apex" ) type

    message struct { Hello string `json:"hello"` } func main() { apex.HandleFunc(func(event json.RawMessage, ctx *apex.Context) (interface{}, error) { var m message if err := json.Unmarshal(event, &m); err != nil { return nil, err } return m, nil }) } 4. main.goΛ༻ҙ
  11. WebΞϓϦ (APIͷϨεϙϯε) { "result": [ { "access_count": "1064", "address": "ळాࢢେொ5-4-16ΞδϚοΫεϏϧ1ʙ2F",

    "category": "110", "closed": "0", "description": "", "location": "", "menu_count": "0", "name": "ळాॳେτϩ1؏130ԁͷళ͋ͬͺΕण࢘", "name_alphabet": "", "name_kana": "͖͋ͨ͸͓͓ͭͱΖ͍͔ͬΜͻΌ͘͞Μ͡Ύ͏͑ΜͷΈͤ͋ͬͺΕͣ͠", "photo_count": "0", "purpose": "", "restaurant_id": "371693" }, ɾ ɹɹɾ ɾ } ] }
  12. WebΞϓϦ (αϯϓϧσʔλ౤ೖ) func main() { fp, err := os.Open("fixtures/restaurants.csv") if

    err != nil { panic(err) } defer fp.Close() reader := csv.NewReader(fp) reader.Comma = ',' reader.LazyQuotes = true client, err := elastic.NewClient( elastic.SetSniff(false), elastic.SetURL(os.Getenv("ES_HOST")), ) if err != nil { // Handle error panic(err) } bulkRequest := client.Bulk() i := 0 for { record, err := reader.Read() if err == io.EOF { break } else if err != nil { panic(err) } restaurant := newRestaurant(record) mapingData := toMapingData(restaurant) indexReq := elastic.NewBulkIndexRequest().Index("ldgourmet").Type("restaurant").Doc(mapingData) bulkRequest = bulkRequest.Add(indexReq) if i%1000 == 0 { bulkResponse, err := bulkRequest.Do() if err != nil { panic(err) } if bulkResponse == nil { panic("expected bulkResponse to be != nil; got nil") } } i++ } if bulkRequest.NumberOfActions() > 0 { bulkRequest.Do() } } ࢖༻ϥΠϒϥϦ: https://github.com/olivere/elastic CSVͷߦΛಡΈࠐΈ ߏ଄ମʹม׵͠ɺ ESʹ1000݅୯ҐͰ όϧΫΠϯαʔτ
  13. WebΞϓϦ (LambdaͰಈ͍͍ͯΔϓϩάϥϜ) func main() { apex.HandleFunc(func(event json.RawMessage, ctx *apex.Context) (interface{},

    error) { client, err := elastic.NewClient( elastic.SetSniff(false), elastic.SetURL(os.Getenv("ES_HOST")), ) if err != nil { panic(err) } var in input if err := json.Unmarshal(event, &in); err != nil { panic(err) } q := elastic.NewQueryStringQuery(in.SearchWord) q = q.DefaultField("name") searchResult, err := client.Search(). Index("ldgourmet"). Query(q). Sort("name", true). From(0).Size(30). Pretty(true). Do() if err != nil { panic(err) } var out output var ttyp restaurant for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) { if t, ok := item.(restaurant); ok { out.Restaurants = append(out.Restaurants, t) } } return out, nil }) } ࢖༻ϥΠϒϥϦ: https://github.com/olivere/elastic ESͷσʔλݕࡧ ES͔ΒͷऔಘσʔλΛ ߏ଄ମʹม׵
  14. Lambdaͷσόοά͕໘౗ • ͍͍ͪͪσϓϩΠ͢Δඞཁ͕͋Δ(ServerlessͰ͋ Ε͹ϩʔΧϧͰಈ͘ͷͰෆཁ) • ApexͰϥϯλΠϜΛGoʹͨ͠৔߹ɺprint debug Λ͢Δํ๏͕Θ͔Βͳ͔ͬͨ • Tips

    • apex logs ίϚϯυ͕ศར BQFYMPHTTFBSDI BXTMBNCEBBQFYTBNQMF@TFBSDI45"353FRVFTU*EEBFBGDCGB 7FSTJPO BXTMBNCEBBQFYTBNQMF@TFBSDI&/%3FRVFTU*EEBFBGDCGB BXTMBNCEBBQFYTBNQMF@TFBSDI3&10353FRVFTU*EEBFBGDCGB %VSBUJPONT #JMMFE%VSBUJPONT .FNPSZ4J[F.# .BY.FNPSZ6TFE.#
  15. Lambdaʹ͓͚ΔϩάͷՄࢹԽ | ෼ੳ | Τϥʔ௨஌ • CloudWatch LogsͰݟΕΔʹ͸ݟΕΔ͕ࡉ͔͍෼ ੳ͕ඞཁͳͱ͖ͳͲ͸গʑΊΜͲ͏ •

    CloudWatch Logs͸͋͘·ͰࢀߟϨϕϧͱׂΓ͖ ΓɺS3΁όονΤΫεϙʔτ͠ɺdatadog΍ bugsnagͳͲ޷͖ͳαʔϏεʹσʔλϩʔυ͢Δ
  16. API Gatewayͷೝূ • API KeyͰڐ༰ग़དྷͳ͍৔߹ͷߟྀ͕ඞཁ • IAM or Cognito or

    Custom Authorizer • Custom AuthorizerΛར༻ͯ͠JWT or OAuthͰ
  17. Elasticsearch Serviceͷαϙʔτঢ়گ • VPCະରԠ • ηΩϡϦςΟͷ୲อ͸IAMͱಛఆIPڐՄ • ϨΠςϯγ͕ؾʹͳΔ৔߹͸VPCରԠ·Ͱ͸ࣗલͰؤுΔ ͔͠ແ͍ •

    ଴ͬͯΕ͹͍ͣΕରԠ͸͞ΕΔ͸ͣ • Elasticsearchͷόʔδϣϯ͕બ΂ͳ͍ + ௥ै͕஗͍ • 2016/06/25ݱࡏ 1.5.2ɻຊՈ͸2.3.3ɻ • ࠷৽ͷόʔδϣϯΛར༻͍ͨ͠৔߹͸ࣗલͰؤுΔ͔͠ແ ͍ • Ҡߦ࣌ʹݹ͍όʔδϣϯ΁ͷϦετΞ͕ग़དྷͳ͍ͷͰ஫ҙ