Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced API Calling in Go

Advanced API Calling in Go

Go 1.9 Release Party in Tokyo での発表資料です by @__timakin__

5bfed9aa3a9ebccb0c0f0cb65ee9e012?s=128

Seiji Takahashi

August 22, 2017
Tweet

Transcript

  1. Advanced API Calling in Go @__timakin__ / Go 1.9 Release

    Party in Tokyo
  2. ࣗݾ঺հ • Github: timakin / Twitter: @__timakin__ • ŧŔŕŪ Go

    Ŭ Contribute ŢŦ • Gunosy ৽نࣄۀ։ൃࣨ ॴଐ • Go / Swift • Recently Contributed / Created • giginet/xcprofiler • mercari/gaurun • timakin/gopli • timakin/ssm2env • timakin/md2mid • timakin/gonvert
  3. Copyright© Gunosy Inc. All Rights Reserved 3 Go / Python

    ΤϯδχΞืूத ▶https://gunosy.co.jp/recruit/ Gunosy͸ɺ౦ژେֶʹ௨͏3ਓͷֶੜͷ
 ʮ৘ใΛੈքதͷਓʹ࠷దʹಧ͚͍ͨʯͱ͍͏૝͍͔Β࢝·Γ·ͨ͠ɻ ౦ূϚβʔζ্৔ɺ࿡ຊ໦ώϧζ΁ͷΦϑΟεҠసΛܦͯɺ
 େ͖͘੒௕͍ͯ͠ΔձࣾͰ׆༂͍ͨ͠ϝϯόʔΛืू͍ͯ͠·͢ɻ
  4. ࠓ೔ͷςʔϚ

  5. GoͰAPI Client࡞Δͱ͖ͷ ΧελϚΠζྫ

  6. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ

  7. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ

  8. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ

  9. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ

  10. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ

  11. ௨ৗͷAPIϦΫΤετ

  12. ௨ৗͷϦΫΤετ

  13. ಠࣗAPI Client

  14. ಠࣗͷAPI ClientΛఆٛ͢Δ

  15. ಠࣗͷAPI ClientΛఆٛ͢Δ API Clientͷߏ଄ମΛఆٛ͢Δ

  16. ಠࣗͷAPI ClientΛఆٛ͢Δ ίϯετϥΫλΛॻ͘

  17. ಠࣗͷAPI ClientΛఆٛ͢Δ ڞ௨ϝιουΛఆٛ͢Δ

  18. ಠࣗͷAPI ClientΛఆٛ͢Δ Ϧιʔε͝ͱͷAPI ίʔϧΛఆٛ͢Δɻ ͜ͷล͸௨ৗͷrepository૚Ͱ΍ͬͯΔ͜ͱͱ
 ͦ͜·ͰมΘΒͳ͍ɻ

  19. goroutine

  20. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ

  21. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env

  22. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter

    Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ
  23. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter

    Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ • Ͱ΋10ݸ͔ͣͭ͠औಘͰ͖ͳ͍

  24. goroutineͰͷฒߦcall AWS ClientΛ࣋ͬͨߏ଄ମΛఆٛ͢Δ

  25. goroutineͰͷฒߦcall AWS SSMʢύϥϝʔλετΞʣ͔Β
 औͬͯདྷͨValueΛmapʹͯ͠ฦ͢ϝιουΛఆٛɻ

  26. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ

  27. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ

  28. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར

  29. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར ɾeg.Go಺෦ͰฒߦॲཧΛߦ͏

  30. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢

  31. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢ ɾcontextͷλΠϜΞ΢τ௨஌Λड͚औͬͨΒͦͷ࣌఺Ͱऴྃ

  32. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ

  33. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ ɾchannelܦ༝Ͱؼͬͯདྷͨ݁ՌΛॱ࣍mapʹ٧ΊΔ

  34. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ

  35. Configuration

  36. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ

  37. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ

  38. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ • ྫʣmercari/gaurun

  39. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢

  40. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢ ɾhttp.ClientͷϑΟʔϧυʹTransportΛ౉ͯ͠ΫϥΠΞϯτੜ੒

  41. Proxy

  42. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ

  43. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠

  44. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ

  45. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ • ྫʣkazeburo/chocon

  46. Proxy

  47. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε

  48. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport

    • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ
  49. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport

    • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ • http.Client • http.Transport Λ಺෦ʹ࣋ͬͨɺHTTPϓϩτίϧͰͷ௨ ৴Λ͓͜ͳ͏ͨΊͷૉͷΫϥΠΞϯτ࣮૷ • func (c *Client) Do(…) ͸தͰRoundTripΛݺΜͰ͍Δ
  50. Proxy • kazeburo/chocon಺Ͱ͸…

  51. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ɺRoundTripperΛ࣋ͬͨߏ଄ମ

  52. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ʢHostͷஔ׵ɺBodyͷcopyʣ

  53. Proxy ɾServeHTTPΛ࣮૷ ɾProxy͕ड͚औͬͨresponseΛɺAPIClient޲͚ʹcopyͯ͠
 ɹฦ٫ͯ͋͛͠Δ

  54. Retry

  55. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠

  56. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go

  57. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go

  58. AWS-SDK-Go • NinjaΈ͍ͨͳίʔυ͍ͬͺ͍͋Δ

  59. AWS-SDK-Go ಠࣗRetryerΛఆٛ͢Δ

  60. AWS-SDK-Go WithRetryerͰɺΧελϜϦτϥΠϠʔΛઃఆͯ͋͛͠Δɻ ʢcontextύοέʔδͷWithValueతͳʣ

  61. AWS-SDK-Go Requestੜ੒࣌ʹ΋RetryerΛҾ͖౉͢

  62. AWS-SDK-Go ϦτϥΠՄೳ͔Ͳ͏͔ΛνΣοΫͯ͠ɺtrueͳΒΧ΢ϯτ૿Ճ

  63. AWS-SDK-Go νΣοΫޙɺϦτϥΠՄೳͳΒϦΫΤετ಺༰Λίϐʔ

  64. ୅ସखஈ • ಠࣗRetryerΛఆٛͤͣʹɺcenkalti/backoffͳͲͷ exponential backoff࣮૷Λ࢖͏ํ๏΋͋Γ

  65. ྫʣgo-datadog-api

  66. Paging

  67. Paging • ϖʔδϯά༻ͷAPIΛ࡞ΔͷͰ͸ͳ͘ɺ
 ϖʔδϯάରԠͷAPIʹɺ೚ҙͷϖʔδ·Ͱ͸
 ϦΫΤετΛ౤͛ଓ͚͍ͨ • ྫʣEC2ύϥϝʔλʔετΞʹอଘ͞Εͨ
 ΩʔͷҰཡΛऔಘ͍ͨ͠ɻ͕ɺ
 10ݸ͔ͣͭ͠ฦͬͯ͜ͳ͍…

  68. AWS-SDK-Go • ࠶Ninja

  69. Paging request.Paginationʂʂʂʂ ϦΫΤετͷstructʹ߹Θͤͯɺϖʔδϯάॲཧͷ࣮૷Λ࣋ͭ

  70. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ

  71. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ Ҿ਺͕௕͗͢ΔͷͰݟ͑ͳ͍͕ɺ
 ίʔϧόοΫ͕ઃఆͯ͋ͬͯ͠ɺ
 ݁Ռͱʮ࠷ޙͷϖʔδ͔ʯ
 ͷϑϥάΛฦ͢

  72. ·ͱΊ

  73. ·ͱΊ • ੈͷதʹ͸৭ʑͳAPI Client͕͍Δ
 ΈΜͳҧͬͯΈΜͳ͍͍ • ඞཁͳΒΧελϜClientɺͳ͍͠ϦΫΤετ
 ؅ཧ༻ͷstructΛ࣮૷͢Δ • aws-sdk-go͸͕͢͞ʹ΍Γ͗͢ͳͷͰɺ


    ͪΐ͏Ͳ͍͍ϋϯυϥΛ࣮૷͠·͠ΐ͏
  74. Thank you!