Advanced API Calling in Go

Transcript

1. Advanced API Calling in Go @__timakin__ / Go 1.9 Release

   Party in Tokyo

2. ࣗݾ঺հ • Github: timakin / Twitter: @__timakin__ • ŧŔŕŪ Go

   Ŭ Contribute ŢŦ • Gunosy ৽نࣄۀ։ൃࣨ ॴଐ • Go / Swift • Recently Contributed / Created • giginet/xcprofiler • mercari/gaurun • timakin/gopli • timakin/ssm2env • timakin/md2mid • timakin/gonvert

3. Copyright© Gunosy Inc. All Rights Reserved 3 Go / Python

   ΤϯδχΞืूத ▶https://gunosy.co.jp/recruit/ Gunosy͸ɺ౦ژେֶʹ௨͏3ਓͷֶੜͷ
 ʮ৘ใΛੈքதͷਓʹ࠷దʹಧ͚͍ͨʯͱ͍͏૝͍͔Β࢝·Γ·ͨ͠ɻ ౦ূϚβʔζ্৔ɺ࿡ຊ໦ώϧζ΁ͷΦϑΟεҠసΛܦͯɺ
 େ͖͘੒௕͍ͯ͠ΔձࣾͰ׆༂͍ͨ͠ϝϯόʔΛืू͍ͯ͠·͢ɻ

4. ࠓ೔ͷςʔϚ

5. GoͰAPI Client࡞Δͱ͖ͷ ΧελϚΠζྫ

6. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ

7. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ

8. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ

9. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ

10. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ

11. ௨ৗͷAPIϦΫΤετ

12. ௨ৗͷϦΫΤετ

13. ಠࣗAPI Client

14. ಠࣗͷAPI ClientΛఆٛ͢Δ

15. ಠࣗͷAPI ClientΛఆٛ͢Δ API Clientͷߏ଄ମΛఆٛ͢Δ

16. ಠࣗͷAPI ClientΛఆٛ͢Δ ίϯετϥΫλΛॻ͘

17. ಠࣗͷAPI ClientΛఆٛ͢Δ ڞ௨ϝιουΛఆٛ͢Δ

18. ಠࣗͷAPI ClientΛఆٛ͢Δ Ϧιʔε͝ͱͷAPI ίʔϧΛఆٛ͢Δɻ ͜ͷล͸௨ৗͷrepository૚Ͱ΍ͬͯΔ͜ͱͱ
 ͦ͜·ͰมΘΒͳ͍ɻ

19. goroutine

20. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ

21. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env

22. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter

    Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ

23. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter

    Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ • Ͱ΋10ݸ͔ͣͭ͠औಘͰ͖ͳ͍


24. goroutineͰͷฒߦcall AWS ClientΛ࣋ͬͨߏ଄ମΛఆٛ͢Δ

25. goroutineͰͷฒߦcall AWS SSMʢύϥϝʔλετΞʣ͔Β
 औͬͯདྷͨValueΛmapʹͯ͠ฦ͢ϝιουΛఆٛɻ

26. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ

27. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ

28. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར

29. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར ɾeg.Go಺෦ͰฒߦॲཧΛߦ͏

30. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢

31. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢ ɾcontextͷλΠϜΞ΢τ௨஌Λड͚औͬͨΒͦͷ࣌఺Ͱऴྃ

32. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ

33. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ ɾchannelܦ༝Ͱؼͬͯདྷͨ݁ՌΛॱ࣍mapʹ٧ΊΔ

34. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ

35. Configuration

36. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ

37. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ

38. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ • ྫʣmercari/gaurun

39. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢

40. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢ ɾhttp.ClientͷϑΟʔϧυʹTransportΛ౉ͯ͠ΫϥΠΞϯτੜ੒

41. Proxy

42. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ

43. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠

44. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ

45. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ • ྫʣkazeburo/chocon

46. Proxy

47. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε

48. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport

    • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ

49. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport

    • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ • http.Client • http.Transport Λ಺෦ʹ࣋ͬͨɺHTTPϓϩτίϧͰͷ௨ ৴Λ͓͜ͳ͏ͨΊͷૉͷΫϥΠΞϯτ࣮૷ • func (c *Client) Do(…) ͸தͰRoundTripΛݺΜͰ͍Δ

50. Proxy • kazeburo/chocon಺Ͱ͸…

51. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ɺRoundTripperΛ࣋ͬͨߏ଄ମ

52. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ʢHostͷஔ׵ɺBodyͷcopyʣ

53. Proxy ɾServeHTTPΛ࣮૷ ɾProxy͕ड͚औͬͨresponseΛɺAPIClient޲͚ʹcopyͯ͠
 ɹฦ٫ͯ͋͛͠Δ

54. Retry

55. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠

56. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go

57. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go

58. AWS-SDK-Go • NinjaΈ͍ͨͳίʔυ͍ͬͺ͍͋Δ

59. AWS-SDK-Go ಠࣗRetryerΛఆٛ͢Δ

60. AWS-SDK-Go WithRetryerͰɺΧελϜϦτϥΠϠʔΛઃఆͯ͋͛͠Δɻ ʢcontextύοέʔδͷWithValueతͳʣ

61. AWS-SDK-Go Requestੜ੒࣌ʹ΋RetryerΛҾ͖౉͢

62. AWS-SDK-Go ϦτϥΠՄೳ͔Ͳ͏͔ΛνΣοΫͯ͠ɺtrueͳΒΧ΢ϯτ૿Ճ

63. AWS-SDK-Go νΣοΫޙɺϦτϥΠՄೳͳΒϦΫΤετ಺༰Λίϐʔ

64. ୅ସखஈ • ಠࣗRetryerΛఆٛͤͣʹɺcenkalti/backoffͳͲͷ exponential backoff࣮૷Λ࢖͏ํ๏΋͋Γ

65. ྫʣgo-datadog-api

66. Paging

67. Paging • ϖʔδϯά༻ͷAPIΛ࡞ΔͷͰ͸ͳ͘ɺ
 ϖʔδϯάରԠͷAPIʹɺ೚ҙͷϖʔδ·Ͱ͸
 ϦΫΤετΛ౤͛ଓ͚͍ͨ • ྫʣEC2ύϥϝʔλʔετΞʹอଘ͞Εͨ
 ΩʔͷҰཡΛऔಘ͍ͨ͠ɻ͕ɺ
 10ݸ͔ͣͭ͠ฦͬͯ͜ͳ͍…

68. AWS-SDK-Go • ࠶Ninja

69. Paging request.Paginationʂʂʂʂ ϦΫΤετͷstructʹ߹Θͤͯɺϖʔδϯάॲཧͷ࣮૷Λ࣋ͭ

70. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ

71. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ Ҿ਺͕௕͗͢ΔͷͰݟ͑ͳ͍͕ɺ
 ίʔϧόοΫ͕ઃఆͯ͋ͬͯ͠ɺ
 ݁Ռͱʮ࠷ޙͷϖʔδ͔ʯ
 ͷϑϥάΛฦ͢

72. ·ͱΊ

73. ·ͱΊ • ੈͷதʹ͸৭ʑͳAPI Client͕͍Δ
 ΈΜͳҧͬͯΈΜͳ͍͍ • ඞཁͳΒΧελϜClientɺͳ͍͠ϦΫΤετ
 ؅ཧ༻ͷstructΛ࣮૷͢Δ • aws-sdk-go͸͕͢͞ʹ΍Γ͗͢ͳͷͰɺ


    ͪΐ͏Ͳ͍͍ϋϯυϥΛ࣮૷͠·͠ΐ͏

74. Thank you!