Advanced API Calling in Go

Go 1.9 Release Party in Tokyo での発表資料です by @__timakin__

1. Advanced API Calling in Go @__timakin__ / Go 1.9 Release Party in Tokyo
2. ࣗݾ঺հ • Github: timakin / Twitter: @__timakin__ • ŧŔŕŪ Go Ŭ Contribute ŢŦ • Gunosy ৽نࣄۀ։ൃࣨ ॴଐ • Go / Swift • Recently Contributed / Created • giginet/xcprofiler • mercari/gaurun • timakin/gopli • timakin/ssm2env • timakin/md2mid • timakin/gonvert
3. Copyright© Gunosy Inc. All Rights Reserved 3 Go / Python ΤϯδχΞืूத ▶https://gunosy.co.jp/recruit/ Gunosy͸ɺ౦ژେֶʹ௨͏3ਓͷֶੜͷ
 ʮ৘ใΛੈքதͷਓʹ࠷దʹಧ͚͍ͨʯͱ͍͏૝͍͔Β࢝·Γ·ͨ͠ɻ ౦ূϚβʔζ্৔ɺ࿡ຊ໦ώϧζ΁ͷΦϑΟεҠసΛܦͯɺ
 େ͖͘੒௕͍ͯ͠ΔձࣾͰ׆༂͍ͨ͠ϝϯόʔΛืू͍ͯ͠·͢ɻ
4. ࠓ೔ͷςʔϚ
5. GoͰAPI Client࡞Δͱ͖ͷ ΧελϚΠζྫ
6. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ
7. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ
8. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ
9. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ
10. ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ
11. ௨ৗͷAPIϦΫΤετ
12. ௨ৗͷϦΫΤετ
13. ಠࣗAPI Client
14. ಠࣗͷAPI ClientΛఆٛ͢Δ
15. ಠࣗͷAPI ClientΛఆٛ͢Δ API Clientͷߏ଄ମΛఆٛ͢Δ
16. ಠࣗͷAPI ClientΛఆٛ͢Δ ίϯετϥΫλΛॻ͘
17. ಠࣗͷAPI ClientΛఆٛ͢Δ ڞ௨ϝιουΛఆٛ͢Δ
18. ಠࣗͷAPI ClientΛఆٛ͢Δ Ϧιʔε͝ͱͷAPI ίʔϧΛఆٛ͢Δɻ ͜ͷล͸௨ৗͷrepository૚Ͱ΍ͬͯΔ͜ͱͱ
 ͦ͜·ͰมΘΒͳ͍ɻ
19. goroutine
20. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ
21. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env
22. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ
23. goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳ਺ճAPIίʔϧ
 ͢Δ৔߹͸΋ͪΖΜgoroutine࢖͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter Store)ʹอଘ͞Ε͍ͯΔ
 ಛఆͷKeyʹͻ΋ͮ͘ValueΛऔ͖ͬͯͯɺ
 ݁ՌΛmapʹೖΕ͍ͨ • Ͱ΋10ݸ͔ͣͭ͠औಘͰ͖ͳ͍

24. goroutineͰͷฒߦcall AWS ClientΛ࣋ͬͨߏ଄ମΛఆٛ͢Δ
25. goroutineͰͷฒߦcall AWS SSMʢύϥϝʔλετΞʣ͔Β
 औͬͯདྷͨValueΛmapʹͯ͠ฦ͢ϝιουΛఆٛɻ
26. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ
27. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ
28. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར
29. goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛ଴ͪड͚Δchannelʢࠓճ͸”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹ෼ׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ
 ɹɾ͜͏͍͏෼ׂ΋channelܦ༝Ͱ݁ՌΛड͚औΔͱ
 ɹɹͦͷ··rangeʹ౉ͤͯศར ɾeg.Go಺෦ͰฒߦॲཧΛߦ͏
30. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢
31. goroutineͰͷฒߦcall ɾ੒ޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖౉͢ ɾcontextͷλΠϜΞ΢τ௨஌Λड͚औͬͨΒͦͷ࣌఺Ͱऴྃ
32. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ
33. goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ ɾchannelܦ༝Ͱؼͬͯདྷͨ݁ՌΛॱ࣍mapʹ٧ΊΔ
34. goroutineͰͷฒߦcall x/sync/errgroupͰ
 ɾContextͷϋϯυϦϯά
 ɾϦΫΤετࣗମͷΤϥʔॲཧ
 ɾgoroutineʹΑΔฒߦϦΫΤετ
 Λ؅ཧ͢Δɻ
35. Configuration
36. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ
37. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ
38. ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ΋͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ • ྫʣmercari/gaurun
39. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢
40. ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮૷Λ࣋ͬͨߏ଄ମ)ʹɺ ɹTimeout΍ConnͷઃఆΛ౉͢ ɾhttp.ClientͷϑΟʔϧυʹTransportΛ౉ͯ͠ΫϥΠΞϯτੜ੒
41. Proxy
42. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ
43. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠
44. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ
45. Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖׵͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹ੾Γସ͍͑ͨ • ྫʣkazeburo/chocon
46. Proxy
47. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε
48. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ
49. Proxy • GoͷHTTPϦΫΤετͷߏ੒ཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport • http.RoundTripper Λ࣮૷ͨ͠ߏ଄ମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮૷Λ࣋ͭ • http.Client • http.Transport Λ಺෦ʹ࣋ͬͨɺHTTPϓϩτίϧͰͷ௨ ৴Λ͓͜ͳ͏ͨΊͷૉͷΫϥΠΞϯτ࣮૷ • func (c *Client) Do(…) ͸தͰRoundTripΛݺΜͰ͍Δ
50. Proxy • kazeburo/chocon಺Ͱ͸…
51. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ɺRoundTripperΛ࣋ͬͨߏ଄ମ
52. Proxy ϦΫΤετ಺༰ͷॻ͖׵͑ʢHostͷஔ׵ɺBodyͷcopyʣ
53. Proxy ɾServeHTTPΛ࣮૷ ɾProxy͕ड͚औͬͨresponseΛɺAPIClient޲͚ʹcopyͯ͠
 ɹฦ٫ͯ͋͛͠Δ
54. Retry
55. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠
56. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go
57. Retry • ϦΫΤετʹࣦഊͯ͠΋ɺࢦఆճ਺·Ͱ͸
 ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go
58. AWS-SDK-Go • NinjaΈ͍ͨͳίʔυ͍ͬͺ͍͋Δ
59. AWS-SDK-Go ಠࣗRetryerΛఆٛ͢Δ
60. AWS-SDK-Go WithRetryerͰɺΧελϜϦτϥΠϠʔΛઃఆͯ͋͛͠Δɻ ʢcontextύοέʔδͷWithValueతͳʣ
61. AWS-SDK-Go Requestੜ੒࣌ʹ΋RetryerΛҾ͖౉͢
62. AWS-SDK-Go ϦτϥΠՄೳ͔Ͳ͏͔ΛνΣοΫͯ͠ɺtrueͳΒΧ΢ϯτ૿Ճ
63. AWS-SDK-Go νΣοΫޙɺϦτϥΠՄೳͳΒϦΫΤετ಺༰Λίϐʔ
64. ୅ସखஈ • ಠࣗRetryerΛఆٛͤͣʹɺcenkalti/backoffͳͲͷ exponential backoff࣮૷Λ࢖͏ํ๏΋͋Γ
65. ྫʣgo-datadog-api
66. Paging
67. Paging • ϖʔδϯά༻ͷAPIΛ࡞ΔͷͰ͸ͳ͘ɺ
 ϖʔδϯάରԠͷAPIʹɺ೚ҙͷϖʔδ·Ͱ͸
 ϦΫΤετΛ౤͛ଓ͚͍ͨ • ྫʣEC2ύϥϝʔλʔετΞʹอଘ͞Εͨ
 ΩʔͷҰཡΛऔಘ͍ͨ͠ɻ͕ɺ
 10ݸ͔ͣͭ͠ฦͬͯ͜ͳ͍…
68. AWS-SDK-Go • ࠶Ninja
69. Paging request.Paginationʂʂʂʂ ϦΫΤετͷstructʹ߹Θͤͯɺϖʔδϯάॲཧͷ࣮૷Λ࣋ͭ
70. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ
71. Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ੒͢Δ Ҿ਺͕௕͗͢ΔͷͰݟ͑ͳ͍͕ɺ
 ίʔϧόοΫ͕ઃఆͯ͋ͬͯ͠ɺ
 ݁Ռͱʮ࠷ޙͷϖʔδ͔ʯ
 ͷϑϥάΛฦ͢
72. ·ͱΊ
73. ·ͱΊ • ੈͷதʹ͸৭ʑͳAPI Client͕͍Δ
 ΈΜͳҧͬͯΈΜͳ͍͍ • ඞཁͳΒΧελϜClientɺͳ͍͠ϦΫΤετ
 ؅ཧ༻ͷstructΛ࣮૷͢Δ • aws-sdk-go͸͕͢͞ʹ΍Γ͗͢ͳͷͰɺ
 ͪΐ͏Ͳ͍͍ϋϯυϥΛ࣮૷͠·͠ΐ͏
74. Thank you!