マイクロチームでの高速な新規開発を支える開発・分析基盤 by @__timakin__ Builderscon2017でのプレゼン内容です。
ϚΠΫϩνʔϜͰͷߴͳ৽ن։ൃΛࢧ͑Δ։ൃɾੳج൫@__timakin__builderscon 2017
View Slide
ࣗݾհ• timakinʢಡΈ: ͪ·͖Μʣ• https://github.com/timakin• https://twitter.com/__timakin__• Gunosy৽نࣄۀ։ൃࣨ• Contributor of Golang <- New!• Go/SwiftΛϝΠϯʹॻ͍͍ͯ·͢ɻ
Copyright© Gunosy Inc. All Rights Reserved 3Go / Python ΤϯδχΞืूத▶https://gunosy.co.jp/recruit/Gunosyɺ౦ژେֶʹ௨͏3ਓͷֶੜͷ ʮใΛੈքதͷਓʹ࠷దʹಧ͚͍ͨʯͱ͍͏͍͔Β࢝·Γ·ͨ͠ɻ౦ূϚβʔζ্ɺຊώϧζͷΦϑΟεҠసΛܦͯɺ େ͖͍ͯ͘͠ΔձࣾͰ׆༂͍ͨ͠ϝϯόʔΛืू͍ͯ͠·͢ɻ
ࠓ͢͜ͱ• ʮLUCRAʯॳظϦϦʔε·ͰͷಓͷΓͱ੍݅• ͍։ൃظؒͰͬͨ͜ͱɺΒͳ͔ͬͨ͜ͱ• ։ൃج൫ͷհ• ॊೈͳԾઆݕূ͕Մೳͳڥͮ͘Γ
ݶΒΕͨϦιʔεԼͰͷ ߴͳ։ൃख๏খ͍͞νʔϜͰPDCAΛ ճͤΔੳج൫
ݶΒΕͨϦιʔεԼͰͷ ߴͳ։ൃख๏
࣮ྫɿʮLUCRAʯ
ʮLUCRAʯॳظϦϦʔε·ͰͷಓͷΓͱ੍݅
εέδϡʔϧͱ੍݅• εέδϡʔϧ• 3ϲ݄ؒ (3݄~5݄)• ։ൃνʔϜͷن• ৽نϓϩδΣΫτۃྗগͰʂ• ࣄͰGoͷAPIɺSwiftͰͷiOSΞϓϦ։ൃܦݧͳ͠
εέδϡʔϧͱ੍݅• ։ൃཁ݅• iOSΫϥΠΞϯτ
εέδϡʔϧͱ੍݅• ։ൃཁ݅• iOSΫϥΠΞϯτ• هࣄɺάϧʔϐϯάใɺϢʔβʔೝূଞϦιʔεશମʹؔΘΔAPI
εέδϡʔϧͱ੍݅• ։ൃཁ݅• iOSΫϥΠΞϯτ• هࣄɺάϧʔϐϯάใɺϢʔβʔೝূଞϦιʔεશମʹؔΘΔAPI• هࣄੜɺػցֶशج൫• Ϋϩʔϥʔ / ίϯςϯπྨ / هࣄςϯϓϨੜAPI
εέδϡʔϧͱ੍݅• ։ൃཁ݅• iOSΫϥΠΞϯτ• هࣄɺάϧʔϐϯάใɺϢʔβʔೝূଞϦιʔεશମʹؔΘΔAPI• هࣄੜɺػցֶशج൫• Ϋϩʔϥʔ / ίϯςϯπྨ / هࣄςϯϓϨੜAPI• पลڥ• Ωϟογϡػߏ / Τϥʔ, syslogࢹ / ཧπʔϧ• Pushج൫ / ϩάੳج൫ / ABςετج൫
εέδϡʔϧͱ੍݅• ϙΠϯτ1. νʔϜͷਓۃྗ͑Δɻ2. ߏங͢Δج൫ʹରͯ͠։ൃظ͕͍ؒɻ3. ੳ͕Ͱ͖ΔڥΛ࠷ॳ͔Βߏங͢Δɻ
࣮ࡍʹԿΛ͔ͬͨʁ
࣮ࡍͷεέδϡʔϧ ݄• API• ϞοΫඋʢεϐʔυײʹ߹Θͣ1͘Β͍͔͠ʹཱͨͣʣ• هࣄɺάϧʔϐϯάใAPIͷ࡞ + stgڥߏங• ΫϥΠΞϯτ• UtilɺAPIClientɺLoggingɺϧʔςΟϯάपΓͷඋ• ػցֶशɺهࣄੜج൫• Ϋϩʔϥʔ࡞ + stgʹॾʑͷΠϯϑϥߏஙԼճΓͷඋͱϓϩτλΠϓ࡞
࣮ࡍͷεέδϡʔϧ ݄• API• ϢʔβʔೝূAPIͷ࡞• ΫϥΠΞϯτͭͳ͗͜Έ• push, ABج൫, Ωϟογϡؔ࿈ͷௐ, ϩάج൫ͷ࡞• ΫϥΠΞϯτ• هࣄҰཡɺৄࡉը໘σβΠϯͯࠐΈ• ೝূɺpushɺར༻نɾFAQ֤छઃఆɺڧ੍Ξοϓσʔτ• ػցֶशɺهࣄੜج൫• هࣄྨɺΫϩʔϥʔɺؔ࿈هࣄੜͷ࡞ɺstgڥߏங࣮٧Ί + σβΠϯͯࠐΈ
࣮ࡍͷεέδϡʔϧ ݄• API• ϩάج൫ɺϛυϧΣΞ(Τϥʔ, syslogࢹɺmemd)ͷsetup• ֤छهࣄࣗಈੜج൫ɺϩάج൫ͱͷͭͳ͗͜Έ• ຊ൪ڥߏங• ΫϥΠΞϯτ• σβΠϯϒϥογϡΞοϓɺจݴɾಈ࡞मਖ਼ɺਃ࡞ۀ• ػցֶशɺهࣄੜج൫• ྨɺείΞ͚ϩδοΫͷमਖ਼ɺଞAPIͷमਖ਼ɺ֤छຊ൪ڥߏஙϛυϧΣΞՃɺ֤APIͭͳ͗͜Έɺຊ൪ڥߏங
ॕʂϦϦʔεʂ
͜ͷظؒͷ։ൃνʔϜͷঢ়گ
աࠅͳεέδϡʔϧ
աࠅͳεέδϡʔϧۀଓ͖ͷʑ
աࠅͳεέδϡʔϧۀଓ͖ͷʑߥΕΔਓؒؔ
ʁ
ΪϦΪϦ͚ͩͲΦϯεέ
ΪϦΪϦ͚ͩͲΦϯεέఆ࣌Ͱࣦྱ
ΪϦΪϦ͚ͩͲΦϯεέఆ࣌Ͱࣦྱ࿕Β͔ͳ ৬
• Δ / Βͳ͍ͷΓ͚• ج൫෦ͷ๛ͳϊϋ
͍։ൃظؒͰͬͨ͜ͱɺΒͳ͔ͬͨ͜ͱ
Δ/Βͳ͍ͷج४• ΞϓϦͷίϯηϓτΛ࠷খݶͷൣғͰ ݕূ͢Δͷʹඞཁ͔
Δ/Βͳ͍ͷج४• ΞϓϦͷίϯηϓτΛ࠷খݶͷൣғͰ ݕূ͢Δͷʹඞཁ͔• ඦສDAUʹεέʔϧ͢Δମ੍͕͏͔
Δ/Βͳ͍ͷج४• ΞϓϦͷίϯηϓτΛ࠷খݶͷൣғͰ ݕূ͢Δͷʹඞཁ͔• ඦສDAUʹεέʔϧ͢Δମ੍͕͏͔• ੳɾӡ༻ɾվળͷϑϩʔΛΓͳ͘౿ΉͨΊͷ ػೳ͕ἧ͍ͬͯΔ͔
• هࣄྨɾੜج൫• ੳج൫• ೝূɺ هࣄͷҰཡɺӾཡɺ γΣΞ• ※৹ࠪʹ͍+αΔ Βͳ͍• Android• ݫີͳϦϑΝΫλ• “ߴͰແବͷͳ͍”ج൫• ݕࡧ• ϑΥϩʔɾ͓ؾʹೖΓ• Ӿཡཤྺػೳ
ඞཁͳͷهࣄ͕ݟΒΕͯɺ ݕূɺӡ༻Մೳͳج൫Λ࡞Δ͜ͱ
طଘϓϩμΫτ͔Β Ԡ༻Ͱ͖Δ෦Λ βΫβΫऔͬͯ͜Α͏
Πϯϑϥ
Πϯϑϥߏஙฤ• جຊతʹAWS OpsworksͰɺࣗࣾͷΧελϜChefϨγϐΛϕʔεʹڥߏங͢Δ• gunosy/gunosy_opsworks_cookbook
Πϯϑϥߏஙฤ
Πϯϑϥߏஙฤ• middleware• papertrail, td_agent, datadog, nginx, memcached• ଞɺmysql, jq, einhornʢGoͷAPIͷgraceful restart ༻్ʣͳͲΛΆͪΆͪೖΕΔ
Πϯϑϥߏஙฤ• codenize-tools• miamʢIAMཧʣ• piculetʢSGʣ• roadworkerʢRoute53ʣ• radiosondeʢCloudWatch Logsʣ• monosasiʢCloudWatch Eventsʣ
git pushͨ͠ΒCircleCI্Ͱ֤ίϚϯυΛୟ͖ɺSlackʹapply݁ՌͷJSONϖΠϩʔυΛ௨
API
ͱ͜ΖͰ
GunosyGoͷར༻ྺ͕͍ʂ
ྺ࢙͕ੵΈॏͳΔͱ ੜ·ΕΔͷ
ศརͰझ͕͋ΔUtility܈
gunosy/go
APIฤ• gunosy/go• mysql, redis, memd, fluent, airbrakeͳͲ ϛυϧΣΞΫϥΠΞϯτ܈͕ೖͬͯΔ• jwt authenticator, fluent-loggerͳͲɺ ֤APIͰඞཁʹͳΔutilΛ·ͱΊͯΔ• OpsWorks࠷దͳઃఆಡΈࠐΈ
APIฤ• gunosy/goͷϝϦοτ / σϝϦοτ• ϝϦοτ• OpsWorksͰڥߏஙͯ͠deploy͢Εɺ API͕͍͍͙ͩͨ͢ಈ͘Α͏ʹͳͬͯΔɻ• JWTೝূͳͲ΄΅࣮ʹࠩҟ͕ͳ͍ͷͰɺ ڞ௨ϥΠϒϥϦʹ͢ΔϝϦοτ͕େ͖͍• ӡ༻ɺࢹपΓͷઃఆ͕OpsWorksͱͷ๊͖߹ΘͤͰ ͔ͳΓγϣʔτΧοτͰ͖Δ
APIฤ• gunosy/goͷϝϦοτ / σϝϦοτ• σϝϦοτ• ϛυϧΣΞ܈ɺͻ͍ͯ AWSͷґଘ͕ߴ͍ʂ• loggerͷܗࣜมߋͳͲ͕ॊೈʹͰ͖ͳ͍
APIฤ• पลπʔϧ• SQLεΩʔϚgooseʢGomigration toolʣ Ͱཧ• graceful restart༻ʹɺeinhornͰϓϩηεཧ
APIฤ• ϑϨʔϜϫʔΫ• guregu/kami
APIฤ• guregu/kamiͷ• ContextʢGoͷrequest-scopedͳΛೖΕΔശΈ͍ͨͳͭʣΛ݁ߏཞ༻͍ͯ͠Δɻ• ΰουΦϒδΣΫτ • panicΛѲΓͭͿͯ͘͠ΕΔ
APIฤ• ύοέʔδཧ• ࠷ۙݫີʹͳΓ࢝Ίͨ• աڈؾ߹͍ Ͱ࠷৽ʹ͍͔͍ͭͤͯͨ
APIฤ• ֤ͷղফʹ͚ͯ• AWS OpsWorksͷґଘͷߴ͞• ϑϨʔϜϫʔΫΛguregu/kami -> net/httpʹҠߦͯ͠ɺϛυϧΣΞ܈ϞσϧͷґଘΛղফɻ• DIύλʔϯͰॻ͘͜ͱͰɺͳΜͰglobalʹ٧Ί͍ͯͨॻ͖ํ͔Β٫
APIฤ• ֤ͷղফʹ͚ͯ• όʔδϣϯཧ• ґଘύοέʔδglideʹҠߦͯ͠ɺ దͳόʔδϣϯཧΛɻ
APIฤ• σϓϩΠϑϩʔ• git push• S3ʹGoͷόΠφϦ੩తϑΝΠϧΛ٧ΊͨtarΛ Ξοϓϩʔυ• OpsWorksͷAPIΛୟ͍ͯS3͔ΒtarΛ͖࣋ͬͯͯ ల։ɺrestart͢Δ
ref: https://speakerdeck.com/kyokomi/gunosy-apitimugokai-fa-huro
APIฤ• CI• ॳظ։ൃͰϏϧυʹ࣌ؒΛऔΒΕΔͷѹతແବ• => CircleCI2.0ରԠʂ
APIฤ• ײಈͷཛྷ• goͷvendor, npm install assets:precompileͳͲ Ωϟογϡ͢Ε ͍͍ͩͨ1~2minsͰऴΘΔ
APIฤ• Ϋϩʔϥʔɺྨ• GunosyͰഓͬͨػցֶशج൫ϊϋΛϑϧ׆༻• ϑϨʔϜϫʔΫ: Django• δϣϒεέδϡʔϥ: Celery• σʔλߏ: GunosyFeed Ver.2• ref: https://www.slideshare.net/mosa_siru/ss-65205273
APIฤ• هࣄྨɺੜʹඞཁͳAPI܈• Crawler• GunosyFeedʹଇͬͨAPIΛୟ͍ͯσʔλΛऔͬͯ͘Δ• HTMLαχλΠζɺϑΟʔϧυใͷύʔαʔػߏ• CategoryClassifier• هࣄͷΧςΰϦྨث• URL(υϝΠϯ)ʹΑΔϧʔϧϕʔεͷྨ + ػցֶशϞσϧʹΑΔྨ• NGఆ• TabGenerator• ಛఆΧςΰϦ(λϒ)ͷهࣄҰཡΛੜ• ElasticSearchʹੵͨ͠ӾཡɾΫϦοΫϩά͔ΒείΞϦϯάͯ͠อଘ• खಈϐοΫΞοϓͷӡ༻͚API
APIฤ• Ωϟογϡ• ϩʔΧϧͱϦϞʔτͷ̎ஈ֊• memcached• هࣄɺݕࡧ݁ՌɺϚελʔσʔλͷΈΩϟογϡ
ref: https://www.slideshare.net/mosa_siru/go-memcached-microservices
APIฤ• Pushج൫• mercari/gaurunΛfork(gunosy/gaurun)• gaurunͷόΠφϦΛS3ʹஔ͍ͱ͍ͯɺ OpsWorksͰdeploy• OpsWorksͷCustom JSONͷઃఆΛ gaurunͷઃఆϑΝΠϧʹஔ͖͑ΔϨγϐ
ʂʂ
"1/T($1 '$.AdminͰPush༧Push APIGaurun APIBulk Publish௨ड৴Ұ࿈ͷ௨ϑϩʔ
ΞʔΩςΫνϟ
ΞʔΩςΫνϟฤ• ϚΠΫϩαʔϏεʹ͗͢͠ͳ͍
ref: https://www.slideshare.net/mosa_siru/ss-64839846
ΞʔΩςΫνϟฤ• ͳͥϚΠΫϩαʔϏεʹ͠ͳ͍ʁ• APIؒͷΦʔόʔϔου
ΞʔΩςΫνϟฤ• ͳͥϚΠΫϩαʔϏεʹ͠ͳ͍ʁ• APIؒͷΦʔόʔϔου• SGઃఆͳͲͷఔʹΑΔ։ൃͷԼ
ΞʔΩςΫνϟฤ• ͳͥϚΠΫϩαʔϏεʹ͠ͳ͍ʁ• APIؒͷΦʔόʔϔου• SGઃఆͳͲͷఔʹΑΔ։ൃͷԼ• σϓϩΠ͕͠ΜͲ͍
ΞʔΩςΫνϟฤ• ͳͥϚΠΫϩαʔϏεʹ͠ͳ͍ʁ• APIؒͷΦʔόʔϔου• SGઃఆͳͲͷఔʹΑΔ։ൃͷԼ• σϓϩΠ͕͠ΜͲ͍• ཧը໘ͷͨΊʹര͢ΔCRUD API܈
ΞʔΩςΫνϟฤ• LUCRAͰϚΠΫϩαʔϏεʹ͗͢͠ͳ͍• هࣄऩूपΓ• ྨɺΫϩʔϥʔɺهࣄੜݸผʹɻ• Client͚ͷAPI• هࣄɺάϧʔϐϯάใɺϢʔβʔใϝΠϯAPI͕શ෦ฦ͢• PushɺABςετཧAPI͚ͩݸผʹɻ
iOS
iOSΫϥΠΞϯτฤ• MVVM + Reactive• Extension, APIClient, LoggerपΓͷUtil࠶ར༻Մೳ
iOSΫϥΠΞϯτฤ• ϩάૹ৴पΓͷίʔυ͕ଟ͍ -> σόοάେม• LogDebugger͕͋ΔͷͰϩά͕ૹΒΕ͔ͨΛͦͷͰσόοάͰ͖Δ• aikizoku/SKLogDebugger-iOS
iOSΫϥΠΞϯτฤ• ίϯύΠϧ࣌ؒͷݮ• Ϗϧυͪແҙࣝͷ͏ͪʹ૿͍͑͢ແବͳ࣌ؒ• -Xfrontend -debug-time-function-bodies ্هϑϥάͰɺϝιου୯ҐͷίϯύΠϧ࣌ؒΛܭଌ• giginet/xcprofilerͰίϯύΠϧ͕͍ॱʹ֬ೝ
ms୯ҐͰϝιου͝ͱͷίϯύΠϧ࣌ؒܦաͷ্ҐΛදࣔͯ͘͠ΕΔɻ
iOSΫϥΠΞϯτฤ• ίϯύΠϧ࣌ؒͷݮ• 50msҎ্ͷϝιου͕૿͑ͨΒվળ• giginet/danger-xcprofilerͰΞϥʔτ௨Մೳ
iOSΫϥΠΞϯτฤ• ىಈ࣌ؒͷݮ• DYLD_PRINT_STATISTICSϑϥάͷઃఆ• ґଘϥΠϒϥϦͷ੍• ܕਪͷݮ• SDKͷηοτΞοϓͷฒྻԽ
։ൃڥ·ͱΊ• Infrastructure as Codeపఈ & apply݁ՌͷՄࢹԽ• एׯͷґଘɺటष͍࣮ڐ༰ͭͭ͠ɺ AWS࠷దͳڥʹ͔ͬΔ͜ͱͰΛग़ͯ͠Δ• ࠷ॳ͔ΒΞʔΩςΫνϟΛϚΠΫϩʹ͗͢͠ͳ͍• ίϯύΠϧɺCIͳͲͷ܁Γฦ͠࡞ۀແବͳ࣌ؒΛ ա͝͞ͳ͍Α͏ʹɺࣗಈԽ & ߴԽ
খ͍͞νʔϜͰPDCAΛ ճͤΔੳج൫
࣮ࡍͲΜͳϩάΛ ૹ͍ͬͯΔ͔ʁ
͜ͷը໘ͰͲΜͳϩά͕ ൃੜ͠͏Δ͔ʁ
• launchΠϕϯτϩά• fore/backgroundϩά• push heartbeatϩά• ೝূ, initAPI callϩά• ॳظදࣔ࣌هࣄimpϩά• ΧςΰϦεϫΠϓભҠϩά• ΧςΰϦΫϦοΫભҠϩά• εΫϩʔϧ࣌impϩά• هࣄΫϦοΫϩά• pull-refreshϩά• refresh࣌impϩά• push௨։෧ϩά
• launchΠϕϯτϩά• fore/backgroundϩά• push heartbeatϩά• ೝূ, initAPI callϩά• ॳظදࣔ࣌هࣄimpϩά• ΧςΰϦεϫΠϓભҠϩά• ΧςΰϦΫϦοΫભҠϩά• εΫϩʔϧ࣌impϩά• هࣄΫϦοΫϩά• pull-refreshϩά• refresh࣌impϩά• push௨։෧ϩά͍ͬͺ͍
Ͳ͏͍͏ϑϩʔͰ ૹ͍ͬͯΔ͔ʁ
ੳσʔλϑϩʔ
ੳσʔλϑϩʔొɾӾཡɾ ΫϦοΫϩάͷऩू
ੳσʔλϑϩʔొɾӾཡɾ ΫϦοΫϩάͷऩूtd_agentʹΑΔ ϩάͷू
ੳσʔλϑϩʔొɾӾཡɾ ΫϦοΫϩάͷऩूtd_agentʹΑΔ ϩάͷूϩάछผ͝ͱʹ όοΫΞοϓ & ྲྀ͠ࠐΈ
ੳσʔλϑϩʔొɾӾཡɾ ΫϦοΫϩάͷऩूtd_agentʹΑΔ ϩάͷूϩάछผ͝ͱʹ όοΫΞοϓ & ྲྀ͠ࠐΈ Redash্Ͱੳ
• ΫϥΠΞϯτ͔Βͷϩάcookpad/puree-ios Λ௨ͯ͡redshiftʹૹΔ• td_agent͕ೖͬͨlogूαʔόʔΛհͯ͠ɺ ՃࡁΈσʔλΛRedashʹྲྀ͢• όοΫΞοϓͷͨΊʹkinesisΛڬΉੳσʔλϑϩʔ
࣮ࡍͷLogఆٛ (iOS)
Ͳ͏ͬͯσʔλΛݟΔ͔ʁ
ੳڥ• RedashΛར༻• ओཁKPIఆظతʹSlackʹ௨• ӡ༻ɺ։ൃΘͣνʔϜશһ͕ੳ༻SQLΛॻ͘
ͲΜͳΛ͍ͬͯΔ͔ʁ
ओཁKPI• DAU• 14ޙܧଓ• 7ޙܧଓ• 1ޙܧଓ• ۙ1࣌ؒهࣄCTR• ొܦ࿏ผ֫ಘ• هࣄΫϦοΫճ/DAU• εϫΠϓճ/DAU• ͓ؾʹೖΓొ/DAU• શମ / ొܦ࿏ผίϗʔτ• Pushڐঢ়گ• ΧςΰϦผهࣄӾཡ
ੳ݁Ռͷར༻• ʮ͓͠ΌΕͳΞϓϦʯͷϒϥϯυΠϝʔδΛٻ͍͔ͨ͠Βͱ͍ͬͯɺ ࠜڌͳ͘ʮϑΝογϣϯʯʮίεϝʯΛਪ͢ ͱ͍͏ҙࢥܾఆઈରʹ͠ͳ͍ɻ• ΧςΰϦผͷσʔλΛݟͯɺߴ͍CTRRR͕ ༧͞ΕΔPushػೳՃ͔͠ߦΘͳ͍ɻ
·ͱΊ• iOSΫϥΠΞϯτ͔Βܗͭͭ͠ϩάΛૹ৴• ը໘୯ҐɺΞΫγϣϯ୯ҐͰऔΕΔ͚ͩͷϩάΛऔΔ• ੳπʔϧRedashΛ༻͍͓ͯΓɺνʔϜશମͰ SQLͷϝϯςφϯεΛߦ͏• σʔλͰ֬৴͕࣋ͯͳ͍ࢪࡦۃྗڐ͞ͳ͍
։ൃख๏·ͱΊ• Infrastructure as Codeపఈ & apply݁ՌͷՄࢹԽ• एׯͷґଘɺటष͍࣮ڐ༰ͭͭ͠ɺ AWS࠷దͳڥʹ͔ͬΔ͜ͱͰΛग़ͯ͠Δ• ࠷ॳ͔ΒΞʔΩςΫνϟΛϚΠΫϩʹ͗͢͠ͳ͍• ίϯύΠϧɺCIͳͲͷ܁Γฦ͠࡞ۀແବͳ࣌ؒΛ ա͝͞ͳ͍Α͏ʹɺࣗಈԽ & ߴԽ
ੳج൫·ͱΊ• iOSΫϥΠΞϯτ͔Βܗͭͭ͠ϩάΛૹ৴• ը໘୯ҐɺΞΫγϣϯ୯ҐͰऔΕΔ͚ͩͷϩάΛऔΔ• ੳπʔϧRedashΛ༻͍͓ͯΓɺνʔϜશମͰ SQLͷϝϯςφϯεΛߦ͏• σʔλͰ֬৴͕࣋ͯͳ͍ࢪࡦۃྗڐ͞ͳ͍
͝੩ௌ༗͏͍͟͝·ͨ͠ʂ