1000万ダウンロードアプリ『メルカリ』を支える技術

Slide 1

Slide 1 text

1000ສμ΢ϯϩʔυΞϓϦ ʰϝϧΧϦʱ Λࢧ͑Δٕज़ גࣜձࣾϝϧΧϦ ΤϯδχΞ ٱอୡ඙ bokko@mercariapp.com dots. summit 2015

Slide 2

Slide 2 text

ࣗݾ঺հ • ٱอୡ඙(bokko) • @cubicdaiya • Software Engineer in Infrastructure Engineering • OSS Contributor • e.g. nginx, mruby, ngx_mruby, ngx_lua, h2o, etc…

Slide 3

Slide 3 text

ʰϝϧΧϦʱʹ͍ͭͯ

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

ओཁKPIσʔλ μ΢ϯϩʔυ਺ ߪೖֹۚ ग़඼਺ ਺஋ 1000ສDLಥഁ ݄ؒ਺ेԯԁ 1೔਺ेສ඼Ҏ্ ΞϓϦΛެ։͔ͯ͠Β໿1೥ͱ൒೥Ͱ 1000ສμ΢ϯϩʔυୡ੒ʂ(2013೥7݄ެ։)

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

ʰϝϧΧϦʱΛࢧ͑Δ̋̋

Slide 8

Slide 8 text

τϐοΫ • ϝϧΧϦΛࢧ͑Δ։ൃମ੍ • ϝϧΧϦΛࢧ͑ΔΠϯϑϥ • ϝϧΧϦΛࢧ͑ΔαϒγεςϜ • ϝϧΧϦΛࢧ͑ΔGo

Slide 9

Slide 9 text

τϐοΫ • ϝϧΧϦΛࢧ͑Δ։ൃମ੍ • ϝϧΧϦΛࢧ͑ΔΠϯϑϥ • ϝϧΧϦΛࢧ͑ΔαϒγεςϜ • ϝϧΧϦΛࢧ͑ΔGo

Slide 10

Slide 10 text

ʰϝϧΧϦʱΛࢧ͑Δ։ൃମ੍ • ΤϯδχΞ໿20ਓ(QAɺCS෦໳΋͋Δ) • ΠϯϑϥνʔϜ͸ࣗ෼ؚΊͯ5ਓ • ৭ΜͳόοΫΤϯυΛ࣋ͭΤϯδχΞ͕ू·ͬͯ·͢ • Ϛωʔδϟ৬΍্ڃΤϯδχΞ৬΋͋Γ·͕͢ɺϑϥοτͳ งғؾͰ͢

Slide 11

Slide 11 text

ʰϝϧΧϦʱΛࢧ͑Δ։ൃମ੍ • Github্ͰPull Requestϕʔεͷ։ൃ • RedmineͰλεΫ؅ཧ • Slackϑϧ׆༻

Slide 12

Slide 12 text

ʰϝϧΧϦʱΛࢧ͑Δ։ൃମ੍ • Qiita:TeamͰυΩϡϝϯτڞ༗ • Travis-CIʹΑΔܧଓతΠϯςάϨʔγϣϯ • Ansible PlaybookʹΑΔϓϩϏδϣχϯά • DockerʹΑΔݸਓ։ൃ؀ڥ(Vagrantͷਓ΋͍·͢)

Slide 13

Slide 13 text

τϐοΫ • ϝϧΧϦΛࢧ͑Δ։ൃମ੍ • ϝϧΧϦΛࢧ͑ΔΠϯϑϥ • ϝϧΧϦΛࢧ͑ΔαϒγεςϜ • ϝϧΧϦΛࢧ͑ΔPHP • ϝϧΧϦΛࢧ͑ΔGo

Slide 14

Slide 14 text

ʰϝϧΧϦʱΛࢧ͑ΔΠϯϑϥ • ͘͞Βͷઐ༻αʔό • ͘͞ΒΫϥ΢υ • Amazon Web Service • Google Cloud Platform • Treasure Data • etc…

Slide 15

Slide 15 text

ʰϝϧΧϦʱΛࢧ͑ΔΠϯϑϥ • αʔό୆਺͸100୆Ҏ্ • ΞϓϦ͸JP൛ͱUS൛͕͋Δ • JP൛͸͘͞ΒɺUS൛͸AWS্Ͱӡ༻ • ࡢ೥ͷޙ൒͔ΒJP൛͸͘͞Βͷઐ༻αʔό͕ϝΠϯ • ioDrive࢖ͬͯ·͢

Slide 16

Slide 16 text

τϐοΫ • ϝϧΧϦΛࢧ͑Δ։ൃମ੍ • ϝϧΧϦΛࢧ͑ΔΠϯϑϥ • ϝϧΧϦΛࢧ͑ΔαϒγεςϜ • ϝϧΧϦΛࢧ͑ΔGo

Slide 17

Slide 17 text

େࡶ೺ͳશମ૾

Slide 18

Slide 18 text

શ෦͓఻͑͢Δͷ͸ແཧͳͷͰ ͍͔ͭ͘঺հ͍͖ͯ͠·͢

Slide 19

Slide 19 text

ϩʔυόϥϯαʔ

Slide 20

Slide 20 text

OHJOY

Slide 21

Slide 21 text

nginx • APIϦΫΤετͷϩʔυόϥϯε • SSLλʔϛωʔγϣϯ & SPDYήʔτ΢ΣΠ • ίϯςϯπΩϟογϡ

Slide 22

Slide 22 text

HTTPS௨৴ͷߴ଎Խ • TLS Session Cache • TLS Session Tickets • SPDY

Slide 23

Slide 23 text

TLS Session Cache • TLSϋϯυγΣΠΫͷηογϣϯ৘ใΛαʔόʹ Ωϟογϡ • nginxͰ͸ڞ༗ϝϞϦ্ʹΩϟογϡ͞ΕΔ • ࣍ճͷTLSϋϯυγΣΠΫΛলུ • CPUϦιʔεͷ࡟ݮ΍ϨΠςϯγղফͷޮՌ͕͋Δ

Slide 24

Slide 24 text

TLS Session Cache with nginx

Slide 25

Slide 25 text

TLS Session Tickets • ҉߸Խͨ͠ηογϣϯ৘ใ(νέοτ)ΛΫϥ ΠΞϯτʹ౉͢ • νέοτΛݩʹTLSηογϣϯΛ࠶։ • HTTPSαʔόෳ਺୆ͰηογϣϯΩϟογϡ Λڞ༗Ͱ͖Δ • αϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺

Slide 26

Slide 26 text

TLS Session Tickets with nginx

Slide 27

Slide 27 text

SPDY • WebΛߴ଎Խ͢ΔͨΊͷϓϩτίϧ • nginx-1.5.10Ҏ߱ͰSPDY3.1͕ར༻Մ • ࠷৽ͷiOS΍AndroidͰ͸αʔό͕ରԠͯ͠ ͍Ε͹ࣗಈతʹSPDY͕ར༻͞ΕΔ

Slide 28

Slide 28 text

SPDY with nginx

Slide 29

Slide 29 text

άϥϑͰݟΔ࠷దԽͷޮՌ 5-44FTTJPO $BDIFc5JDLFUT ಋೖλΠϛϯά

Slide 30

Slide 30 text

άϥϑͰݟΔ࠷దԽͷޮՌ 5-44FTTJPO $BDIFc5JDLFUT ಋೖλΠϛϯά

Slide 31

Slide 31 text

άϥϑͰݟΔ࠷దԽͷޮՌ 41%:ಋೖͨ͠೔ͷϐʔΫ

Slide 32

Slide 32 text

"1*αʔό

Slide 33

Slide 33 text

"QBDIFNPE@QIQ

Slide 34

Slide 34 text

Apache + mod_php • ϝϧΧϦͷதͰҰ൪ٕज़తʹރΕͨίϯϙʔωϯτ • DietCakeͱ͍͏ܰྔͰߴ଎ͳϑϨʔϜϫʔΫΛར༻ • ฏۉϨεϙϯελΠϜ͕ϐʔΫ࣌Ͱ΋100msલޙ • APIαʔό͸΄ͱΜͲϘτϧωοΫʹͳ͍ͬͯͳ͍

Slide 35

Slide 35 text

σʔλετΞ

Slide 36

Slide 36 text

.Z42- 3FEJT NFNDBDIFE 4PMS -VDFOF

Slide 37

Slide 37 text

ϝϧΧϦΛࢧ͑ΔσʔλετΞ • MySQL • αʔϏεͷίΞσʔλ֨ೲ৔ॴ • αʔϏεͷن໛֦େʹ൐͍ਫฏ෼ׂ͕੝Μ • Redisɺmemcached • ΞϓϦέʔγϣϯΩϟογϡϨΠϠʔ • Solr(Lucene) • ݕࡧ

Slide 38

Slide 38 text

ඇಉظॲཧͷͨΊͷ ΩϡʔͱδϣϒϫʔΧʔ

Slide 39

Slide 39 text

2.QIQ1BSBMMFM1SFGPSL

Slide 40

Slide 40 text

ॏ͍ॲཧ͸2.ʹΤϯΩϡʔͯ͠ඇಉظͰ࣮ߦ

Slide 41

Slide 41 text

PHP-Paralle-Prefork • PHPͷpreforkσʔϞϯϑϨʔϜϫʔΫ • ฐࣾͷαʔόαΠυΤϯδχΞ͕։ൃ • ϓϩηε୯ҐͰλεΫΛฒྻ࣮ߦ • Q4M͔ΒσΩϡʔͨ͠λεΫΛฒྻ࣮ߦ͍ͯ͠Δ • PHPͰ΋͜͜·ͰͰ͖Δʂ

Slide 42

Slide 42 text

ϩάσʔλ෼ੳج൫

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

ΞΫηεϩά΍ΞϓϦέʔγϣϯͷϩάΛ 'MVFOUE΍όονͰ֤ॴʹసૹ

Slide 45

Slide 45 text

ϝϧΧϦΛࢧ͑Δ ϩά&σʔλ෼ੳج൫ ίϯϙʔωϯτ ༻్ 'MVFOUE ֤छϩάసૹ 5SFBTVSFEBUB #JH2VFSZ σʔλ෼ੳ 4 ϩάͷόοΫΞοϓ ,JCBOB&MBTUJDTFBSDI ௚ۙͷϩά΍σʔλͷ౷ܭΛνΣοΫ /PSJLSB ϦΞϧλΠϜͰσʔλूܭɾ؂ࢹ

Slide 46

Slide 46 text

τϐοΫ • ϝϧΧϦΛࢧ͑Δ։ൃମ੍ • ϝϧΧϦΛࢧ͑ΔΠϯϑϥ • ϝϧΧϦΛࢧ͑ΔαϒγεςϜ • ϝϧΧϦΛࢧ͑ΔGo

Slide 47

Slide 47 text

ʰϝϧΧϦʱΛࢧ͑ΔGo • PHPͷΠϝʔδ͕ڧ͍ฐࣾͰ͕͢ɺRuby΍Nodeɺ Goͱ͍ͬͨݴޠ΋ར༻͍ͯ͠·͢ • ࠷ۙPerlʹΑΔϞχλϦϯάπʔϧ͕ՃΘΓ·ͨ͠ • ࠓճ͸ϝϧΧϦͰঃʑʹ૿͍͑ͯΔGoͷϓϩμΫτ Λ঺հ͠·͢

Slide 48

Slide 48 text

ʰϝϧΧϦʱΛࢧ͑ΔGo • nginx-build • cachectl • Slackboard • ൚༻ϓογϡ௨஌αʔό(·ͩඇެ։)

Slide 49

Slide 49 text

nginx-build • nginxͷϏϧυπʔϧ • ҎԼͷ࡞ۀΛࣗಈԽ • nginxͷιʔείʔυͷμ΢ϯϩʔυ • ґଘϥΠϒϥϦ΍3rd PartyϞδϡʔϧͷ૊ΈࠐΈ • nginxͷRPM࡞੒࣌ʹར༻͍ͯ͠Δ • https://github.com/cubicdaiya/nginx-build

Slide 50

Slide 50 text

nginx-buildͰnginxΛϏϧυ

Slide 51

Slide 51 text

cachectl • OSͷϖʔδΩϟογϡ੍ޚπʔϧ • ಛఆͷϑΝΠϧͷϖʔδΩϟογϡͷར༻ঢ়گͷ֬ೝ΍ ϖʔδΩϟογϡͷ࡟আ͕Ͱ͖Δ • ϖʔδΩϟογϡΛ࡟আ͢ΔσʔϞϯ(cachectld)͕෇ଐ • APIαʔόͷڊେͳϩάϑΝΠϧΛϖʔδΩϟογϡ͔ Βఆظతʹ௥͍ग़͢ͷʹར༻͍ͯ͠Δ(εϫοϓରࡦ) • https://github.com/cubicdaiya/cachectl

Slide 52

Slide 52 text

cachectldͷઃఆͱىಈ

Slide 53

Slide 53 text

APIαʔόʹcachectldΛ ಋೖͨ݁͠Ռɺ "1*αʔό͕εϫοϓ͠ͳ͘ͳͬͨ DBDIFDUMEΛىಈͨ͠λΠϛϯά

Slide 54

Slide 54 text

Slackboard • SlackϓϩΩγαʔό • SlackͷHook URL΍νϟϯωϧͷ৘ใΛҰݩ؅ཧ&λά෇͚ • Slackͷઃఆ͕֤αʔόʹ෼ࢄ͢ΔͷΛ๷͙ • ྡͷ੮ͷਓᐌ͘ʮ͋͋ɺikachanΈ͍ͨͳ΍ͭͶʯ • ikachan஌Βͳ͔ͬͨ… • https://github.com/cubicdaiya/slackboard

Slide 55

Slide 55 text

4MBDLͷઃఆ͸TMBDLCPBSEαʔόͰҰݩ؅ཧ

Slide 56

Slide 56 text

खൈ͖γϯϓϧͳWeb UI ҰମԿετϥοϓͳΜͩɾɾɾ

Slide 57

Slide 57 text

൚༻ϓογϡ௨஌αʔό • HTTPϕʔεͷγϯϓϧͳAPIͱे෼ͳϞχλϦϯάػೳ • े෼ͳύϑΥʔϚϯε(਺ઍʙ਺ສQPS) • δϟʔφϧϩάʹΑΔΫϥογϡϦΧόϦʔػೳ • ୹࣌ؒͰ਺ेສ݅ͷϓογϡ௨஌Λߦ͏͜ͱ͕Ͱ͖Δ • ઌʑि͋ͨΓ͔Βຊ൪Քಇத • ͍͍Ͷɺίϝϯτɺ঎඼ߪೖͷϓογϡ௨஌ɺetc… • ΋͏ͪΐͬͱ࣮੷ੵΜͩΒOSSԽ͍ͨ͠

Slide 58

Slide 58 text

൚༻ϓογϡ௨஌γεςϜͷΞʔΩςΫνϟ PS CBUDIαʔό

Slide 59

Slide 59 text

HTTP API "1* ղઆ 1045QVTI ϓογϡ௨஌ϦΫΤετड෇ (&5TUBUHP (PϥϯλΠϜͷ׆ಈঢ়گΛऔಘ (&5TUBUBQQ ಺෦Ωϡʔ νϟωϧ ͷ࢖༻ྔ΍ ϓογϡͷ੒ޭࣦഊ਺ͷऔಘ (&5DPOpHBQQ αʔόઃఆ 50.- Λऔಘ

Slide 60

Slide 60 text

POST /pushͷϦΫΤετϘσΟ

Slide 61

Slide 61 text

GET /stat/go $ curl -s http://127.0.0.1:1056/stat/go { "time": 1423270089195474419, "go_version": "go1.4", "go_os": "linux", "go_arch": "amd64", "cpu_num": 24, "goroutine_num": 55, "gomaxprocs": 24, "cgo_call_num": 124016, "memory_alloc": 10309424, "memory_total_alloc": 295474683712, "memory_sys": 25557240, "memory_lookups": 214058, "memory_mallocs": 2118264299, "memory_frees": 2118230111, ・・・

Slide 62

Slide 62 text

GET /stat/app $ curl -s http://127.0.0.1:1056/stat/app { "queue_max": 40960, // 内部キューのサイズ "queue_usage": 0, // 内部キューの使用量(すぐ捌けるので大体0) "ios": { "push_success": 31804, // iOSプッシュ通知の成功数 "push_error": 17 // iOSプッシュ通知の失敗数 }, "android": { "push_success": 18683, // Androidプッシュ通知の成功数 "push_error": 233 // Androidプッシュ通知の失敗数 } $

Slide 63

Slide 63 text

GoΛબ୒͢Δཧ༝ • ύϑΥʔϚϯε͕ྑͯ͘ੜ࢈ੑ͕ߴ͍ • ͲͷϓϩμΫτ΋਺࣌ؒͰ࠷ॳͷόʔδϣϯΛॻ͍ͨ • ࣮ߦϑΝΠϧ͕୯ҰόΠφϦͳͷͰ؅ཧָ͕ • ฒྻॲཧ͕؆୯ʹॻ͚Δ • PHPͷऑ͍෦෼Λ͏·͘ิ׬Ͱ͖Δ

Slide 64

Slide 64 text

·ͱΊ • ϝϧΧϦ͸μ΢ϯϩʔυ਺1000ສΛӽ͑ΔਓؾϑϦϚΞϓϦ • ͘͞Β΍AWS౳ͷΠϯϑϥ΍Ϋϥ΢υαʔϏεͷ্ ʹߏங͞Ε͍ͯΔ • ֤αϒγεςϜ͸ׂΓͱΦʔιυοΫε • ݴޠ͸PHP͚ͩͰͳ͘Go΍Node౳ɺෳ਺ͷݴޠΛঢ় گʹԠͯ͡࢖͍෼͚ͯ·͢

Slide 65

Slide 65 text

ਓࡐืूதʂ