Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
みんなのIMAPを可視化する / Visualize IMAP Everybody
linyows
July 14, 2020
Programming
2
180
みんなのIMAPを可視化する / Visualize IMAP Everybody
Fukuoka.go#16 で発表した資料です。Junly 14, 2020
linyows
July 14, 2020
Tweet
Share
More Decks by linyows
See All by linyows
linyows
1
230
linyows
0
110
linyows
0
330
linyows
0
490
linyows
1
800
linyows
7
1.4k
linyows
1
3.4k
linyows
0
110
linyows
6
1.1k
Other Decks in Programming
See All in Programming
grapecity_dev
0
180
grapecity_dev
0
180
manfredsteyer
PRO
0
150
antonshilov
1
130
manfredsteyer
PRO
0
110
ryokbt
2
300
abeta
1
210
tooppoo
0
190
heistak
2
130
joergneumann
0
150
prof18
0
1.6k
shiz
1
100
Featured
See All Featured
swwweet
206
6.9k
frogandcode
128
20k
hannesfritz
28
950
cromwellryan
104
6.2k
tammielis
237
23k
matthewcrist
73
7.5k
geoffreycrofte
21
930
zakiwarfel
88
3.4k
lauravandoore
10
1.6k
mojombo
359
62k
cassininazir
347
20k
edds
56
9.4k
Transcript
Fukuoka.go#16 Tomohisa Oda / July 14, 2020 VisualizeIMAP Everybody
Fukuoka.go#16 @linyows Blog: tomohisaoda.com Affiliation: GMO pepabo, inc.
Fukuoka.go#16 ๏ ϗεςΟϯάࣄۀ෦Senior Engineering Lead ࣄۀ෦ͷCTOVPoEͷΑ͏ͳͷ ๏ ϖύϘݚڀॴभେֶڞಉݚڀ 2020Պݚඅ࠾IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5, ܰྔίϯςφʹΑΔେنߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
In Pepabo…
Fukuoka.go#16 Private Projects ΞϓϦέʔγϣϯ͕ࣗϦϙδτϦ͔Β࠷৽ͷίʔυΛݕ ͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧ MJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔϦϙδτ ϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ
Fukuoka.go#16 Pepaboϗετ͍ͯ͠·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ຊίίͷ
Fukuoka.go#16 Mail ͍ͬͯ·͔͢ʁ
Fukuoka.go#16 ͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍͍͍ʜ — Chat Type — Messaging Type
Fukuoka.go#16 ΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ͍ͬͯ·͢Ͷʁ
Fukuoka.go#16 େมਃ͠༁͋Γ·ͤΜ
Fukuoka.go#16 େମ(NBJMPS(4VJUFͰ͢ΑͶʁ
Fukuoka.go#16 ͍ʹ͑͠ͷٕज़ͱ͍͑ɺMailʹมΘΔ ΦʔϓϯͳMessagingγεςϜͳ͍
Fukuoka.go#16 MailͷΈ
Fukuoka.go#16 Architecture RFC 5598 ʹ͋Γ·͕͢ʜ
Fukuoka.go#16 Architecture ͬͱγϯϓϧͳͭ ͜ΕͰ࠷খߏͳͷͰࠜຊతʹ ΣϒΑΓෳࡶ Mail Submission Agent :587 Mail
Transferring Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP
Fukuoka.go#16 ϩϦϙοϓϔςϜϧʹ͍ͭͯ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ ϝʔϧαʔόʹΔ
ϝʔϧΞΧϯτ Max ϝʔϧαʔόͷ ؒʹૹΔϝʔϧ ௨ʢDeliveredʣ
Fukuoka.go#16 Our Challenges ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻
ѱҙͷͳ͍େྔϝʔϧ৴ ๏ ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ
Fukuoka.go#16 ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻ ѱҙͷͳ͍େྔϝʔϧ৴ ๏
ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ Our Challenges
Fukuoka.go#16 ๏ ͍͍ετϨʔδΛങ͑ղܾ͢Δ ๏ Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠ ๏ ݱঢ়ɺෛՙͷߴ͍ΞΧϯτΛҠઃͯ͠Ϧιʔε ফඅΛۉ͢ઓུ ๏ ࣌ʹSFTUBSUʜ
IMAP Disk I/O
Fukuoka.go#16 ๏ ओʹPrometheusΛ͍ͬͯΔ ๏ ֤Proxyϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔεࢹ ๏ mtailͰϝʔϧΩϡʔେखϓϩόΠμʔͷόϯεϝʔϧͷࢹ ๏ blackbox_exporterͰsmtp, pop,
imapͷPortࢹ ݱঢ়ͷࢹेͰͳ͍ Monitoring
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16 imap_exporterΛ࡞ͬͨ
Fukuoka.go#16 ๏ ϛυϧΣΞʹґଘ͠ͳ͍ΑΓநతͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ ๏ Connection /
Login / List / Select / Fetch / Logout ๏ IMAP ClientɺطଘϞδϡʔϧΛ͏ͷͰͳࣗ͘લͰ࣮͢ΔʢIMAPͷؾ࣋ͪʹͳ Δʣ imap_exporter
Fukuoka.go#16 ๏ RFC2060ΛΑʔ͠ಡΉͧͱ ࢥͬͨΒ్தͰ૬ͳϘϦϡʔ Ϝ͕͋Δ͜ͱʹؾͮ͘ ๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠ ϖʔδ IMAPͷ༷RFC2060
Fukuoka.go#16 ๏ ͜Μͳ͜ͱ͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢ ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮ɾӡ༻ 2000/11
Fukuoka.go#16 ๏ IMAPʹͭͷεςʔτ͕͋Γ ๏ ະೝূ ๏ ೝূࡁ ๏ બࡁ ๏
ϩάΞτ ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛͬͯαʔόʔͱΫϥΠΞϯ τͷίϛϡχέʔγϣϯΛߦ͏ IMAP ίϚϯυ λά
Fukuoka.go#16 IMAP ΫϥΠΞϯτͷ࣮
Fukuoka.go#16 Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λ ఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮
Fukuoka.go#16 telnetopensslΛ͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ
Fukuoka.go#16 ๏ ࠨͷάϥϑखݩ͔ΒαϯϓϧΞΧϯτʹ ଓͨ͠ྫ ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕͱ͍ ͏ఔ
๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹؒҧ͍ͳ͍ ๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊ લΛ͚ͭͨ First Loading Latency
Fukuoka.go#16 ๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ ๏ IMAPϓϩτίϧSMTPͱҧͬͯ૬ෳࡶ ๏ ϓϩτίϧ༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮Ͱ͖ͨ ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छ
ΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂ Conclusion
Fukuoka.go#16 Messaging Reliability EngineeringʢMREʣɺਓͱਓͱ͕ιϑτΣΞΛ௨ͯ͡ ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτ ΣΞγεςϜΛ࡞͠·͢ɻMREɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞Δ Site Reliability EngineeringͷจԽΛMessagingͰड͚ܧ͍ͩͷͰ͢ɻجຊ తͳ͜ͱSREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻ΛιϑτΣ
ΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂ Messaging Reliability Engineeringͱ
Fukuoka.go#16 Thank youaʕ⊙౪⚆ʔ