みんなのIMAPを可視化する / Visualize IMAP Everybody

みんなのIMAPを可視化する / Visualize IMAP Everybody

Fukuoka.go#16 で発表した資料です。Junly 14, 2020

5d769d109697012317c09c6a27a6a4bf?s=128

linyows

July 14, 2020
Tweet

Transcript

  1. Fukuoka.go#16 Tomohisa Oda / July 14, 2020 VisualizeIMAP Everybody

  2. Fukuoka.go#16 @linyows Blog: tomohisaoda.com Affiliation: GMO pepabo, inc.

  3. Fukuoka.go#16 ๏ ϗεςΟϯάࣄۀ෦Senior Engineering Lead ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ ๏ ϖύϘݚڀॴ۝भେֶڞಉݚڀ 2020Պݚඅ࠾୒IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5, ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ

    In Pepabo…
  4. Fukuoka.go#16 Private Projects ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔Β࠷৽ͷίʔυΛݕ ஌ͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧ MJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔ΍Ϧϙδτ ϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ

  5. Fukuoka.go#16 Pepabo͸ϗετ͍ͯ͠·͢

  6. Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,

    FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓ΍ϔςϜϧ͕ϗετ͍ͯ͠Δ΋ͷ ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
  7. Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,

    FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓ΍ϔςϜϧ͕ϗετ͍ͯ͠Δ΋ͷ ຊ೔͸ίίͷ࿩
  8. Fukuoka.go#16 Mail ࢖͍ͬͯ·͔͢ʁ

  9. Fukuoka.go#16 ͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍΍͍΍͍΍ʜ — Chat Type — Messaging Type

  10. Fukuoka.go#16 ΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ࢖͍ͬͯ·͢Ͷʁ

  11. Fukuoka.go#16 େมਃ͠༁͋Γ·ͤΜ

  12. Fukuoka.go#16 େମ(NBJMPS(4VJUFͰ͢ΑͶʁ

  13. Fukuoka.go#16 ͍ʹ͑͠ͷٕज़ͱ͸͍͑ɺMailʹมΘΔ ΦʔϓϯͳMessagingγεςϜ͸ͳ͍

  14. Fukuoka.go#16 Mailͷ࢓૊Έ

  15. Fukuoka.go#16 Architecture RFC 5598 ʹ͋Γ·͕͢ʜ

  16. Fukuoka.go#16 Architecture ΋ͬͱγϯϓϧͳ΍ͭ ͜ΕͰ࠷খߏ੒ͳͷͰࠜຊతʹ ΢ΣϒΑΓෳࡶ Mail Submission Agent :587 Mail

    Transferring Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP
  17. Fukuoka.go#16 ϩϦϙοϓ΍ϔςϜϧʹ͍ͭͯ

  18. Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳن໛ײ

  19. Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳن໛ײ ϝʔϧαʔό୆ʹ৐Δ

    ϝʔϧΞΧ΢ϯτ Max ϝʔϧαʔόͷ਺ ෼ؒʹૹΔϝʔϧ ௨਺ʢDeliveredʣ
  20. Fukuoka.go#16 Our Challenges ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ໰୊ ΞΧ΢ϯτ৐ͬऔΓ΍ෆਖ਼ར༻໰୊

    ѱҙͷͳ͍େྔϝʔϧ഑৴ ๏ ड৴ ๏ ϑΟογϯάɺ΢ΟϧεɺεύϜϝʔϧഉআ໰୊ ๏ Disk I/Oෛՙ໰୊
  21. Fukuoka.go#16 ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ໰୊ ΞΧ΢ϯτ৐ͬऔΓ΍ෆਖ਼ར༻໰୊ ѱҙͷͳ͍େྔϝʔϧ഑৴ ๏

    ड৴ ๏ ϑΟογϯάɺ΢ΟϧεɺεύϜϝʔϧഉআ໰୊ ๏ Disk I/Oෛՙ໰୊ Our Challenges
  22. Fukuoka.go#16 ๏ ͍͍ετϨʔδΛങ͑͹ղܾ͢Δ ๏ ௿Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠ ๏ ݱঢ়ɺෛՙͷߴ͍ΞΧ΢ϯτΛҠઃͯ͠Ϧιʔε ফඅΛۉ͢ઓུ ๏ ࣌ʹ͸SFTUBSUʜ

    IMAP Disk I/O ໰୊
  23. Fukuoka.go#16 ๏ ओʹPrometheusΛ࢖͍ͬͯΔ ๏ ֤Proxy΍ϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔε؂ࢹ ๏ mtailͰϝʔϧΩϡʔ΍େखϓϩόΠμʔ΁ͷό΢ϯεϝʔϧͷ؂ࢹ ๏ blackbox_exporterͰsmtp, pop,

    imapͷPort؂ࢹ ݱঢ়ͷ؂ࢹ͸े෼Ͱ͸ͳ͍ Monitoring
  24. Fukuoka.go#16

  25. Fukuoka.go#16

  26. Fukuoka.go#16

  27. Fukuoka.go#16 imap_exporterΛ࡞ͬͨ

  28. Fukuoka.go#16 ๏ ϛυϧ΢ΣΞʹґଘ͠ͳ͍ΑΓந৅తͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ ๏ Connection /

    Login / List / Select / Fetch / Logout ๏ IMAP Client͸ɺطଘϞδϡʔϧΛ࢖͏ͷͰͳࣗ͘લͰ࣮૷͢ΔʢIMAPͷؾ࣋ͪʹͳ Δʣ imap_exporter
  29. Fukuoka.go#16 ๏ RFC2060ΛΑʔ͠ಡΉͧͱ ࢥͬͨΒ్தͰ૬౰ͳϘϦϡʔ Ϝ͕͋Δ͜ͱʹؾͮ͘ ๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠ ϖʔδ IMAPͷ࢓༷͸RFC2060

  30. Fukuoka.go#16 ๏ ͜Μͳ͜ͱ΋͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢ ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮૷ɾӡ༻ 2000/11

  31. Fukuoka.go#16 ๏ IMAPʹ͸ͭͷεςʔτ͕͋Γ ๏ ະೝূ ๏ ೝূࡁ ๏ બ୒ࡁ ๏

    ϩάΞ΢τ ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛ࢖ͬͯαʔόʔͱΫϥΠΞϯ τͷίϛϡχέʔγϣϯΛߦ͏ IMAP ίϚϯυ λά
  32. Fukuoka.go#16 IMAP ΫϥΠΞϯτͷ࣮૷

  33. Fukuoka.go#16 Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λ ൑ఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮૷

  34. Fukuoka.go#16 telnet΍opensslΛ࢖͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ

  35. Fukuoka.go#16 ๏ ࠨͷάϥϑ͸खݩ͔ΒαϯϓϧΞΧ΢ϯτʹ઀ ଓͨ͠ྫ ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕ͹ͱ͍ ͏ఔ౓

    ๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹ͸ؒҧ͍ͳ͍ ๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊ લΛ͚ͭͨ First Loading Latency
  36. Fukuoka.go#16 ๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ ๏ IMAPϓϩτίϧ͸SMTPͱҧͬͯ૬౰ෳࡶ ๏ ϓϩτίϧ࢓༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮૷Ͱ͖ͨ ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छ

    ΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂ Conclusion
  37. Fukuoka.go#16 Messaging Reliability EngineeringʢMREʣ͸ɺਓͱਓͱ͕ιϑτ΢ΣΞΛ௨ͯ͡ ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτ ΢ΣΞγεςϜΛ࡞੒͠·͢ɻMRE͸ɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞Δ Site Reliability EngineeringͷจԽΛMessaging෼໺Ͱड͚ܧ͍ͩ΋ͷͰ͢ɻجຊ తͳ͜ͱ͸SREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻Λιϑτ΢Σ

    ΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂ Messaging Reliability Engineeringͱ͸
  38. Fukuoka.go#16 Thank youaʕ⊙౪⚆ʔ