みんなのIMAPを可視化する / Visualize IMAP Everybody

Fukuoka.go#16 で発表した資料です。Junly 14, 2020

Transcript

1. Fukuoka.go#16
   Tomohisa Oda / July 14, 2020
   VisualizeIMAP Everybody
   

   View Slide

2. Fukuoka.go#16
   @linyows
   Blog: tomohisaoda.com
   Affiliation: GMO pepabo, inc.
   

   View Slide

3. Fukuoka.go#16
   ๏ ϗεςΟϯάࣄۀ෦Senior Engineering Lead
   ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ
   ๏ ϖύϘݚڀॴ۝भେֶڞಉݚڀ
   2020Պݚඅ࠾୒IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5,
   ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
   In Pepabo…
   

   View Slide

4. Fukuoka.go#16
   Private Projects
   ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔Β࠷৽ͷίʔυΛݕ
   ஌ͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧ
   MJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔ΍Ϧϙδτ
   ϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ
   

   View Slide

5. Fukuoka.go#16
   Pepabo͸ϗετ͍ͯ͠·͢
   

   View Slide

6. Fukuoka.go#16
   ๏ Domain — DNS
   ๏ Web — HTTP, SSH, FTP and WebDav…
   ๏ Mail — SMTP, POP, IMAP…
   ๏ Database — (MySQL, SQLite…)
   ϩϦϙοϓ΍ϔςϜϧ͕ϗετ͍ͯ͠Δ΋ͷ
   ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
   

   View Slide

7. Fukuoka.go#16
   ๏ Domain — DNS
   ๏ Web — HTTP, SSH, FTP and WebDav…
   ๏ Mail — SMTP, POP, IMAP…
   ๏ Database — (MySQL, SQLite…)
   ϩϦϙοϓ΍ϔςϜϧ͕ϗετ͍ͯ͠Δ΋ͷ
   ຊ೔͸ίίͷ࿩
   

   View Slide

8. Fukuoka.go#16
   Mail ࢖͍ͬͯ·͔͢ʁ
   

   View Slide

9. Fukuoka.go#16
   ͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍΍͍΍͍΍ʜ
   — Chat Type
   — Messaging Type
   

   View Slide

10. Fukuoka.go#16
    ΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ࢖͍ͬͯ·͢Ͷʁ
    

    View Slide

11. Fukuoka.go#16
    େมਃ͠༁͋Γ·ͤΜ
    

    View Slide

12. Fukuoka.go#16
    େମ(NBJMPS(4VJUFͰ͢ΑͶʁ
    

    View Slide

13. Fukuoka.go#16
    ͍ʹ͑͠ͷٕज़ͱ͸͍͑ɺMailʹมΘΔ
    ΦʔϓϯͳMessagingγεςϜ͸ͳ͍
    

    View Slide

14. Fukuoka.go#16
    Mailͷ࢓૊Έ
    

    View Slide

15. Fukuoka.go#16
    Architecture
    RFC 5598 ʹ͋Γ·͕͢ʜ
    

    View Slide

16. Fukuoka.go#16
    Architecture
    ΋ͬͱγϯϓϧͳ΍ͭ
    ͜ΕͰ࠷খߏ੒ͳͷͰࠜຊతʹ
    ΢ΣϒΑΓෳࡶ
    
    Mail Submission
    Agent
    :587
    Mail Transferring
    Agent
    :25
    Mail Delivery
    Agent
    :993
    Postfix Postfix Dovecot
    ✉
    ✉
    SMTP
    SMTP
    LMTP
    IMAP
    
    
    

    View Slide

17. Fukuoka.go#16
    ϩϦϙοϓ΍ϔςϜϧʹ͍ͭͯ
    

    View Slide

18. Fukuoka.go#16
    Our Status
    120
    100k 2.2k
    Over /min
    ͜ͷΑ͏ͳن໛ײ
    

    View Slide

19. Fukuoka.go#16
    Our Status
    120
    100k 2.2k
    Over /min
    ͜ͷΑ͏ͳن໛ײ
    ϝʔϧαʔό୆ʹ৐Δ
    ϝʔϧΞΧ΢ϯτ
    Max
    ϝʔϧαʔόͷ਺
    ෼ؒʹૹΔϝʔϧ
    ௨਺ʢDeliveredʣ
    

    View Slide

20. Fukuoka.go#16
    Our Challenges
    ๏ ૹ৴
    ๏ IP BanɺBlack ListೖΓ໰୊
    ΞΧ΢ϯτ৐ͬऔΓ΍ෆਖ਼ར༻໰୊
    ѱҙͷͳ͍େྔϝʔϧ഑৴
    ๏ ड৴
    ๏ ϑΟογϯάɺ΢ΟϧεɺεύϜϝʔϧഉআ໰୊
    ๏ Disk I/Oෛՙ໰୊
    

    View Slide

21. Fukuoka.go#16
    ๏ ૹ৴
    ๏ IP BanɺBlack ListೖΓ໰୊
    ΞΧ΢ϯτ৐ͬऔΓ΍ෆਖ਼ར༻໰୊
    ѱҙͷͳ͍େྔϝʔϧ഑৴
    ๏ ड৴
    ๏ ϑΟογϯάɺ΢ΟϧεɺεύϜϝʔϧഉআ໰୊
    ๏ Disk I/Oෛՙ໰୊
    Our Challenges
    

    View Slide

22. Fukuoka.go#16
    ๏ ͍͍ετϨʔδΛങ͑͹ղܾ͢Δ
    ๏ ௿Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠
    ๏ ݱঢ়ɺෛՙͷߴ͍ΞΧ΢ϯτΛҠઃͯ͠Ϧιʔε
    ফඅΛۉ͢ઓུ
    ๏ ࣌ʹ͸SFTUBSUʜ
    IMAP Disk I/O ໰୊
    

    View Slide

23. Fukuoka.go#16
    ๏ ओʹPrometheusΛ࢖͍ͬͯΔ
    ๏ ֤Proxy΍ϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔε؂ࢹ
    ๏ mtailͰϝʔϧΩϡʔ΍େखϓϩόΠμʔ΁ͷό΢ϯεϝʔϧͷ؂ࢹ
    ๏ blackbox_exporterͰsmtp, pop, imapͷPort؂ࢹ
    ݱঢ়ͷ؂ࢹ͸े෼Ͱ͸ͳ͍
    Monitoring
    

    View Slide

24. Fukuoka.go#16
    

    View Slide

25. Fukuoka.go#16
    

    View Slide

26. Fukuoka.go#16
    

    View Slide

27. Fukuoka.go#16
    imap_exporterΛ࡞ͬͨ
    

    View Slide

28. Fukuoka.go#16
    ๏ ϛυϧ΢ΣΞʹґଘ͠ͳ͍ΑΓந৅తͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ
    ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ
    ๏ Connection / Login / List / Select / Fetch / Logout
    ๏ IMAP Client͸ɺطଘϞδϡʔϧΛ࢖͏ͷͰͳࣗ͘લͰ࣮૷͢ΔʢIMAPͷؾ࣋ͪʹͳ
    Δʣ
    imap_exporter
    

    View Slide

29. Fukuoka.go#16
    ๏ RFC2060ΛΑʔ͠ಡΉͧͱ
    ࢥͬͨΒ్தͰ૬౰ͳϘϦϡʔ
    Ϝ͕͋Δ͜ͱʹؾͮ͘
    ๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠
    ϖʔδ
    IMAPͷ࢓༷͸RFC2060
    

    View Slide

30. Fukuoka.go#16
    ๏ ͜Μͳ͜ͱ΋͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢
    ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮૷ɾӡ༻ 2000/11
    

    View Slide

31. Fukuoka.go#16
    ๏ IMAPʹ͸ͭͷεςʔτ͕͋Γ
    ๏ ະೝূ
    ๏ ೝূࡁ
    ๏ બ୒ࡁ
    ๏ ϩάΞ΢τ
    ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛ࢖ͬͯαʔόʔͱΫϥΠΞϯ
    τͷίϛϡχέʔγϣϯΛߦ͏
    IMAP ίϚϯυ
    λά
    

    View Slide

32. Fukuoka.go#16
    IMAP ΫϥΠΞϯτͷ࣮૷
    

    View Slide

33. Fukuoka.go#16
    Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λ
    ൑ఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮૷
    

    View Slide

34. Fukuoka.go#16
    telnet΍opensslΛ࢖͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ
    

    View Slide

35. Fukuoka.go#16
    ๏ ࠨͷάϥϑ͸खݩ͔ΒαϯϓϧΞΧ΢ϯτʹ઀
    ଓͨ͠ྫ
    ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ
    ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕ͹ͱ͍
    ͏ఔ౓
    ๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹ͸ؒҧ͍ͳ͍
    ๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊
    લΛ͚ͭͨ
    First Loading Latency
    

    View Slide

36. Fukuoka.go#16
    ๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ
    ๏ IMAPϓϩτίϧ͸SMTPͱҧͬͯ૬౰ෳࡶ
    ๏ ϓϩτίϧ࢓༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮૷Ͱ͖ͨ
    ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छ
    ΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂ
    Conclusion
    

    View Slide

37. Fukuoka.go#16
    Messaging Reliability EngineeringʢMREʣ͸ɺਓͱਓͱ͕ιϑτ΢ΣΞΛ௨ͯ͡
    ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτ
    ΢ΣΞγεςϜΛ࡞੒͠·͢ɻMRE͸ɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞Δ
    Site Reliability EngineeringͷจԽΛMessaging෼໺Ͱड͚ܧ͍ͩ΋ͷͰ͢ɻجຊ
    తͳ͜ͱ͸SREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻Λιϑτ΢Σ
    ΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂ
    Messaging Reliability Engineeringͱ͸
    

    View Slide

38. Fukuoka.go#16
    Thank youaʕ⊙౪⚆ʔ
    

    View Slide