Fukuoka.go#16 で発表した資料です。Junly 14, 2020
Fukuoka.go#16Tomohisa Oda / July 14, 2020VisualizeIMAP Everybody
View Slide
Fukuoka.go#16@linyowsBlog: tomohisaoda.comAffiliation: GMO pepabo, inc.
Fukuoka.go#16๏ ϗεςΟϯάࣄۀ෦Senior Engineering Leadࣄۀ෦ͷCTOVPoEͷΑ͏ͳͷ๏ ϖύϘݚڀॴभେֶڞಉݚڀ2020Պݚඅ࠾IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5,ܰྔίϯςφʹΑΔେنߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽIn Pepabo…
Fukuoka.go#16Private ProjectsΞϓϦέʔγϣϯ͕ࣗϦϙδτϦ͔Β࠷৽ͷίʔυΛݕͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧMJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔϦϙδτϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ
Fukuoka.go#16Pepaboϗετ͍ͯ͠·͢
Fukuoka.go#16๏ Domain — DNS๏ Web — HTTP, SSH, FTP and WebDav…๏ Mail — SMTP, POP, IMAP…๏ Database — (MySQL, SQLite…)ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
Fukuoka.go#16๏ Domain — DNS๏ Web — HTTP, SSH, FTP and WebDav…๏ Mail — SMTP, POP, IMAP…๏ Database — (MySQL, SQLite…)ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷຊίίͷ
Fukuoka.go#16Mail ͍ͬͯ·͔͢ʁ
Fukuoka.go#16͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍͍͍ʜ— Chat Type— Messaging Type
Fukuoka.go#16΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ͍ͬͯ·͢Ͷʁ
Fukuoka.go#16େมਃ͠༁͋Γ·ͤΜ
Fukuoka.go#16େମ(NBJMPS(4VJUFͰ͢ΑͶʁ
Fukuoka.go#16͍ʹ͑͠ͷٕज़ͱ͍͑ɺMailʹมΘΔΦʔϓϯͳMessagingγεςϜͳ͍
Fukuoka.go#16MailͷΈ
Fukuoka.go#16ArchitectureRFC 5598 ʹ͋Γ·͕͢ʜ
Fukuoka.go#16Architectureͬͱγϯϓϧͳͭ͜ΕͰ࠷খߏͳͷͰࠜຊతʹΣϒΑΓෳࡶ Mail SubmissionAgent:587Mail TransferringAgent:25Mail DeliveryAgent:993Postfix Postfix Dovecot✉✉SMTPSMTPLMTPIMAP
Fukuoka.go#16ϩϦϙοϓϔςϜϧʹ͍ͭͯ
Fukuoka.go#16Our Status120100k 2.2kOver /min͜ͷΑ͏ͳنײ
Fukuoka.go#16Our Status120100k 2.2kOver /min͜ͷΑ͏ͳنײϝʔϧαʔόʹΔϝʔϧΞΧϯτMaxϝʔϧαʔόͷؒʹૹΔϝʔϧ௨ʢDeliveredʣ
Fukuoka.go#16Our Challenges๏ ૹ৴๏ IP BanɺBlack ListೖΓΞΧϯτͬऔΓෆਖ਼ར༻ѱҙͷͳ͍େྔϝʔϧ৴๏ ड৴๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ๏ Disk I/Oෛՙ
Fukuoka.go#16๏ ૹ৴๏ IP BanɺBlack ListೖΓΞΧϯτͬऔΓෆਖ਼ར༻ѱҙͷͳ͍େྔϝʔϧ৴๏ ड৴๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ๏ Disk I/OෛՙOur Challenges
Fukuoka.go#16๏ ͍͍ετϨʔδΛങ͑ղܾ͢Δ๏ Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠๏ ݱঢ়ɺෛՙͷߴ͍ΞΧϯτΛҠઃͯ͠ϦιʔεফඅΛۉ͢ઓུ๏ ࣌ʹSFTUBSUʜIMAP Disk I/O
Fukuoka.go#16๏ ओʹPrometheusΛ͍ͬͯΔ๏ ֤Proxyϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔεࢹ๏ mtailͰϝʔϧΩϡʔେखϓϩόΠμʔͷόϯεϝʔϧͷࢹ๏ blackbox_exporterͰsmtp, pop, imapͷPortࢹݱঢ়ͷࢹेͰͳ͍Monitoring
Fukuoka.go#16
Fukuoka.go#16imap_exporterΛ࡞ͬͨ
Fukuoka.go#16๏ ϛυϧΣΞʹґଘ͠ͳ͍ΑΓநతͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ๏ Connection / Login / List / Select / Fetch / Logout๏ IMAP ClientɺطଘϞδϡʔϧΛ͏ͷͰͳࣗ͘લͰ࣮͢ΔʢIMAPͷؾ࣋ͪʹͳΔʣimap_exporter
Fukuoka.go#16๏ RFC2060ΛΑʔ͠ಡΉͧͱࢥͬͨΒ్தͰ૬ͳϘϦϡʔϜ͕͋Δ͜ͱʹؾͮ͘๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠ϖʔδIMAPͷ༷RFC2060
Fukuoka.go#16๏ ͜Μͳ͜ͱ͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮ɾӡ༻ 2000/11
Fukuoka.go#16๏ IMAPʹͭͷεςʔτ͕͋Γ๏ ະೝূ๏ ೝূࡁ๏ બࡁ๏ ϩάΞτ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛͬͯαʔόʔͱΫϥΠΞϯτͷίϛϡχέʔγϣϯΛߦ͏IMAP ίϚϯυλά
Fukuoka.go#16IMAP ΫϥΠΞϯτͷ࣮
Fukuoka.go#16Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮
Fukuoka.go#16telnetopensslΛ͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ
Fukuoka.go#16๏ ࠨͷάϥϑखݩ͔ΒαϯϓϧΞΧϯτʹଓͨ͠ྫ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕͱ͍͏ఔ๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹؒҧ͍ͳ͍๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊લΛ͚ͭͨFirst Loading Latency
Fukuoka.go#16๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ๏ IMAPϓϩτίϧSMTPͱҧͬͯ૬ෳࡶ๏ ϓϩτίϧ༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮Ͱ͖ͨ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂConclusion
Fukuoka.go#16Messaging Reliability EngineeringʢMREʣɺਓͱਓͱ͕ιϑτΣΞΛ௨ͯ͡ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτΣΞγεςϜΛ࡞͠·͢ɻMREɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞ΔSite Reliability EngineeringͷจԽΛMessagingͰड͚ܧ͍ͩͷͰ͢ɻجຊతͳ͜ͱSREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻ΛιϑτΣΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂMessaging Reliability Engineeringͱ
Fukuoka.go#16Thank youaʕ⊙౪⚆ʔ