Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Fukuoka.go #17 Jun 10, 2021 の発表資料です。

Transcript

1. Fukuoka.go #17 Tomohisa Oda / Jun 10, 2021 Transparent SMTP Proxy in Go GoͰͭ͘ΔಁաܕSMTPϓϩΩγ
2. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo, inc. Fukuoka.go and Fukuoka.ts Organizer
3. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo, inc. Fukuoka.go and Fukuoka.ts Organizer ࠷ۙɺNotionʹࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ
4. Fukuoka.go #17 ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ ๏ ϖύϘݚڀॴ ۝भେֶڞಉݚڀ 2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/ ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ In Pepabo…
5. Fukuoka.go #17 Private Projects ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔ Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ ΠͱσϦόϦʔΛߦ͏πʔϧ linuxͷuser໊લղܾΛGitHubͷ Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε ݖݶऀ͔Βߦ͏πʔϧ ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ New!!
6. Fukuoka.go #17 ϝʔϧͷ࢓૊ΈͱϝʔϧϗεςΟϯάͷ՝୊
7. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 DNS MX TXT(SPF, DKIM, DMARC…)
8. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ϝʔϧϗεςΟϯάʹ͓͍ͯ
   ϝʔϧΛ૬खઌʹ
   ౸ୡͤ͞Δ͜ͱ͕ॏཁ
9. Fukuoka.go #17 ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ
10. Fukuoka.go #17 ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith %/4#-ʹొ࿥͞ΕΔͳͲ
    Ϩϐϡςʔγϣϯʢධ൑ʣʹӨڹ͢Δ
11. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏ ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊
12. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏ ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊ ϝʔϧૹ৴࣌ͷͲΜͳৼΔ෣͍͕ड৴ڋ൱ʹࢸ͔ͬͨʁ
    ෼ੳ͠ݕ஌Ͱ͖ΔΑ͏ʹ͍ͨ͠ʂ
13. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ͜ͷ΍ΓͱΓΛ෼ੳͰ͖Δɺ͔ͭ
    ໰୊Λݕ஌Ͱ͖Ε͹
    ૹ৴ΛϒϩοΫͰ͖Δ࢓૊Έ͕ඞཁ
14. Fukuoka.go #17 ࣮ݱํ๏ͷൺֱ
15. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ
16. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ
17. Fukuoka.go #17 GoͰͭ͘ΔಁաܕSMTPϓϩΩγ
18. Fukuoka.go #17 Ͳ͏΍ͬͯಁաܕϓϩΩγΛ࣮ݱ͢Δ͔ ๏ Port25൪ѼΛDNATͰશͯProxyʹૹΔ ๏ getsockopt(2)ͰѼઌมߋલͷΞυϨεΛऔಘ͢Δ
19. Fukuoka.go #17 SMTP Example via: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol 4.51͸04*
    -ͷͱͯ΋γϯϓϧͳϓϩτίϧ
20. Fukuoka.go #17 Goͷio.CopyΛ࢖͑͹ָউͳΜ͡Όͳ͍ʁ
21. Fukuoka.go #17 Transparent Proxy Go Mail Transfer Agent Postfix alice@example.com Mail Submission Agent Postfix bob@example.net io.Copy(internalConn, externalConn) io.Copy(externalConn, internalConn)
22. Fukuoka.go #17 PoC ී௨ʹϝʔϧΛૹड৴Ͱ͖ͨͷͰ׬શʹ͏·͍ͬ͘ ͨͱࢥͬͨɻ
    ͔͠͠ɺHPSPVUJOF಺ͷม਺Λग़ྗͯ͠ΈΔͱɺ ్த͔Βจࣈ͕ಡΊͳ͍ײ͡ʹ
23. Fukuoka.go #17 STARTTLSͷ͜ͱΛ๨Ε͍ͯͨ!!!
24. Fukuoka.go #17 μ΢ϯάϨʔυ߈ܸରԠʹ͢Δ
25. Fukuoka.go #17 ୯ʹJP$PQZͰ͸ͳ͘
    5-4ίωΫγϣϯ·Ͱ ੍ޚ͢Δඞཁ͕͋ͬͨ
26. Fukuoka.go #17 45"355-4ʹରԠͯ͠ͳ͍ Ϩεϙϯεʹ͢Δ
27. Fukuoka.go #17 όοϑΝϦϯάͨ͠σʔλΛ5-4 ίωΫγϣϯͱͦͷޙͷఆܕ΍Γ औΓޙʹૹ৴͢Δ
28. Fukuoka.go #17 Demo: ࣌ؒͳ͍͔΋
29. Fukuoka.go #17 ๏ Integration Testͷख๏Λߟ͑ΔʢPostfix ґଘΛͲ͏͢΂͖͔ʣ ๏ ऩूͨ͠৘ใΛDatabaseʹ஝ੵͱ෼ੳ ๏ IptablesґଘϧʔςΟϯάΛBPF/XDPΛ ࢖͏Α͏ʹ͢Δ ๏ Production΁ΧφϦΞϦϦʔε ࠓޙͷܭը — github.com/linyows/warp
30. Fukuoka.go #17 ๏ ϝʔϧͷ౸ୡੑʹؔΘΔ̐ͭͷ߲໨͕͋Δ ๏ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊͸ड৴ڋ൱ͷཧ༝͕Θ͔Βͳ͍͜ͱ ๏ ϝʔϧૹ৴࣌ͷΞΧ΢ϯτ͝ͱͷৼΔ෣͍ͱड৴ڋ൱ͷؔ܎Λݟ͚ͭΔͨΊͷಁաܕϓϩ Ωγ ๏ ಁաܕϓϩΩγʹ͢Δʹ͸DNATͱgetsockoptΛ࢖ͬͨ ๏ SMTPͷಁաܕϓϩΩγͷ࣮૷͸io.Copy͢Δ͚ͩͰ͸μϝͰμ΢ϯάϨʔυ߈ܸΛରԠ ͢Δඞཁ͕͋Γ࣮૷͕গ͠ෳࡶ Conclusion
31. Fukuoka.go #17 Thank you \ʕ⊙౪⚆ʔ/