Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Transcript

1. Fukuoka.go #17 Tomohisa Oda / Jun 10, 2021 Transparent SMTP

   Proxy in Go GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

2. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo,

   inc. Fukuoka.go and Fukuoka.ts Organizer

3. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo,

   inc. Fukuoka.go and Fukuoka.ts Organizer ࠷ۙɺNotionʹࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ

4. Fukuoka.go #17 ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ ๏ ϖύϘݚڀॴ

   ۝भେֶڞಉݚڀ 2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/ ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ In Pepabo…

5. Fukuoka.go #17 Private Projects ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔ Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ ΠͱσϦόϦʔΛߦ͏πʔϧ linuxͷuser໊લղܾΛGitHubͷ Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε ݖݶऀ͔Βߦ͏πʔϧ

   ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ New!!

6. Fukuoka.go #17 ϝʔϧͷ࢓૊ΈͱϝʔϧϗεςΟϯάͷ՝୊

7. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

   Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 DNS MX TXT(SPF, DKIM, DMARC…)

8. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

   Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ϝʔϧϗεςΟϯάʹ͓͍ͯ
   ϝʔϧΛ૬खઌʹ
   ౸ୡͤ͞Δ͜ͱ͕ॏཁ

9. Fukuoka.go #17 ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ

   ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ

10. Fukuoka.go #17 ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP,

    υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith %/4#-ʹొ࿥͞ΕΔͳͲ
    Ϩϐϡςʔγϣϯʢධ൑ʣʹӨڹ͢Δ

11. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏

    ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊

12. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏

    ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊ ϝʔϧૹ৴࣌ͷͲΜͳৼΔ෣͍͕ड৴ڋ൱ʹࢸ͔ͬͨʁ
    ෼ੳ͠ݕ஌Ͱ͖ΔΑ͏ʹ͍ͨ͠ʂ

13. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

    Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ͜ͷ΍ΓͱΓΛ෼ੳͰ͖Δɺ͔ͭ
    ໰୊Λݕ஌Ͱ͖Ε͹
    ૹ৴ΛϒϩοΫͰ͖Δ࢓૊Έ͕ඞཁ

14. Fukuoka.go #17 ࣮ݱํ๏ͷൺֱ

15. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail

    Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ

16. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail

    Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ

17. Fukuoka.go #17 GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

18. Fukuoka.go #17 Ͳ͏΍ͬͯಁաܕϓϩΩγΛ࣮ݱ͢Δ͔ ๏ Port25൪ѼΛDNATͰશͯProxyʹૹΔ ๏ getsockopt(2)ͰѼઌมߋલͷΞυϨεΛऔಘ͢Δ

19. Fukuoka.go #17 SMTP Example via: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol 4.51͸04*
    -ͷͱͯ΋γϯϓϧͳϓϩτίϧ

20. Fukuoka.go #17 Goͷio.CopyΛ࢖͑͹ָউͳΜ͡Όͳ͍ʁ

21. Fukuoka.go #17 Transparent Proxy Go Mail Transfer Agent Postfix alice@example.com

    Mail Submission Agent Postfix bob@example.net io.Copy(internalConn, externalConn) io.Copy(externalConn, internalConn)

22. Fukuoka.go #17 PoC ී௨ʹϝʔϧΛૹड৴Ͱ͖ͨͷͰ׬શʹ͏·͍ͬ͘ ͨͱࢥͬͨɻ
    ͔͠͠ɺHPSPVUJOF಺ͷม਺Λग़ྗͯ͠ΈΔͱɺ ్த͔Βจࣈ͕ಡΊͳ͍ײ͡ʹ

23. Fukuoka.go #17 STARTTLSͷ͜ͱΛ๨Ε͍ͯͨ!!!

24. Fukuoka.go #17 μ΢ϯάϨʔυ߈ܸରԠʹ͢Δ

25. Fukuoka.go #17 ୯ʹJP$PQZͰ͸ͳ͘
    5-4ίωΫγϣϯ·Ͱ ੍ޚ͢Δඞཁ͕͋ͬͨ

26. Fukuoka.go #17 45"355-4ʹରԠͯ͠ͳ͍ Ϩεϙϯεʹ͢Δ

27. Fukuoka.go #17 όοϑΝϦϯάͨ͠σʔλΛ5-4 ίωΫγϣϯͱͦͷޙͷఆܕ΍Γ औΓޙʹૹ৴͢Δ

28. Fukuoka.go #17 Demo: ࣌ؒͳ͍͔΋

29. Fukuoka.go #17 ๏ Integration Testͷख๏Λߟ͑ΔʢPostfix ґଘΛͲ͏͢΂͖͔ʣ ๏ ऩूͨ͠৘ใΛDatabaseʹ஝ੵͱ෼ੳ ๏ IptablesґଘϧʔςΟϯάΛBPF/XDPΛ

    ࢖͏Α͏ʹ͢Δ ๏ Production΁ΧφϦΞϦϦʔε ࠓޙͷܭը — github.com/linyows/warp

30. Fukuoka.go #17 ๏ ϝʔϧͷ౸ୡੑʹؔΘΔ̐ͭͷ߲໨͕͋Δ ๏ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊͸ड৴ڋ൱ͷཧ༝͕Θ͔Βͳ͍͜ͱ ๏ ϝʔϧૹ৴࣌ͷΞΧ΢ϯτ͝ͱͷৼΔ෣͍ͱड৴ڋ൱ͷؔ܎Λݟ͚ͭΔͨΊͷಁաܕϓϩ Ωγ ๏

    ಁաܕϓϩΩγʹ͢Δʹ͸DNATͱgetsockoptΛ࢖ͬͨ ๏ SMTPͷಁաܕϓϩΩγͷ࣮૷͸io.Copy͢Δ͚ͩͰ͸μϝͰμ΢ϯάϨʔυ߈ܸΛରԠ ͢Δඞཁ͕͋Γ࣮૷͕গ͠ෳࡶ Conclusion

31. Fukuoka.go #17 Thank you \ʕ⊙౪⚆ʔ/