Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Fukuoka.go #17 Jun 10, 2021 の発表資料です。

5d769d109697012317c09c6a27a6a4bf?s=128

linyows

June 10, 2021
Tweet

Transcript

  1. Fukuoka.go #17 Tomohisa Oda / Jun 10, 2021 Transparent SMTP

    Proxy in Go GoͰͭ͘ΔಁաܕSMTPϓϩΩγ
  2. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo,

    inc. Fukuoka.go and Fukuoka.ts Organizer
  3. Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo,

    inc. Fukuoka.go and Fukuoka.ts Organizer ࠷ۙɺNotionʹࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ
  4. Fukuoka.go #17 ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ ๏ ϖύϘݚڀॴ

    ۝भେֶڞಉݚڀ 2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/ ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ In Pepabo…
  5. Fukuoka.go #17 Private Projects ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔ Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ ΠͱσϦόϦʔΛߦ͏πʔϧ linuxͷuser໊લղܾΛGitHubͷ Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε ݖݶऀ͔Βߦ͏πʔϧ

    ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ New!!
  6. Fukuoka.go #17 ϝʔϧͷ࢓૊ΈͱϝʔϧϗεςΟϯάͷ՝୊

  7. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

    Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 DNS MX TXT(SPF, DKIM, DMARC…)
  8. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

    Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ϝʔϧϗεςΟϯάʹ͓͍ͯ ϝʔϧΛ૬खઌʹ ౸ୡͤ͞Δ͜ͱ͕ॏཁ
  9. Fukuoka.go #17 ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ

    ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ
  10. Fukuoka.go #17 ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP,

    υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith %/4#-ʹొ࿥͞ΕΔͳͲ Ϩϐϡςʔγϣϯʢධ൑ʣʹӨڹ͢Δ
  11. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏

    ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊
  12. Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏

    ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊ ϝʔϧૹ৴࣌ͷͲΜͳৼΔ෣͍͕ड৴ڋ൱ʹࢸ͔ͬͨʁ ෼ੳ͠ݕ஌Ͱ͖ΔΑ͏ʹ͍ͨ͠ʂ
  13. Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587

    Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 alice@example.com DNS MX TXT(SPF, DKIM, DMARC…) bob@example.net Bob Alice ͜ͷ΍ΓͱΓΛ෼ੳͰ͖Δɺ͔ͭ ໰୊Λݕ஌Ͱ͖Ε͹ ૹ৴ΛϒϩοΫͰ͖Δ࢓૊Έ͕ඞཁ
  14. Fukuoka.go #17 ࣮ݱํ๏ͷൺֱ

  15. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail

    Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑ ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ
  16. Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail

    Transfer Agent :25 Postfix Postfix alice@example.com Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix alice@example.com Mail Submission Agent :587 Postfix bob@example.net Mail Submission Agent :587 Postfix bob@example.net ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑ ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ
  17. Fukuoka.go #17 GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

  18. Fukuoka.go #17 Ͳ͏΍ͬͯಁաܕϓϩΩγΛ࣮ݱ͢Δ͔ ๏ Port25൪ѼΛDNATͰશͯProxyʹૹΔ ๏ getsockopt(2)ͰѼઌมߋલͷΞυϨεΛऔಘ͢Δ

  19. Fukuoka.go #17 SMTP Example via: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol 4.51͸04*-ͷͱͯ΋γϯϓϧͳϓϩτίϧ

  20. Fukuoka.go #17 Goͷio.CopyΛ࢖͑͹ָউͳΜ͡Όͳ͍ʁ

  21. Fukuoka.go #17 Transparent Proxy Go Mail Transfer Agent Postfix alice@example.com

    Mail Submission Agent Postfix bob@example.net io.Copy(internalConn, externalConn) io.Copy(externalConn, internalConn)
  22. Fukuoka.go #17 PoC ී௨ʹϝʔϧΛૹड৴Ͱ͖ͨͷͰ׬શʹ͏·͍ͬ͘ ͨͱࢥͬͨɻ ͔͠͠ɺHPSPVUJOF಺ͷม਺Λग़ྗͯ͠ΈΔͱɺ ్த͔Βจࣈ͕ಡΊͳ͍ײ͡ʹ

  23. Fukuoka.go #17 STARTTLSͷ͜ͱΛ๨Ε͍ͯͨ!!!

  24. Fukuoka.go #17 μ΢ϯάϨʔυ߈ܸରԠʹ͢Δ

  25. Fukuoka.go #17 ୯ʹJP$PQZͰ͸ͳ͘ 5-4ίωΫγϣϯ·Ͱ ੍ޚ͢Δඞཁ͕͋ͬͨ

  26. Fukuoka.go #17 45"355-4ʹରԠͯ͠ͳ͍ Ϩεϙϯεʹ͢Δ

  27. Fukuoka.go #17 όοϑΝϦϯάͨ͠σʔλΛ5-4 ίωΫγϣϯͱͦͷޙͷఆܕ΍Γ औΓޙʹૹ৴͢Δ

  28. Fukuoka.go #17 Demo: ࣌ؒͳ͍͔΋

  29. Fukuoka.go #17 ๏ Integration Testͷख๏Λߟ͑ΔʢPostfix ґଘΛͲ͏͢΂͖͔ʣ ๏ ऩूͨ͠৘ใΛDatabaseʹ஝ੵͱ෼ੳ ๏ IptablesґଘϧʔςΟϯάΛBPF/XDPΛ

    ࢖͏Α͏ʹ͢Δ ๏ Production΁ΧφϦΞϦϦʔε ࠓޙͷܭը — github.com/linyows/warp
  30. Fukuoka.go #17 ๏ ϝʔϧͷ౸ୡੑʹؔΘΔ̐ͭͷ߲໨͕͋Δ ๏ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊͸ड৴ڋ൱ͷཧ༝͕Θ͔Βͳ͍͜ͱ ๏ ϝʔϧૹ৴࣌ͷΞΧ΢ϯτ͝ͱͷৼΔ෣͍ͱड৴ڋ൱ͷؔ܎Λݟ͚ͭΔͨΊͷಁաܕϓϩ Ωγ ๏

    ಁաܕϓϩΩγʹ͢Δʹ͸DNATͱgetsockoptΛ࢖ͬͨ ๏ SMTPͷಁաܕϓϩΩγͷ࣮૷͸io.Copy͢Δ͚ͩͰ͸μϝͰμ΢ϯάϨʔυ߈ܸΛରԠ ͢Δඞཁ͕͋Γ࣮૷͕গ͠ෳࡶ Conclusion
  31. Fukuoka.go #17 Thank you \ʕ⊙౪⚆ʔ/