Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

by https://speakerdeck.com/linyows

Slide 1

Slide 1 text

Fukuoka.go #17 Tomohisa Oda / Jun 10, 2021 Transparent SMTP Proxy in Go GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

Slide 2

Slide 2 text

Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo, inc. Fukuoka.go and Fukuoka.ts Organizer

Slide 3

Slide 3 text

Fukuoka.go #17 @linyows Blog: tomohisaoda.com Af fi liation: GMO pepabo, inc. Fukuoka.go and Fukuoka.ts Organizer ࠷ۙɺNotionʹࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ

Slide 4

Slide 4 text

Fukuoka.go #17 ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ ๏ ϖύϘݚڀॴ ۝भେֶڞಉݚڀ 2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/ ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ In Pepabo…

Slide 5

Slide 5 text

Fukuoka.go #17 Private Projects ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔ Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ ΠͱσϦόϦʔΛߦ͏πʔϧ linuxͷuser໊લղܾΛGitHubͷ Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε ݖݶऀ͔Βߦ͏πʔϧ ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ New!!

Slide 6

Slide 6 text

Fukuoka.go #17 ϝʔϧͷ࢓૊ΈͱϝʔϧϗεςΟϯάͷ՝୊

Slide 7

Slide 7 text

Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 DNS MX TXT(SPF, DKIM, DMARC…)

Slide 8

Slide 8 text

Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 [email protected] DNS MX TXT(SPF, DKIM, DMARC…) [email protected] Bob Alice ϝʔϧϗεςΟϯάʹ͓͍ͯ ϝʔϧΛ૬खઌʹ ౸ୡͤ͞Δ͜ͱ͕ॏཁ

Slide 9

Slide 9 text

Fukuoka.go #17 ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ

Slide 10

Slide 10 text

Fukuoka.go #17 ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ ๏ ड৴ଆαʔόͷՄ༻ੑ ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith %/4#-ʹొ࿥͞ΕΔͳͲ Ϩϐϡςʔγϣϯʢධ൑ʣʹӨڹ͢Δ

Slide 11

Slide 11 text

Fukuoka.go #17 ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢ ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ ๏ ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕ εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Fukuoka.go #17 ✉ Architecture 💻 🖥 Mail Submission Agent :587 Mail Transfer Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP 📱 📱 [email protected] DNS MX TXT(SPF, DKIM, DMARC…) [email protected] Bob Alice ͜ͷ΍ΓͱΓΛ෼ੳͰ͖Δɺ͔ͭ ໰୊Λݕ஌Ͱ͖Ε͹ ૹ৴ΛϒϩοΫͰ͖Δ࢓૊Έ͕ඞཁ

Slide 14

Slide 14 text

Fukuoka.go #17 ࣮ݱํ๏ͷൺֱ

Slide 15

Slide 15 text

Fukuoka.go #17 ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ Mail Transfer Agent :25 Mail Transfer Agent :25 Postfix Postfix [email protected] Transparent Proxy :25 Mail Transfer Agent :25 ? Postfix [email protected] Mail Submission Agent :587 Postfix [email protected] Mail Submission Agent :587 Postfix [email protected] ✉ ✉ ✉ ✉ ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑ ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ 4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏ ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Fukuoka.go #17 GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

Slide 18

Slide 18 text

Fukuoka.go #17 Ͳ͏΍ͬͯಁաܕϓϩΩγΛ࣮ݱ͢Δ͔ ๏ Port25൪ѼΛDNATͰશͯProxyʹૹΔ ๏ getsockopt(2)ͰѼઌมߋલͷΞυϨεΛऔಘ͢Δ

Slide 19

Slide 19 text

Fukuoka.go #17 SMTP Example via: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol 4.51͸04*-ͷͱͯ΋γϯϓϧͳϓϩτίϧ

Slide 20

Slide 20 text

Fukuoka.go #17 Goͷio.CopyΛ࢖͑͹ָউͳΜ͡Όͳ͍ʁ

Slide 21

Slide 21 text

Fukuoka.go #17 Transparent Proxy Go Mail Transfer Agent Postfix [email protected] Mail Submission Agent Postfix [email protected] io.Copy(internalConn, externalConn) io.Copy(externalConn, internalConn)

Slide 22

Slide 22 text

Fukuoka.go #17 PoC ී௨ʹϝʔϧΛૹड৴Ͱ͖ͨͷͰ׬શʹ͏·͍ͬ͘ ͨͱࢥͬͨɻ ͔͠͠ɺHPSPVUJOF಺ͷม਺Λग़ྗͯ͠ΈΔͱɺ ్த͔Βจࣈ͕ಡΊͳ͍ײ͡ʹ

Slide 23

Slide 23 text

Fukuoka.go #17 STARTTLSͷ͜ͱΛ๨Ε͍ͯͨ!!!

Slide 24

Slide 24 text

Fukuoka.go #17 μ΢ϯάϨʔυ߈ܸରԠʹ͢Δ

Slide 25

Slide 25 text

Fukuoka.go #17 ୯ʹJP$PQZͰ͸ͳ͘ 5-4ίωΫγϣϯ·Ͱ ੍ޚ͢Δඞཁ͕͋ͬͨ

Slide 26

Slide 26 text

Fukuoka.go #17 45"355-4ʹରԠͯ͠ͳ͍ Ϩεϙϯεʹ͢Δ

Slide 27

Slide 27 text

Fukuoka.go #17 όοϑΝϦϯάͨ͠σʔλΛ5-4 ίωΫγϣϯͱͦͷޙͷఆܕ΍Γ औΓޙʹૹ৴͢Δ

Slide 28

Slide 28 text

Fukuoka.go #17 Demo: ࣌ؒͳ͍͔΋

Slide 29

Slide 29 text

Fukuoka.go #17 ๏ Integration Testͷख๏Λߟ͑ΔʢPostfix ґଘΛͲ͏͢΂͖͔ʣ ๏ ऩूͨ͠৘ใΛDatabaseʹ஝ੵͱ෼ੳ ๏ IptablesґଘϧʔςΟϯάΛBPF/XDPΛ ࢖͏Α͏ʹ͢Δ ๏ Production΁ΧφϦΞϦϦʔε ࠓޙͷܭը — github.com/linyows/warp

Slide 30

Slide 30 text

Fukuoka.go #17 ๏ ϝʔϧͷ౸ୡੑʹؔΘΔ̐ͭͷ߲໨͕͋Δ ๏ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊͸ड৴ڋ൱ͷཧ༝͕Θ͔Βͳ͍͜ͱ ๏ ϝʔϧૹ৴࣌ͷΞΧ΢ϯτ͝ͱͷৼΔ෣͍ͱड৴ڋ൱ͷؔ܎Λݟ͚ͭΔͨΊͷಁաܕϓϩ Ωγ ๏ ಁաܕϓϩΩγʹ͢Δʹ͸DNATͱgetsockoptΛ࢖ͬͨ ๏ SMTPͷಁաܕϓϩΩγͷ࣮૷͸io.Copy͢Δ͚ͩͰ͸μϝͰμ΢ϯάϨʔυ߈ܸΛରԠ ͢Δඞཁ͕͋Γ࣮૷͕গ͠ෳࡶ Conclusion

Slide 31

Slide 31 text

Fukuoka.go #17 Thank you \ʕ⊙౪⚆ʔ/