Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Goでつくる透過型SMTPプロキシ / Transparent SMTP proxy in Go

Fukuoka.go #17 Jun 10, 2021 の発表資料です。

linyows

June 10, 2021
Tweet

More Decks by linyows

Other Decks in Programming

Transcript

  1. Fukuoka.go #17
    Tomohisa Oda / Jun 10, 2021
    Transparent SMTP Proxy in Go
    GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

    View full-size slide

  2. Fukuoka.go #17
    @linyows
    Blog: tomohisaoda.com


    Af
    fi
    liation: GMO pepabo, inc.


    Fukuoka.go and Fukuoka.ts Organizer

    View full-size slide

  3. Fukuoka.go #17
    @linyows
    Blog: tomohisaoda.com


    Af
    fi
    liation: GMO pepabo, inc.


    Fukuoka.go and Fukuoka.ts Organizer
    ࠷ۙɺNotionʹࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ

    View full-size slide

  4. Fukuoka.go #17
    ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead


    ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ


    ๏ ϖύϘݚڀॴ ۝भେֶڞಉݚڀ


    2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/


    ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
    In Pepabo…

    View full-size slide

  5. Fukuoka.go #17
    Private Projects
    ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔
    Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ
    ΠͱσϦόϦʔΛߦ͏πʔϧ
    linuxͷuser໊લղܾΛGitHubͷ
    Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε
    ݖݶऀ͔Βߦ͏πʔϧ
    ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ
    ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ
    ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ
    New!!

    View full-size slide

  6. Fukuoka.go #17
    ϝʔϧͷ࢓૊ΈͱϝʔϧϗεςΟϯάͷ՝୊

    View full-size slide

  7. Fukuoka.go #17

    Architecture
    💻 🖥
    Mail Submission


    Agent
    :587
    Mail Transfer


    Agent
    :25
    Mail Delivery


    Agent
    :993
    Postfix Postfix Dovecot


    SMTP
    SMTP
    LMTP
    IMAP
    📱
    📱
    DNS
    MX TXT(SPF, DKIM, DMARC…)

    View full-size slide

  8. Fukuoka.go #17

    Architecture
    💻 🖥
    Mail Submission


    Agent
    :587
    Mail Transfer


    Agent
    :25
    Mail Delivery


    Agent
    :993
    Postfix Postfix Dovecot


    SMTP
    SMTP
    LMTP
    IMAP
    📱
    📱
    [email protected]
    DNS
    MX TXT(SPF, DKIM, DMARC…)
    [email protected]
    Bob Alice
    ϝʔϧϗεςΟϯάʹ͓͍ͯ
    ϝʔϧΛ૬खઌʹ
    ౸ୡͤ͞Δ͜ͱ͕ॏཁ

    View full-size slide

  9. Fukuoka.go #17
    ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ
    via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith
    ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ


    ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ


    ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ


    ๏ ड৴ଆαʔόͷՄ༻ੑ

    View full-size slide

  10. Fukuoka.go #17
    ๏ ϝʔϧΠϯϑϥͷ݈શੑʢωοτϫʔΫͱDNSʣ


    ๏ ϝʔϧͷίϯςϯπʢఴ෇ϑΝΠϧ, ຊจ, ϔομʔͳͲʣ


    ๏ ϝʔϧͷૹ৴ݩʢૹ৴ݩIP, υϝΠϯ, ૹ৴ྔͳͲʣ


    ๏ ड৴ଆαʔόͷՄ༻ੑ
    ϝʔϧͷ౸ୡੑʹӨڹΛ༩͑Δ̐ͭͷ͜ͱ
    via: https://sendgrid.kke.co.jp/blog/?p=4494 by Jillian Smith
    %/4#-ʹొ࿥͞ΕΔͳͲ
    Ϩϐϡςʔγϣϯʢධ൑ʣʹӨڹ͢Δ

    View full-size slide

  11. Fukuoka.go #17
    ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢
    ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ


    ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ


    ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ


    ๏ ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕
    εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ
    ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊

    View full-size slide

  12. Fukuoka.go #17
    ๏ ༷ʑͳϢʔβ͕࢖͏ϚϧνςφϯτͰ͋ΔͨΊɺϝʔϧΛ೔ৗతʹͨ͘͞Μૹ৴͢
    ΔΞΧ΢ϯτ΋͋Ε͹ɺීஈ͸গͳ͍͕ͨ·ʹେྔૹ৴͢ΔΞΧ΢ϯτ΋͋Δ


    ๏ ͦ΋ͦ΋ΞΧ΢ϯτΛ৐ͬऔΒΕΔέʔε΋͋Δ


    ๏ ड৴ऀ͕ͳͥड͚औΓڋ൱Λͨ͠ͷ͔͕෼͔Γʹ͍͘ʢDMARCରԠ͕͋Ε͹ผʣ


    ๏ ϝʔϧ͸૒ํͷ΍ΓͱΓͳͷͰɺૹ৴ଆ͕εύϜߦҝͱࢥ͍ͬͯͳͯ͘΋ड৴ऀ͕
    εύϜߦҝͩͱײ͡Δ͜ͱ΋͋Γ͏Δ
    ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊
    ϝʔϧૹ৴࣌ͷͲΜͳৼΔ෣͍͕ड৴ڋ൱ʹࢸ͔ͬͨʁ
    ෼ੳ͠ݕ஌Ͱ͖ΔΑ͏ʹ͍ͨ͠ʂ

    View full-size slide

  13. Fukuoka.go #17

    Architecture
    💻 🖥
    Mail Submission


    Agent
    :587
    Mail Transfer


    Agent
    :25
    Mail Delivery


    Agent
    :993
    Postfix Postfix Dovecot


    SMTP
    SMTP
    LMTP
    IMAP
    📱
    📱
    [email protected]
    DNS
    MX TXT(SPF, DKIM, DMARC…)
    [email protected]
    Bob Alice
    ͜ͷ΍ΓͱΓΛ෼ੳͰ͖Δɺ͔ͭ
    ໰୊Λݕ஌Ͱ͖Ε͹
    ૹ৴ΛϒϩοΫͰ͖Δ࢓૊Έ͕ඞཁ

    View full-size slide

  14. Fukuoka.go #17
    ࣮ݱํ๏ͷൺֱ

    View full-size slide

  15. Fukuoka.go #17
    ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ
    Mail Transfer


    Agent
    :25
    Mail Transfer


    Agent
    :25
    Postfix Postfix
    [email protected]
    Transparent


    Proxy
    :25
    Mail Transfer


    Agent
    :25
    ? Postfix
    [email protected]
    Mail Submission


    Agent
    :587
    Postfix
    [email protected]
    Mail Submission


    Agent
    :587
    Postfix
    [email protected]
    ✉ ✉


    ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ
    4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏
    ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ

    View full-size slide

  16. Fukuoka.go #17
    ϝʔϧϦϨʔ vs. ಁաܕϓϩΩγ
    Mail Transfer


    Agent
    :25
    Mail Transfer


    Agent
    :25
    Postfix Postfix
    [email protected]
    Transparent


    Proxy
    :25
    Mail Transfer


    Agent
    :25
    ? Postfix
    [email protected]
    Mail Submission


    Agent
    :587
    Postfix
    [email protected]
    Mail Submission


    Agent
    :587
    Postfix
    [email protected]
    ✉ ✉


    ϝʔϧϦϨʔΛߦ͏ͱϗοϓ͕૿͑
    ϝʔϧΩϡʔͷ؅ཧ͕ෳࡶʹͳΔ
    4.51ड͚ྲྀ͢ಁաܕϓϩΩγͩͱ
    ϝʔϧΩϡʔ؅ཧ͕ݱߦͱมΘΒͣ
    γϯϓϧͰ͋Δɻ͔͠͠ɺͦͷΑ͏
    ͳϛυϧ΢ΣΞ͕ݟ౰ͨΒͳ͍ɻ

    View full-size slide

  17. Fukuoka.go #17
    GoͰͭ͘ΔಁաܕSMTPϓϩΩγ

    View full-size slide

  18. Fukuoka.go #17
    Ͳ͏΍ͬͯಁաܕϓϩΩγΛ࣮ݱ͢Δ͔
    ๏ Port25൪ѼΛDNATͰશͯProxyʹૹΔ


    ๏ getsockopt(2)ͰѼઌมߋલͷΞυϨεΛऔಘ͢Δ

    View full-size slide

  19. Fukuoka.go #17
    SMTP Example
    via: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol
    4.51͸04*-ͷͱͯ΋γϯϓϧͳϓϩτίϧ

    View full-size slide

  20. Fukuoka.go #17
    Goͷio.CopyΛ࢖͑͹ָউͳΜ͡Όͳ͍ʁ

    View full-size slide

  21. Fukuoka.go #17
    Transparent


    Proxy
    Go
    Mail Transfer


    Agent
    Postfix
    [email protected]
    Mail Submission


    Agent
    Postfix
    [email protected]
    io.Copy(internalConn, externalConn)
    io.Copy(externalConn, internalConn)

    View full-size slide

  22. Fukuoka.go #17
    PoC
    ී௨ʹϝʔϧΛૹड৴Ͱ͖ͨͷͰ׬શʹ͏·͍ͬ͘
    ͨͱࢥͬͨɻ
    ͔͠͠ɺHPSPVUJOF಺ͷม਺Λग़ྗͯ͠ΈΔͱɺ
    ్த͔Βจࣈ͕ಡΊͳ͍ײ͡ʹ

    View full-size slide

  23. Fukuoka.go #17
    STARTTLSͷ͜ͱΛ๨Ε͍ͯͨ!!!

    View full-size slide

  24. Fukuoka.go #17
    μ΢ϯάϨʔυ߈ܸରԠʹ͢Δ

    View full-size slide

  25. Fukuoka.go #17
    ୯ʹJP$PQZͰ͸ͳ͘
    5-4ίωΫγϣϯ·Ͱ
    ੍ޚ͢Δඞཁ͕͋ͬͨ

    View full-size slide

  26. Fukuoka.go #17
    45"355-4ʹରԠͯ͠ͳ͍
    Ϩεϙϯεʹ͢Δ

    View full-size slide

  27. Fukuoka.go #17
    όοϑΝϦϯάͨ͠σʔλΛ5-4
    ίωΫγϣϯͱͦͷޙͷఆܕ΍Γ
    औΓޙʹૹ৴͢Δ

    View full-size slide

  28. Fukuoka.go #17
    Demo: ࣌ؒͳ͍͔΋

    View full-size slide

  29. Fukuoka.go #17
    ๏ Integration Testͷख๏Λߟ͑ΔʢPostfix
    ґଘΛͲ͏͢΂͖͔ʣ


    ๏ ऩूͨ͠৘ใΛDatabaseʹ஝ੵͱ෼ੳ


    ๏ IptablesґଘϧʔςΟϯάΛBPF/XDPΛ
    ࢖͏Α͏ʹ͢Δ


    ๏ Production΁ΧφϦΞϦϦʔε
    ࠓޙͷܭը — github.com/linyows/warp

    View full-size slide

  30. Fukuoka.go #17
    ๏ ϝʔϧͷ౸ୡੑʹؔΘΔ̐ͭͷ߲໨͕͋Δ


    ๏ ϝʔϧϗεςΟϯάʹ͓͚Δ՝୊͸ड৴ڋ൱ͷཧ༝͕Θ͔Βͳ͍͜ͱ


    ๏ ϝʔϧૹ৴࣌ͷΞΧ΢ϯτ͝ͱͷৼΔ෣͍ͱड৴ڋ൱ͷؔ܎Λݟ͚ͭΔͨΊͷಁաܕϓϩ
    Ωγ


    ๏ ಁաܕϓϩΩγʹ͢Δʹ͸DNATͱgetsockoptΛ࢖ͬͨ


    ๏ SMTPͷಁաܕϓϩΩγͷ࣮૷͸io.Copy͢Δ͚ͩͰ͸μϝͰμ΢ϯάϨʔυ߈ܸΛରԠ
    ͢Δඞཁ͕͋Γ࣮૷͕গ͠ෳࡶ
    Conclusion

    View full-size slide

  31. Fukuoka.go #17
    Thank you \ʕ⊙౪⚆ʔ/

    View full-size slide