透過型SMTPプロキシによるメール送信集約とキュー輻輳回避の検討 / A Study on Aggregation of Email Transfer and Avoidance of QueueCongestion using a Transparent SMTP Proxy

2021年7月2日 マルチメディア，分散，協調とモバイル(DICOMO2021)シンポジウムでの発表資料です。
https://tsys.jp/dicomo/2021/program/program.html#7H

Transcript

1. ಁաܕSMTPϓϩΩγʹΑΔϝʔϧ
   ૹ৴ू໿ͱΩϡʔ᫔᫓ճආͷݕ౼
   খా஌ԝ, ኍ઒༏, ۙ౻Ӊஐ࿕(GMOϖύϘ), ౢ٢ོ෉, ּݪٛߊ(۝भେֶ)
   ϚϧνϝσΟΞɺ෼ࢄɺڠௐͱϞόΠϧ (DICOMO 2021) γϯϙδ΢Ϝ — 2021-07-02
   

   View Slide

2. 1. എܠͱ໨త
   
   
   2. ఏҊख๏ͱ༧උ࣮ݧ
   
   
   3. ֓೦࣮ূ
   
   
   4. ·ͱΊͱࠓޙͷ՝୊
   ໨࣍
   Tomohisa Oda
   
   
   ID: @linyows
   
   
   ॴଐ: GMOϖύϘגࣜձࣾ
   
   
   ϗεςΟϯάࣄۀ෦
   

   View Slide

3. എܠͱ໨త
   Section 1
   

   View Slide

4. എܠͱ໨త
   ٻΊΒΕΔϝʔϧͷ҆ఆੑ΍ηΩϡϦςΟ
   • ϝʔϧ͸ґવͱͯ͠޿͘࢖ΘΕ͓ͯΓɺWebαʔϏεͷೝূ༻్ͳͲॏཁͳ໾
   ׂΛ࣋ͭ
   
   
   • ϝʔϧϗεςΟϯάࣄۀऀ͸ϋʔυ΢ΣΞͷϦιʔεޮ཰ΛߴΊΔͨΊɺΞΧ
   ΢ϯτ΍υϝΠϯΛߴूੵԽ͠௿ίετʹͳΔΑ͏ʹ͍ͯ͠Δ
   
   
   • ·ͨSPAMϝʔϧૹ৴΍େྔૹ৴ͳͲʹΑΔϨϐϡςʔγϣϯ௿Լ͔Βϝʔϧ
   ഑ૹ஗ԆʹͳΒͳ͍Α͏౒ྗ͍ͯ͠Δ
   
   
   • ϝʔϧϗεςΟϯάࣄۀऀ͸ϝʔϧͷ҆ఆੑ΍ηΩϡϦςΟΛܧଓతʹ୲อ͠
   ͳ͚Ε͹ͳΒͳ͍
   4
   

   View Slide

5. • άϩʔόϧIPv4ΞυϨε਺ͷ੍໿ʹΑΓɺ֎෦΁ͷϝʔϧૹ৴αʔό͸ͳΔ
   ΂͘ू໿͢Δߏ੒
   എܠͱ໨త
   ϝʔϧૹ৴ʹ͓͚Δैདྷख๏
   5
   Mail User Agent Mail Submission Agent
   Mail Transfer Agent
   Mail Delivery Agent
   Mail Transfer Agent (MX)
   Mail User Agent
   Mail Hosting
   📧
   👩💻 👨💻
   133.54.XXX.XXX
   

   View Slide

6. ू໿ͨ͠ϝʔϧૹ৴αʔόͰϝʔϧૹ৴ΩϡʔΛ؅ཧ͢Δ͜ͱʹͳΔͷͰ໰୊
   ͷ͋ΔΞΧ΢ϯτʹΑΓૹ৴Ωϡʔ͕٧·Δͱɺ໰୊ͷͳ͍ΞΧ΢ϯτʹ΋Ө
   ڹ͠ૹ৴஗Ԇʹͭͳ͕Δ͜ͱ͕ଟ͍
   
   
   • ର޲ͷMTA͔ΒΈͯϨϐϡςʔγϣϯ͕௿͘ड৴ϨʔτΛ੍ݶ͞Εͨ৔߹
   
   
   • ૹ৴αʔόͷIP͕͋ΔDNSBLʹొ࿥͞Εɺର޲ͷMTA͕ͦͷDNSBLΛར༻͠
   ͍ͯͯड৴ڋ൱Λ͞Εͨ৔߹
   
   
   ର޲MTAͷৼΔ෣͍มԽΛݕ஌͠ʹ͍͘
   എܠͱ໨త
   ैདྷख๏ͷ՝୊
   6
   

   View Slide

7. 7
   Mail User Agent
   Mail User Agent
   Mail Hosting
   Mail Transfer Agent (MX)
   Mail Transfer Agent (MX)
   Mail Transfer Agent (MX)
   Mail Submission Agent
   Mail Transfer Agent
   Mail Delivery Agent
   Mail Transfer Agent (MX)
   Mail User Agent
   📧
   👩💻 👨💻
   🧟
   📧
   📧
   📧
   📧
   ✉
   🦠
   ✉
   🎣
   📧
   📧
   📧
   📧
   📧
   📧
   📧
   📧
   📧
   💭
   ❓
   ❌
   👨🔧
   🗑
   ✉
   🎣
   ✉
   🎣 Operation
   133.5.XXX.XXX
   

   View Slide

8. ߴूੵϚϧνςφϯτܕͷϝʔϧϗεςΟϯάʹ͓͍ͯϝʔϧૹ৴αʔόͷߏ
   ੒ख๏Λݕ౼͢Δ
   
   
   • ར༻͢ΔάϩʔόϧIPΞυϨεΛݮΒ͢
   
   
   • ໰୊ͷಛఆϝʔϧૹ৴ͷӨڹΛݶఆతʹ͢Δ
   
   
   • ո͍͠ϝʔϧૹ৴΍ର޲ͷMTAϨεϙϯεͳͲ໰୊ݕ஌ʹܨ͕ΔΑ͏ͳ৘ใ
   Λऔಘ͢Δ
   എܠͱ໨త
   ຊݚڀͷ໨త
   8
   

   View Slide

9. ఏҊख๏ͱ༧උ࣮ݧ
   Section 2
   

   View Slide

10. • ϝʔϧૹ৴αʔόΛςφϯτຖίϯςφʹ෼཭͢Δ*1ઌߦݚڀΛࢀর
    
    
    • ίϯςφ͔Β֎΁ͷSMTP௨৴ΛಁաܕSMTPϓϩΩγʹసૹ͢Δ
    
    
    • ಁաܕSMTPϓϩΩγ͸MXϗετʹ઀ଓ͢Δ
    
    
    • ಁաܕSMTPϓϩΩγͰ͸SMTPͷίϚϯυ΍MXϗετ͔ΒͷϨεϙϯεͳͲ
    ΛϩάͳͲʹग़ྗ͢Δ
    ఏҊख๏ͱ༧උ࣮ݧ
    ಁաܕSMTPϓϩΩγͷఏҊ
    10
    *1: দຊ ྄հ, খా ஌ԝ (GMOϖύϘגࣜձࣾ ϖύϘݚڀॴ), ּݪ ٛߊ, ౢ٢ ོ෉ (۝भେֶ ৘ใج൫ݚڀ։ൃηϯλʔ), ۚࢠ ߊհ (۝भେֶ αΠ
    όʔηΩϡϦςΟηϯλʔ), ܀ྛ ݈ଠ࿠ (GMOϖύϘגࣜձࣾ ϖύϘݚڀॴ), Ԭଜ ߞೋ (۝भେֶ ৘ใج൫ݚڀ։ൃηϯλʔ)ʮਫ਼៛ͳղੳͱ੍ޚ͕
    Մೳͳ ߃ৗੑͷ͋Δϝʔϧج൫ʯϚϧνϝσΟΞɼ෼ࢄɼڠௐͱϞόΠϧ(DICOMO2018)γϯϙδ΢Ϝ, pp. 1383 - 1389, July 2018
    

    View Slide

11. 11
    Mail User Agent
    Mail Hosting
    Mail Submission Agent Mail Delivery Agent
    Mail Transfer Agent (MX)
    Mail User Agent
    👩💻 👨💻
    Transparent SMTP Proxy
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Routing
    📧
    📧
    📧
    133.5.XXX.XXX
    

    View Slide

12. • ਖ਼ৗͳPost
    fi
    xѼʹϝʔϧΛૹΓଓ͚ɺ߹ΘͤͯSMTPͷԠ౴͕ۃ୺ʹ஗͍
    mxtarpitѼʹϝʔϧΛૹΓɺड৴஗ԆΛൺֱͨ͠
    ૹ৴αʔό ड৴αʔό
    ఏҊख๏ͱ༧උ࣮ݧ
    ϝʔϧૹ৴Ωϡʔ෼཭ʹΑΔ᫔᫓ͷӨڹ͕ݶఆ͞ΕΔ࣮ݧ
    12
    Benchmark Script
    Post
    fi
    x mxtarpit
    Post
    fi
    x
    Shared
    Separated
    Post
    fi
    x Post
    fi
    x
    

    View Slide

13. 13
    ϝʔϧड৴਺ͱ࣌ؒܦա
    

    View Slide

14. • ίϯςφͰૹ৴αʔόΛ෼ࢄ͠ͳ͕Β΋ग़ޱΛू໿͢Δ͜ͱͰɺςφϯτ෼
    ཭ͱ࢖༻͢ΔάϩʔόϧIPΞυϨεΛݮΒ͢͜ͱͷཱ͕྆Ͱ͖Δ
    
    
    • ϝʔϧૹ৴Ωϡʔ͸ू໿͞Εͳ͍ͷͰΩϡʔ᫔᫓ΛճආͰ͖ɺΩϡʔ٧·Γ
    ͸ಛఆςφϯτʹݶఆͰ͖Δ
    
    
    • ૹ৴઀ଓݩIP΍MXϗετͷϨεϙϯεϝοηʔδͳͲɺϝʔϧૹ৴ʹؔΘΔ
    ৘ใऩूʹΑΓո͍͠ৼΔ෣͍ͷΞΧ΢ϯτͷݕ஌ͱ੍ޚͰ͖ΔΑ͏ʹͳΔ
    ఏҊख๏ͱ༧උ࣮ݧ
    ಁաܕSMTPϓϩΩγʹΑΔԸܙ
    14
    

    View Slide

15. ֓೦࣮ূ
    Section 3
    

    View Slide

16. Output Log
    iptables
    • ಁաܕSMTPϓϩΩγ͸SMTPίϚϯυ΍MXϗετͷϨ
    εϙϯεΛϩάʹग़ྗ͢Δ
    
    
    • ϝʔϧૹ৴ίϯςφͱಁաܕSMTPϓϩΩγ͕ಉҰϗε
    τͰ͋ΔલఏͰͦͷؒͷ௨৴͸ฏจɺಁաܕSMTPϓϩ
    ΩγͱMXϗετ͸STARTTLSରԠΛ͢Δ
    
    
    • ͦͷͨΊɺϝʔϧૹ৴αʔόʹSTARTTLSΛඇରԠͱͯ͠
    ϨεϙϯεΛॻ͖׵͑TLSμ΢ϯάϨʔυΛߦ͏
    
    
    ϦϙδτϦ: https://github.com/linyows/warp
    ֓೦࣮ূ
    ϓϩτλΠϓͷ࣮૷
    16
    Mail Transfer Agent (MX)
    Transparent SMTP Proxy
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Mail Transfer
    Agent Container
    Host
    

    View Slide

17. 17
    ϓϩτλΠϓ࣮૷ͷγʔέϯεਤ
    

    View Slide

18. 2021/02/06 14:50:48 connected from 192.168.30.40:5749
    3
    
    2021/02/06 14:50:48 connected to 192.168.30.50:2
    5
    
    2021/02/06 14:50:48 <- 220 receiver ESMTP Postfix (Ubuntu)\r\
    n
    
    2021/02/06 14:50:48 -> EHLO sender\r\
    n
    
    2021/02/06 14:50:48 |< 250-receiver\r\n250-PIPELINING\r\n250-SIZE 10240000\r\n250-VRFY\r\n250-ETRN\r\n250-STARTTLS\r\n250-
    ENHANCEDSTATUSCODES\r\n250-8BITMIME\r\n250-DSN\r\n250-SMTPUTF8\r\n250 CHUNKING\r\
    n
    
    2021/02/06 14:50:48 <- 250-receiver\r\n250-PIPELINING\r\n250-SIZE 10240000\r\n250-VRFY\r\n250-ETRN\r\n250-
    ENHANCEDSTATUSCODES\r\n250-8BITMIME\r\n250-DSN\r\n250-SMTPUTF8\r\n250 CHUNKING\r\
    n
    
    2021/02/06 14:50:48 |> STARTTLS\r\
    n
    
    2021/02/06 14:50:48 >| MAIL FROM: SIZE=327\r\nRCPT TO: ORCPT=rfc822;root@receiver\r\nDATA\r\
    n
    
    2021/02/06 14:50:48 |< 220 2.0.0 Ready to start TLS\r\
    n
    
    2021/02/06 14:50:48 |> EHLO sender\r\
    n
    
    2021/02/06 14:50:48 pipe locked for tls connectio
    n
    
    2021/02/06 14:50:48 |< 250-receiver\r\n250-PIPELINING\r\n250-SIZE 10240000\r\n250-VRFY\r\n250-ETRN\r\n250-
    ENHANCEDSTATUSCODES\r\n250-8BITMIME\r\n250-DSN\r\n250-SMTPUTF8\r\n250 CHUNKING\r\
    n
    
    2021/02/06 14:50:48 tls connected, to pipe unlocke
    d
    
    2021/02/06 14:50:48 -> MAIL FROM: SIZE=327\r\nRCPT TO: ORCPT=rfc822;root@receiver\r\nDATA\r\
    n
    
    2021/02/06 14:50:48 <- 250 2.1.0 Ok\r\n250 2.1.5 Ok\r\n354 End data with .\r\
    n
    
    2021/02/06 14:50:48 -> Received: from sender (localhost [127.0.0.1])\r\n by sender (Postfix) with SMTP id
    45B113EA9B\r\n for ; Sat, 6 Feb 2021 14:50:48 +0000 (UTC)\r\nFrom: \r\nTo:
    \r\nDate: Sat, 6 Feb 2021 14:50:48 +0000 (UTC)\r\nMessage-Id: \r\nSubject: Hi,
    Receiver from Sender\r\n\r\nXXXXXXXXXX\r\n.\r\nQUIT\r\
    n
    
    2021/02/06 14:50:48 <- 250 2.0.0 Ok: queued as 76DAD4113D\r\n221 2.0.0 Bye\r\
    n
    
    2021/02/06 14:50:48 connections closed
    ಁաܕSMTPϓϩΩγ͕ग़ྗ͢Δϩά
    

    View Slide

19. ·ͱΊͱࠓޙͷ՝୊
    Section 4
    

    View Slide

20. • ϝʔϧϗεςΟϯάʹ͓͚Δϝʔϧૹ৴ػೳͷ՝୊͸ɺ࢖༻͢ΔIPΞυϨεΛ
    ݮΒͨ͢Ίʹߦ͏ςφϯτू໿Ͱൃੜ͢Δϝʔϧૹ৴Ωϡʔͷ᫔᫓Ͱ͋Δ
    
    
    • ͦ͜Ͱϝʔϧૹ৴ू໿༻ͷಁաܕSMTPϓϩΩγΛఏҊͨ͠
    
    
    • ϝʔϧૹ৴Ωϡʔͷ෼཭Ͱɺϝʔϧૹ৴஗ԆͷճආΛ֬ೝ͢Δ༧උ࣮ݧΛߦ
    ͳͬͨ
    
    
    • ϓϩτλΠϓ࣮૷ͱಈ࡞ݕূʹΑΓఏҊख๏ͷ֓೦͕࣮ূ͕Ͱ͖ͨ
    ·ͱΊͱࠓޙͷ՝୊
    ຊݚڀͷ·ͱΊ
    20
    

    View Slide

21. • ϝʔϧϗεςΟϯάʹ͓͍ͯ໰୊ͱͳΔΑ͏ͳৼΔ෣͍Λ͢ΔΞΧ΢ϯτΛ
    ݕ஌͢ΔͨΊʹ͸ɺಁաܕSMTPϓϩΩγͰऩू͢ΔMXϗετͷϨεϙϯε
    Ҏ֎ʹϨεϙϯεʹཁͨ࣌ؒ͠ͳͲ௥Ճͷ৘ใΛऔಘ͢Δඞཁ͕͋Δ
    
    
    • औಘͨ͠৘ใΛDatabaseԽ͠ΞΧ΢ϯτ΍υϝΠϯΛείΞϦϯά͢Δ͜ͱ
    Ͱɺར༻͢ΔάϩʔόϧIPΞυϨεΛ࢖͍෼͚Ͱ͋ͬͨΓɺ໰୊͕ղܾ͞ΕΔ
    ·Ͱ௨৴ϨʔτΛԼ͛Δͱ͍ͬͨ͜ͱ͕Ͱ͖Δ
    
    
    • ·ͨɺର޲ͷMTA͔Βड৴੍ݶ͞Ε͍ͯΔ͜ͱͷݕ஌͕ՄೳͱͳΔ
    ·ͱΊͱࠓޙͷ՝୊
    ࠓޙͷ՝୊
    21
    

    View Slide

22. ँɹࣙ
    ຊݚڀ͸ɺJSPSՊݚඅJP20K11791ʮܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗε
    ςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽʯͷॿ੒Λड͚ͨ΋ͷͰ͢ɻ
    

    View Slide