Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Action Mailbox in Action

sylph01
October 03, 2020

Action Mailbox in Action

@ Kaigi on Rails, 2020/10/03

sylph01

October 03, 2020
Tweet

More Decks by sylph01

Other Decks in Programming

Transcript

  1. Action Mailbox in
    Action
    Ryo Kajiwara (sylph01)
    @ Kaigi on Rails, 2020/10/03

    View Slide

  2. ୭ʁ
    sylph01 / ֿݪ ཾ
    Twitter: @s01
    ҉߸ͱ͔Ͱ͖·͢
    Elixirͱ͔Ͱ͖·͢
    Rails·ΔͰΘ͔ΒΜ

    View Slide

  3. View Slide

  4. Rails(మಓ)
    ʹ͸Α͘৐Γ·͢

    View Slide

  5. ΋͏ͪΐͬͱਅ໘໨ʹ
    • ϓϩάϥϚɻ11݄͔Β໺ੜʹͳΓ·͢
    • "Dark Depths of SMTP" (2017) ͱ͍͏
    SMTPͷബ͍ຊΛॻ͖·ͨ͠
    • W3C, IETFͳͲͰηΩϡϦςΟدΓͷ
    ϓϩτίϧͷඪ४Խͷ͓ख఻͍Λͯ͠
    ͍·ͨ͠
    • HTTPS in Local Network, Messaging
    Layer SecurityͳͲ
    • 2020೥͔ΒISOC-JPͷofficerΛ΍ͬͯ
    ·͢

    View Slide

  6. εϥΠυ͸ͪ͜ΒͰݟΕ·͢
    https:/
    /speakerdeck.com/sylph01/action-
    mailbox-in-action

    View Slide

  7. View Slide

  8. TL;DR

    View Slide

  9. SMTP
    Λ΍ΊΖ

    View Slide

  10. ͪΌΜͱͨ͠಺༰঺հ
    ࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ
    ͜ͱͰϝʔϧϓϩτίϧͷجૅͱ
    Railsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜
    ͱΛ໨తͱ͠·͢

    View Slide

  11. ͪΌΜͱͨ͠಺༰঺հ
    • Action Mailboxͱ͸Կ͔
    • ϝʔϧʹؔ܎͢Δ֤छ֓೦
    • ಛʹɺSPF, DKIM, DMARC
    • (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙
    • αϯϓϧΞϓϦ
    • ϝʔϧΛ࢖ͬͨిࢠॻ੶ͷ"social DRM"

    View Slide

  12. લ൒͸աڈʹ͠Ό΂ͬ
    ͨεϥΠυͱࣅͯ·͢
    https://bit.ly/
    2S4SKEG

    View Slide

  13. View Slide

  14. Action Mailboxͱ͸Կ͔
    • Rails 6Ͱ௥Ճ͞Εͨ৽ػೳ
    • Action Mailer͸ϝʔϧΛ ૹ৴͢Δ ͨΊͷ΋ͷ͕ͩɺAction
    Mailbox͸RailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ
    • ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม׵
    • ҰఆظؒܦͬͨΒࣗಈম٫ʢ࡟আʣ

    View Slide

  15. View Slide

  16. Rails Guidesʹ͸MTAʮ΁ͷʯೖΓޱ
    Λඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ
    ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ

    View Slide

  17. ݸਓͰϝʔϧ΍Δͷ͸͓͢͢
    Ί͠·ͤΜ
    • ໎࿭ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬ख
    ʹಧ͖·ͤΜ
    • IMAPαʔόʔΛཱͯΔͱετϨʔδ஍ࠈʹؕΓ·͢
    • SMTPΛ΍ΊΖ

    View Slide

  18. Guidesʹॻ͍ͯ͋Δ
    αʔϏεΛ࢖͓͏
    Mailgun, Mandrill, Postmark,
    SendGrid, Amazon SES...

    View Slide

  19. View Slide

  20. ֓೦ͷ঺հ
    MTA: ڱٛͷʮϝʔϧαʔόʔʯͱ
    ͍ͬͨΒ͜Εͷ͜ͱ
    ͖ͬ͞ڍ͛ͨWebαʔϏε͸MTAͷ໾ׂΛ΍ͬͯ͘ΕΔ
    MUA: ϢʔβʔΤʔδΣϯτʢϝʔϧ
    ιϑτʣ

    View Slide

  21. View Slide

  22. ओʹMTA͕͠Ό΂Δͷ
    ͕SMTP
    ϝʔϧϘοΫεͱMUA͕͠Ό΂Δͷ
    ͕POP3΍IMAP

    View Slide

  23. SPF, DKIM
    ͜ͷϝʔϧ͸ͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔ
    όʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻ
    ͲͪΒ΋DNSͷTXTϨίʔυʹهड़Λߦ͏ɻ
    • SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ
    • DKIM: ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔ͸ൿີ伴Λར༻͠
    ͯϝοηʔδʹॺ໊͢Δɻ

    View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. DMARC
    • ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROM
    ίϚϯυͰ౉͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛ
    औΔ
    • Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ
    ໊ͷҰகΛऔΔ
    ͱ͍͏௥ՃͷೝূΛ͢Δɻࣦഊͨ͠৔߹ʹυϝΠϯΦʔφʔʹ໰
    ୊ͷ͋ΔϝʔϧΛใࠂͰ͖Δ࢓૊Έ΋͋Δɻ

    View Slide

  28. View Slide

  29. View Slide

  30. ࣮ࡍͷઃఆ
    $ dig TXT kaigionrails.s01.ninja @8.8.8.8
    $ dig TXT mail._domainkey.kaigionrails.s01.ninja
    @8.8.8.8
    $ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8
    ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ஋͕ݟΕ·͢ɻ

    View Slide

  31. ઃఆखॱ: ૹ৴ଆ
    ҎԼͷ৘ใΛυϝΠϯϨίʔυʹॻ͘ɻ
    • SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε
    • DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴
    • Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞ
    Λ࡞Δඞཁ͕͋Δ
    • DMARC: DMARCͷೝূʹࣦഊͨ͠৔߹ͷڍಈ
    ड৴ଆͷઃఆ͸ޙͰઆ໌͠·͢ɻ

    View Slide

  32. View Slide

  33. Action Mailbox in
    Action : The Hard Way
    ͋Δ͍͸ɺࣗ෼ͰMTA(Postfix)Λӡ༻
    ͯ͠Action Mailboxʹͭͳ͙࿩

    View Slide

  34. ͜͜·ͰRails Guidesͷ௨Γ
    • config.action_mailbox.ingress = :relay
    • rails credentials:edit ͯ͠ ingress_password Ληοτ
    • Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ
    ͕ΔΑ͏ʹͳΔϋζʂ
    • ͳΓ·ͤΜ

    View Slide

  35. bin/rails
    action_mailbox:ingress:postfix
    ͱ͸Կ͔
    • ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ
    • ༩͑ΒΕͨ INGRESS_PASSWORD Λ࢖ͬͯೝূ͠
    • Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ౉͢
    ΋ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔ
    ͷΛ଴ͪଓ͚·͢

    View Slide

  36. Postfixଆͷઃఆ
    هड़͢΂͖ઃఆϑΝΠϧͱͯ͠ओͳ΋ͷʹ main.cf, master.cf ͱ
    ͍͏΋ͷ͕͋Γ·͢ɻ
    • શମʹؔ͢Δઃఆ͸ main.cf ʹॻ͘
    • master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ͸ master.cf ʹॻ͘
    • ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ

    View Slide

  37. master.cf ͷهड़
    forward_to_rails unix - n n - - pipe
    flags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user}
    • forward_to_rails ͸໊শͳͷͰ೚ҙ
    • unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ
    • ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no
    • flags: X͸ίϚϯυ͕࠷ऴ഑৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱq͸Ξυ
    Ϩεͷѻ͍

    View Slide

  38. transport_maps,
    virtual_alias_maps
    /etc/postfix/transportʹ
    server_name forward_to_rails:
    Λهड़ɺ·ͨ /etc/postfix/virtual_alias_maps ʹ
    @server_name [email protected]_name
    Λهड़ɻ

    View Slide

  39. transport_maps,
    virtual_alias_maps
    main.cf ʹ
    transport_maps = hash:/etc/postfix/transport
    virtual_alias_maps = hash:/etc/postfix/virtual_aliases
    ͱهड़͠ɺҎԼΛ࣮ߦ
    $ sudo postmap /etc/postfix/transport
    $ sudo postmap /etc/postfix/virtual_aliases

    View Slide

  40. ͜Ε͸Կʁ
    • virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂ౰Λߦ͏
    • ͜͜Ͱ͸ɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰό
    ΢ϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠഑৴Λߦ͏
    Α͏ʹ͍ͯ͠Δ
    • transport_maps: ഑ૹܦ࿏ࢦఆΛߦ͏
    • ͜͜Ͱ͸ɺserver_name (ࣗ෼ࣗ਎) ʹདྷͨ΋ͷΛ͢΂ͯ
    master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢

    View Slide

  41. forward.shͷத਎
    #!/bin/sh
    HOME=/home/sylph01
    PATH=/usr/local/bin:$PATH
    cd /home/sylph01/keine &&
    bin/rails action_mailbox:ingress:postfix ...
    ͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻ
    ͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ

    View Slide

  42. ͜͜·Ͱ΍ͬͯϝʔϧ
    ͕ड͚औΕ·͢
    rails sͯ͠଴ͪड͚ͯΈ·͠ΐ͏

    View Slide

  43. ϩʔΧϧʹରͯ͠ϝʔϧͷς
    ετΛ͢Δ৔߹
    mail ίϚϯυΛ࢖͏ͷ͕Α͍Ͱ͢ɻ
    UbuntuͰ͸ mailutils ͷapt-get͕ඞཁɻ
    echo "hogefuga" | mail -s "Subject" -r from-
    [email protected] [email protected]
    EOF͕͸͖ͬΓ͢ΔͷͰecho͋Δ͍͸catͰຊจΛ౉͢͜ͱΛ͓͢
    ͢Ί͠·͢ɻ

    View Slide

  44. View Slide

  45. ActiveRecordͷΦϒ
    δΣΫτ͕ੜ੒͞Ε
    ͨʂ
    Α͘ݟΔͱActiveStorageʹ
    AttachmentΛ഑ஔͨ͠Γ΋͍ͯ͠Δ

    View Slide

  46. ࣮ࡍͲΜͳ΋ͷ͕Ͱ͖ͯΔ͔
    rails c ͯ͠
    m = ActionMailbox::InboundEmail.all
    ͱ͢Δͱड৴ͨ͠ϝʔϧ͢΂͕ͯऔΕ·͢ɻ
    ActionMailbox::InboundEmail ͕ ActiveRecord ͳΫϥεͳͷͰ
    ͍ͭ΋ͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻ
    த਎Λݟͯॲཧ͢ΔͨΊʹ͸ Mail::Message ΦϒδΣΫτ͕ཉ͠
    ͍ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ

    View Slide

  47. View Slide

  48. dev؀ڥͩͱConductor
    ͱ͍͏ศརͳUI͕Ϛ΢
    ϯτ͞Ε·͢
    /rails/conductor/
    action_mailbox/inbound_emails

    View Slide

  49. ϝʔϧ͔Β৘ใΛಘΔ
    • ૹ৴ݩΞυϨε: mail.from ʢ഑ྻͰ༩͑ΒΕΔ͜ͱʹ஫ҙʣ
    • ૹ৴ઌΞυϨε: mail.to ʢ͜Ε΋഑ྻʹͳΔʣ
    • ϔομ: mail.header['header_name'].value
    • Mail::Field ͕ฦͬͯ͘ΔͷͰ#value Ͱੜͷ஋ΛಘΔ
    • ໊݅: mail.subject
    • multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?

    View Slide

  50. ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτ
    ͷϝʔϧʹ͍ͭͯ
    ݱ୅Ͱ͸ISO-2022-JPΛݟ͔͚Δස౓͸ݮͬͯUTF-8ͰΤϯίʔυ
    ͞Εͨϝʔϧ͕େଟ਺ͱ͸ࢥ͍·͕͢…
    mail.decoded ͷ୅ΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCII
    ͱղऍ͞Ε)จࣈԽ͚͢Δݱ৅͕͋Γ·͢ɻ
    charset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱ৅ͩͱࢥ͏ͷͰMail gemʹҙਤత
    ͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ

    View Slide

  51. View Slide

  52. View Slide

  53. multipartͷϝʔϧͷ৔߹
    ʢཁ͢Δʹఴ෇ϑΝΠϧ͕͋Δ৔߹ʣ
    mail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ

    View Slide

  54. ड৴ଆͷSPF/DKIM/DMARC
    ຊ౰͸MTAͷଆͰϑΟϧλ͢Δ΂͖Ͱ͕͢ɺ΍Ζ͏ͱࢥ͑͹Rails
    ͷଆͰ΋ೝূ݁Ռͷ৘ใ͸ಘΒΕ·͢ʢMTAͷΈ͕஌ͬͯΔ৘ใ
    ͕ඞཁͳͷͰ൑ఆ͸Ͱ͖ͳ͍ʣɻ
    • SPFͷ৔߹ postfix-policyd-spf-python Λ௨͢
    • DKIMͷ৔߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ
    ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ෇༩͢Δ
    ͜ͱ͕Ͱ͖ɺͦͷத਎Λݟͯ൑ఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ

    View Slide

  55. View Slide

  56. ࡞ͬͨ΋ͷͷ঺հɺಈ
    ࡞ͷ঺հ
    GH: sylph01/keine
    ͚ʔͶͳͷ͸Dark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓ
    Ϧ͕mokou͔ͩΒͰ͢

    View Slide

  57. View Slide

  58. UUIDͷຒΊࠐΈ͸਺গ
    ͳ͍ͷͰखಈ
    ͿͬͪΌ͚γΣϧεΫϦϓτͱ͔Ͱࣗ
    ಈԽͯ͠΋͍͍

    View Slide

  59. View Slide

  60. ΊΜͲ͍ͷͰϑϩϯτ
    Τϯυ͢Β࡞ͬͯͳ͍

    View Slide

  61. application_mailbox.rb
    class ApplicationMailbox < ActionMailbox::Base
    routing /^(UUIDͷਖ਼نදݱ)@/i => :registration
    routing /^[email protected]/i => :greet
    end
    ଞʹ΋ɺ
    routing -> { (inbound_email) do_something } => :mailbox
    Έ͍ͨʹϝʔϧͷத਎Λ࢖ͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ

    View Slide

  62. greet_mailbox.rb
    class GreetMailbox < ApplicationMailbox
    def process
    GreetMailer.greet(inbound_email).deliver_now
    end
    end
    process ͷதʹॲཧΛॻ͘ɻ
    Action MailerͰฦ͚ͩ͢ɻ

    View Slide

  63. registration_mailbox.rb
    from_address = mail.from.first
    local_part_of_to = mail.to.first.split("@").first
    user = User.find_by(uuid: local_part_of_to)
    if user != nil
    if user.redeemed? == false
    user.email = from_address
    user.save
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    if user.email == from_address
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    bounce_with BounceMailer.already_redeemed(inbound_email)
    end
    end
    else
    bounce_with BounceMailer.bounce(inbound_email)
    end

    View Slide

  64. registration_mailbox.rb
    from_address = mail.from.first
    local_part_of_to = mail.to.first.split("@").first
    user = User.find_by(uuid: local_part_of_to)
    ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λ࢖ͬͯొ࿥͞Ε͍ͯΔ
    UserΛ୳͢ɻ

    View Slide

  65. registration_mailbox.rb
    (user == nilͷ৔߹)
    else
    bounce_with BounceMailer.bounce(inbound_email)
    end
    bounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏৘ใΛ࢒ͭͭ͠Ԡ౴͢Δ
    ͜ͱ͕Ͱ͖·͢ɻ

    View Slide

  66. registration_mailbox.rb
    if user.redeemed? == false
    user.email = from_address
    user.save
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    if user.email == from_address
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    bounce_with BounceMailer.already_redeemed(inbound_email)
    end
    end
    Ҿ͖׵͑ࡁΈ͔Ͳ͏͔ͷ൑ఆɺҾ͖׵͑ͨΒϝʔϧΞυϨεΛอ
    ଘɻ

    View Slide

  67. View Slide

  68. View Slide

  69. Έͳ͞Μʹ΋͓ࢼ͍͠
    ͚ͨͩ·͢
    ϋογϡλάʹ༗ޮͳUUIDΛ10ݸ͹
    Β·͖·͢ʢૣ͍΋ͷউͪʣ
    "Dark Depths of SMTP"͕खʹೖΓ·
    ͢ɻҾ͖׵͑ظݶ͸ࠓ೔·ͰͰ͢

    View Slide

  70. View Slide

  71. ·ͱΊ
    • ϝʔϧʹؔ͢Δ֤छ֓೦ͷ঺հΛ͠·ͨ͠
    • ͕࣌ؒ͋Ε͹΋ͬͱ͜ͷ࿩͕Ͱ͖Δ
    • PostfixͰAction Mailbox࢖͏ํ๏ͷ঺հΛ͠·ͨ͠
    • ૬౰ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ
    • Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷ঺հΛ͠
    ·ͨ͠

    View Slide

  72. Welcome to
    SMTPপ

    View Slide

  73. Special Thanks:
    ٕज़ग़൛αʔΫϧ Cryptic Command
    ͷϝϯόʔʹϝʔϧͷಋ௨ςετ΍จ
    ࣈԽ͚ؔ܎ͷτϥϒϧγϡʔτͰ͓
    ੈ࿩ʹͳΓ·ͨ͠

    View Slide

  74. Questions/Comments?
    plz send to @s01

    View Slide