Action Mailbox in Action

@ Kaigi on Rails, 2020/10/03

Transcript

1. Action Mailbox in
   Action
   Ryo Kajiwara (sylph01)
   @ Kaigi on Rails, 2020/10/03
   

   View Slide

2. ୭ʁ
   sylph01 / ֿݪ ཾ
   Twitter: @s01
   ҉߸ͱ͔Ͱ͖·͢
   Elixirͱ͔Ͱ͖·͢
   Rails·ΔͰΘ͔ΒΜ
   

   View Slide

3. View Slide

4. Rails(మಓ)
   ʹ͸Α͘৐Γ·͢
   

   View Slide

5. ΋͏ͪΐͬͱਅ໘໨ʹ
   • ϓϩάϥϚɻ11݄͔Β໺ੜʹͳΓ·͢
   • "Dark Depths of SMTP" (2017) ͱ͍͏
   SMTPͷബ͍ຊΛॻ͖·ͨ͠
   • W3C, IETFͳͲͰηΩϡϦςΟدΓͷ
   ϓϩτίϧͷඪ४Խͷ͓ख఻͍Λͯ͠
   ͍·ͨ͠
   • HTTPS in Local Network, Messaging
   Layer SecurityͳͲ
   • 2020೥͔ΒISOC-JPͷofficerΛ΍ͬͯ
   ·͢
   

   View Slide

6. εϥΠυ͸ͪ͜ΒͰݟΕ·͢
   https:/
   /speakerdeck.com/sylph01/action-
   mailbox-in-action
   

   View Slide

7. View Slide

8. TL;DR
   

   View Slide

9. SMTP
   Λ΍ΊΖ
   

   View Slide

10. ͪΌΜͱͨ͠಺༰঺հ
    ࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ
    ͜ͱͰϝʔϧϓϩτίϧͷجૅͱ
    Railsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜
    ͱΛ໨తͱ͠·͢
    

    View Slide

11. ͪΌΜͱͨ͠಺༰঺հ
    • Action Mailboxͱ͸Կ͔
    • ϝʔϧʹؔ܎͢Δ֤छ֓೦
    • ಛʹɺSPF, DKIM, DMARC
    • (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙
    • αϯϓϧΞϓϦ
    • ϝʔϧΛ࢖ͬͨిࢠॻ੶ͷ"social DRM"
    

    View Slide

12. લ൒͸աڈʹ͠Ό΂ͬ
    ͨεϥΠυͱࣅͯ·͢
    https://bit.ly/
    2S4SKEG
    

    View Slide

13. View Slide

14. Action Mailboxͱ͸Կ͔
    • Rails 6Ͱ௥Ճ͞Εͨ৽ػೳ
    • Action Mailer͸ϝʔϧΛ ૹ৴͢Δ ͨΊͷ΋ͷ͕ͩɺAction
    Mailbox͸RailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ
    • ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม׵
    • ҰఆظؒܦͬͨΒࣗಈম٫ʢ࡟আʣ
    

    View Slide

15. View Slide

16. Rails Guidesʹ͸MTAʮ΁ͷʯೖΓޱ
    Λඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ
    ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ
    

    View Slide

17. ݸਓͰϝʔϧ΍Δͷ͸͓͢͢
    Ί͠·ͤΜ
    • ໎࿭ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬ख
    ʹಧ͖·ͤΜ
    • IMAPαʔόʔΛཱͯΔͱετϨʔδ஍ࠈʹؕΓ·͢
    • SMTPΛ΍ΊΖ
    

    View Slide

18. Guidesʹॻ͍ͯ͋Δ
    αʔϏεΛ࢖͓͏
    Mailgun, Mandrill, Postmark,
    SendGrid, Amazon SES...
    

    View Slide

19. View Slide

20. ֓೦ͷ঺հ
    MTA: ڱٛͷʮϝʔϧαʔόʔʯͱ
    ͍ͬͨΒ͜Εͷ͜ͱ
    ͖ͬ͞ڍ͛ͨWebαʔϏε͸MTAͷ໾ׂΛ΍ͬͯ͘ΕΔ
    MUA: ϢʔβʔΤʔδΣϯτʢϝʔϧ
    ιϑτʣ
    

    View Slide

21. View Slide

22. ओʹMTA͕͠Ό΂Δͷ
    ͕SMTP
    ϝʔϧϘοΫεͱMUA͕͠Ό΂Δͷ
    ͕POP3΍IMAP
    

    View Slide

23. SPF, DKIM
    ͜ͷϝʔϧ͸ͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔ
    όʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻ
    ͲͪΒ΋DNSͷTXTϨίʔυʹهड़Λߦ͏ɻ
    • SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ
    • DKIM: ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔ͸ൿີ伴Λར༻͠
    ͯϝοηʔδʹॺ໊͢Δɻ
    

    View Slide

24. View Slide

25. View Slide

26. View Slide

27. DMARC
    • ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROM
    ίϚϯυͰ౉͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛ
    औΔ
    • Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ
    ໊ͷҰகΛऔΔ
    ͱ͍͏௥ՃͷೝূΛ͢Δɻࣦഊͨ͠৔߹ʹυϝΠϯΦʔφʔʹ໰
    ୊ͷ͋ΔϝʔϧΛใࠂͰ͖Δ࢓૊Έ΋͋Δɻ
    

    View Slide

28. View Slide

29. View Slide

30. ࣮ࡍͷઃఆ
    $ dig TXT kaigionrails.s01.ninja @8.8.8.8
    $ dig TXT mail._domainkey.kaigionrails.s01.ninja
    @8.8.8.8
    $ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8
    ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ஋͕ݟΕ·͢ɻ
    

    View Slide

31. ઃఆखॱ: ૹ৴ଆ
    ҎԼͷ৘ใΛυϝΠϯϨίʔυʹॻ͘ɻ
    • SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε
    • DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴
    • Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞ
    Λ࡞Δඞཁ͕͋Δ
    • DMARC: DMARCͷೝূʹࣦഊͨ͠৔߹ͷڍಈ
    ड৴ଆͷઃఆ͸ޙͰઆ໌͠·͢ɻ
    

    View Slide

32. View Slide

33. Action Mailbox in
    Action : The Hard Way
    ͋Δ͍͸ɺࣗ෼ͰMTA(Postfix)Λӡ༻
    ͯ͠Action Mailboxʹͭͳ͙࿩
    

    View Slide

34. ͜͜·ͰRails Guidesͷ௨Γ
    • config.action_mailbox.ingress = :relay
    • rails credentials:edit ͯ͠ ingress_password Ληοτ
    • Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ
    ͕ΔΑ͏ʹͳΔϋζʂ
    • ͳΓ·ͤΜ
    

    View Slide

35. bin/rails
    action_mailbox:ingress:postfix
    ͱ͸Կ͔
    • ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ
    • ༩͑ΒΕͨ INGRESS_PASSWORD Λ࢖ͬͯೝূ͠
    • Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ౉͢
    ΋ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔ
    ͷΛ଴ͪଓ͚·͢
    

    View Slide

36. Postfixଆͷઃఆ
    هड़͢΂͖ઃఆϑΝΠϧͱͯ͠ओͳ΋ͷʹ main.cf, master.cf ͱ
    ͍͏΋ͷ͕͋Γ·͢ɻ
    • શମʹؔ͢Δઃఆ͸ main.cf ʹॻ͘
    • master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ͸ master.cf ʹॻ͘
    • ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ
    

    View Slide

37. master.cf ͷهड़
    forward_to_rails unix - n n - - pipe
    flags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user}
    • forward_to_rails ͸໊শͳͷͰ೚ҙ
    • unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ
    • ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no
    • flags: X͸ίϚϯυ͕࠷ऴ഑৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱq͸Ξυ
    Ϩεͷѻ͍
    

    View Slide

38. transport_maps,
    virtual_alias_maps
    /etc/postfix/transportʹ
    server_name forward_to_rails:
    Λهड़ɺ·ͨ /etc/postfix/virtual_alias_maps ʹ
    @server_name existing_user_name@server_name
    Λهड़ɻ
    

    View Slide

39. transport_maps,
    virtual_alias_maps
    main.cf ʹ
    transport_maps = hash:/etc/postfix/transport
    virtual_alias_maps = hash:/etc/postfix/virtual_aliases
    ͱهड़͠ɺҎԼΛ࣮ߦ
    $ sudo postmap /etc/postfix/transport
    $ sudo postmap /etc/postfix/virtual_aliases
    

    View Slide

40. ͜Ε͸Կʁ
    • virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂ౰Λߦ͏
    • ͜͜Ͱ͸ɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰό
    ΢ϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠഑৴Λߦ͏
    Α͏ʹ͍ͯ͠Δ
    • transport_maps: ഑ૹܦ࿏ࢦఆΛߦ͏
    • ͜͜Ͱ͸ɺserver_name (ࣗ෼ࣗ਎) ʹདྷͨ΋ͷΛ͢΂ͯ
    master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢
    

    View Slide

41. forward.shͷத਎
    #!/bin/sh
    HOME=/home/sylph01
    PATH=/usr/local/bin:$PATH
    cd /home/sylph01/keine &&
    bin/rails action_mailbox:ingress:postfix ...
    ͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻ
    ͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ
    

    View Slide

42. ͜͜·Ͱ΍ͬͯϝʔϧ
    ͕ड͚औΕ·͢
    rails sͯ͠଴ͪड͚ͯΈ·͠ΐ͏
    

    View Slide

43. ϩʔΧϧʹରͯ͠ϝʔϧͷς
    ετΛ͢Δ৔߹
    mail ίϚϯυΛ࢖͏ͷ͕Α͍Ͱ͢ɻ
    UbuntuͰ͸ mailutils ͷapt-get͕ඞཁɻ
    echo "hogefuga" | mail -s "Subject" -r from-
    address@yourdomain destination-address@server
    EOF͕͸͖ͬΓ͢ΔͷͰecho͋Δ͍͸catͰຊจΛ౉͢͜ͱΛ͓͢
    ͢Ί͠·͢ɻ
    

    View Slide

44. View Slide

45. ActiveRecordͷΦϒ
    δΣΫτ͕ੜ੒͞Ε
    ͨʂ
    Α͘ݟΔͱActiveStorageʹ
    AttachmentΛ഑ஔͨ͠Γ΋͍ͯ͠Δ
    

    View Slide

46. ࣮ࡍͲΜͳ΋ͷ͕Ͱ͖ͯΔ͔
    rails c ͯ͠
    m = ActionMailbox::InboundEmail.all
    ͱ͢Δͱड৴ͨ͠ϝʔϧ͢΂͕ͯऔΕ·͢ɻ
    ActionMailbox::InboundEmail ͕ ActiveRecord ͳΫϥεͳͷͰ
    ͍ͭ΋ͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻ
    த਎Λݟͯॲཧ͢ΔͨΊʹ͸ Mail::Message ΦϒδΣΫτ͕ཉ͠
    ͍ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ
    

    View Slide

47. View Slide

48. dev؀ڥͩͱConductor
    ͱ͍͏ศརͳUI͕Ϛ΢
    ϯτ͞Ε·͢
    /rails/conductor/
    action_mailbox/inbound_emails
    

    View Slide

49. ϝʔϧ͔Β৘ใΛಘΔ
    • ૹ৴ݩΞυϨε: mail.from ʢ഑ྻͰ༩͑ΒΕΔ͜ͱʹ஫ҙʣ
    • ૹ৴ઌΞυϨε: mail.to ʢ͜Ε΋഑ྻʹͳΔʣ
    • ϔομ: mail.header['header_name'].value
    • Mail::Field ͕ฦͬͯ͘ΔͷͰ#value Ͱੜͷ஋ΛಘΔ
    • ໊݅: mail.subject
    • multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?
    

    View Slide

50. ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτ
    ͷϝʔϧʹ͍ͭͯ
    ݱ୅Ͱ͸ISO-2022-JPΛݟ͔͚Δස౓͸ݮͬͯUTF-8ͰΤϯίʔυ
    ͞Εͨϝʔϧ͕େଟ਺ͱ͸ࢥ͍·͕͢…
    mail.decoded ͷ୅ΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCII
    ͱղऍ͞Ε)จࣈԽ͚͢Δݱ৅͕͋Γ·͢ɻ
    charset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱ৅ͩͱࢥ͏ͷͰMail gemʹҙਤత
    ͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ
    

    View Slide

51. View Slide

52. View Slide

53. multipartͷϝʔϧͷ৔߹
    ʢཁ͢Δʹఴ෇ϑΝΠϧ͕͋Δ৔߹ʣ
    mail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ
    

    View Slide

54. ड৴ଆͷSPF/DKIM/DMARC
    ຊ౰͸MTAͷଆͰϑΟϧλ͢Δ΂͖Ͱ͕͢ɺ΍Ζ͏ͱࢥ͑͹Rails
    ͷଆͰ΋ೝূ݁Ռͷ৘ใ͸ಘΒΕ·͢ʢMTAͷΈ͕஌ͬͯΔ৘ใ
    ͕ඞཁͳͷͰ൑ఆ͸Ͱ͖ͳ͍ʣɻ
    • SPFͷ৔߹ postfix-policyd-spf-python Λ௨͢
    • DKIMͷ৔߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ
    ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ෇༩͢Δ
    ͜ͱ͕Ͱ͖ɺͦͷத਎Λݟͯ൑ఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ
    

    View Slide

55. View Slide

56. ࡞ͬͨ΋ͷͷ঺հɺಈ
    ࡞ͷ঺հ
    GH: sylph01/keine
    ͚ʔͶͳͷ͸Dark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓ
    Ϧ͕mokou͔ͩΒͰ͢
    

    View Slide

57. View Slide

58. UUIDͷຒΊࠐΈ͸਺গ
    ͳ͍ͷͰखಈ
    ͿͬͪΌ͚γΣϧεΫϦϓτͱ͔Ͱࣗ
    ಈԽͯ͠΋͍͍
    

    View Slide

59. View Slide

60. ΊΜͲ͍ͷͰϑϩϯτ
    Τϯυ͢Β࡞ͬͯͳ͍
    

    View Slide

61. application_mailbox.rb
    class ApplicationMailbox < ActionMailbox::Base
    routing /^(UUIDͷਖ਼نදݱ)@/i => :registration
    routing /^sylph01@/i => :greet
    end
    ଞʹ΋ɺ
    routing -> { (inbound_email) do_something } => :mailbox
    Έ͍ͨʹϝʔϧͷத਎Λ࢖ͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ
    

    View Slide

62. greet_mailbox.rb
    class GreetMailbox < ApplicationMailbox
    def process
    GreetMailer.greet(inbound_email).deliver_now
    end
    end
    process ͷதʹॲཧΛॻ͘ɻ
    Action MailerͰฦ͚ͩ͢ɻ
    

    View Slide

63. registration_mailbox.rb
    from_address = mail.from.first
    local_part_of_to = mail.to.first.split("@").first
    user = User.find_by(uuid: local_part_of_to)
    if user != nil
    if user.redeemed? == false
    user.email = from_address
    user.save
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    if user.email == from_address
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    bounce_with BounceMailer.already_redeemed(inbound_email)
    end
    end
    else
    bounce_with BounceMailer.bounce(inbound_email)
    end
    

    View Slide

64. registration_mailbox.rb
    from_address = mail.from.first
    local_part_of_to = mail.to.first.split("@").first
    user = User.find_by(uuid: local_part_of_to)
    ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λ࢖ͬͯొ࿥͞Ε͍ͯΔ
    UserΛ୳͢ɻ
    

    View Slide

65. registration_mailbox.rb
    (user == nilͷ৔߹)
    else
    bounce_with BounceMailer.bounce(inbound_email)
    end
    bounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏৘ใΛ࢒ͭͭ͠Ԡ౴͢Δ
    ͜ͱ͕Ͱ͖·͢ɻ
    

    View Slide

66. registration_mailbox.rb
    if user.redeemed? == false
    user.email = from_address
    user.save
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    if user.email == from_address
    IssuerMailer.with(user: user).issue(inbound_email).deliver_now
    else
    bounce_with BounceMailer.already_redeemed(inbound_email)
    end
    end
    Ҿ͖׵͑ࡁΈ͔Ͳ͏͔ͷ൑ఆɺҾ͖׵͑ͨΒϝʔϧΞυϨεΛอ
    ଘɻ
    

    View Slide

67. View Slide

68. View Slide

69. Έͳ͞Μʹ΋͓ࢼ͍͠
    ͚ͨͩ·͢
    ϋογϡλάʹ༗ޮͳUUIDΛ10ݸ͹
    Β·͖·͢ʢૣ͍΋ͷউͪʣ
    "Dark Depths of SMTP"͕खʹೖΓ·
    ͢ɻҾ͖׵͑ظݶ͸ࠓ೔·ͰͰ͢
    

    View Slide

70. View Slide

71. ·ͱΊ
    • ϝʔϧʹؔ͢Δ֤छ֓೦ͷ঺հΛ͠·ͨ͠
    • ͕࣌ؒ͋Ε͹΋ͬͱ͜ͷ࿩͕Ͱ͖Δ
    • PostfixͰAction Mailbox࢖͏ํ๏ͷ঺հΛ͠·ͨ͠
    • ૬౰ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ
    • Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷ঺հΛ͠
    ·ͨ͠
    

    View Slide

72. Welcome to
    SMTPপ
    

    View Slide

73. Special Thanks:
    ٕज़ग़൛αʔΫϧ Cryptic Command
    ͷϝϯόʔʹϝʔϧͷಋ௨ςετ΍จ
    ࣈԽ͚ؔ܎ͷτϥϒϧγϡʔτͰ͓
    ੈ࿩ʹͳΓ·ͨ͠
    

    View Slide

74. Questions/Comments?
    plz send to @s01
    

    View Slide