Action Mailbox in Action

404139d782ec666acea93dffc86e089f?s=47 sylph01
October 03, 2020

Action Mailbox in Action

@ Kaigi on Rails, 2020/10/03

404139d782ec666acea93dffc86e089f?s=128

sylph01

October 03, 2020
Tweet

Transcript

  1. Action Mailbox in Action Ryo Kajiwara (sylph01) @ Kaigi on

    Rails, 2020/10/03
  2. ୭ʁ sylph01 / ֿݪ ཾ Twitter: @s01 ҉߸ͱ͔Ͱ͖·͢ Elixirͱ͔Ͱ͖·͢ Rails·ΔͰΘ͔ΒΜ

  3. None
  4. Rails(మಓ) ʹ͸Α͘৐Γ·͢

  5. ΋͏ͪΐͬͱਅ໘໨ʹ • ϓϩάϥϚɻ11݄͔Β໺ੜʹͳΓ·͢ • "Dark Depths of SMTP" (2017) ͱ͍͏

    SMTPͷബ͍ຊΛॻ͖·ͨ͠ • W3C, IETFͳͲͰηΩϡϦςΟدΓͷ ϓϩτίϧͷඪ४Խͷ͓ख఻͍Λͯ͠ ͍·ͨ͠ • HTTPS in Local Network, Messaging Layer SecurityͳͲ • 2020೥͔ΒISOC-JPͷofficerΛ΍ͬͯ ·͢
  6. εϥΠυ͸ͪ͜ΒͰݟΕ·͢ https:/ /speakerdeck.com/sylph01/action- mailbox-in-action

  7. None
  8. TL;DR

  9. SMTP Λ΍ΊΖ

  10. ͪΌΜͱͨ͠಺༰঺հ ࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ ͜ͱͰϝʔϧϓϩτίϧͷجૅͱ Railsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜ ͱΛ໨తͱ͠·͢

  11. ͪΌΜͱͨ͠಺༰঺հ • Action Mailboxͱ͸Կ͔ • ϝʔϧʹؔ܎͢Δ֤छ֓೦ • ಛʹɺSPF, DKIM, DMARC

    • (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙ • αϯϓϧΞϓϦ • ϝʔϧΛ࢖ͬͨిࢠॻ੶ͷ"social DRM"
  12. લ൒͸աڈʹ͠Ό΂ͬ ͨεϥΠυͱࣅͯ·͢ https://bit.ly/ 2S4SKEG

  13. None
  14. Action Mailboxͱ͸Կ͔ • Rails 6Ͱ௥Ճ͞Εͨ৽ػೳ • Action Mailer͸ϝʔϧΛ ૹ৴͢Δ ͨΊͷ΋ͷ͕ͩɺAction

    Mailbox͸RailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ • ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม׵ • ҰఆظؒܦͬͨΒࣗಈম٫ʢ࡟আʣ
  15. None
  16. Rails Guidesʹ͸MTAʮ΁ͷʯೖΓޱ Λඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ

  17. ݸਓͰϝʔϧ΍Δͷ͸͓͢͢ Ί͠·ͤΜ • ໎࿭ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬ख ʹಧ͖·ͤΜ • IMAPαʔόʔΛཱͯΔͱετϨʔδ஍ࠈʹؕΓ·͢ • SMTPΛ΍ΊΖ

  18. Guidesʹॻ͍ͯ͋Δ αʔϏεΛ࢖͓͏ Mailgun, Mandrill, Postmark, SendGrid, Amazon SES...

  19. None
  20. ֓೦ͷ঺հ MTA: ڱٛͷʮϝʔϧαʔόʔʯͱ ͍ͬͨΒ͜Εͷ͜ͱ ͖ͬ͞ڍ͛ͨWebαʔϏε͸MTAͷ໾ׂΛ΍ͬͯ͘ΕΔ MUA: ϢʔβʔΤʔδΣϯτʢϝʔϧ ιϑτʣ

  21. None
  22. ओʹMTA͕͠Ό΂Δͷ ͕SMTP ϝʔϧϘοΫεͱMUA͕͠Ό΂Δͷ ͕POP3΍IMAP

  23. SPF, DKIM ͜ͷϝʔϧ͸ͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔ όʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻ ͲͪΒ΋DNSͷTXTϨίʔυʹهड़Λߦ͏ɻ • SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ • DKIM:

    ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔ͸ൿີ伴Λར༻͠ ͯϝοηʔδʹॺ໊͢Δɻ
  24. None
  25. None
  26. None
  27. DMARC • ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROM ίϚϯυͰ౉͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛ औΔ • Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ ໊ͷҰகΛऔΔ ͱ͍͏௥ՃͷೝূΛ͢Δɻࣦഊͨ͠৔߹ʹυϝΠϯΦʔφʔʹ໰

    ୊ͷ͋ΔϝʔϧΛใࠂͰ͖Δ࢓૊Έ΋͋Δɻ
  28. None
  29. None
  30. ࣮ࡍͷઃఆ $ dig TXT kaigionrails.s01.ninja @8.8.8.8 $ dig TXT mail._domainkey.kaigionrails.s01.ninja

    @8.8.8.8 $ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8 ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ஋͕ݟΕ·͢ɻ
  31. ઃఆखॱ: ૹ৴ଆ ҎԼͷ৘ใΛυϝΠϯϨίʔυʹॻ͘ɻ • SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε • DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴

    • Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞ Λ࡞Δඞཁ͕͋Δ • DMARC: DMARCͷೝূʹࣦഊͨ͠৔߹ͷڍಈ ड৴ଆͷઃఆ͸ޙͰઆ໌͠·͢ɻ
  32. None
  33. Action Mailbox in Action : The Hard Way ͋Δ͍͸ɺࣗ෼ͰMTA(Postfix)Λӡ༻ ͯ͠Action

    Mailboxʹͭͳ͙࿩
  34. ͜͜·ͰRails Guidesͷ௨Γ • config.action_mailbox.ingress = :relay • rails credentials:edit ͯ͠

    ingress_password Ληοτ • Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ ͕ΔΑ͏ʹͳΔϋζʂ • ͳΓ·ͤΜ
  35. bin/rails action_mailbox:ingress:postfix ͱ͸Կ͔ • ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ • ༩͑ΒΕͨ INGRESS_PASSWORD Λ࢖ͬͯೝূ͠ •

    Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ౉͢ ΋ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔ ͷΛ଴ͪଓ͚·͢
  36. Postfixଆͷઃఆ هड़͢΂͖ઃఆϑΝΠϧͱͯ͠ओͳ΋ͷʹ main.cf, master.cf ͱ ͍͏΋ͷ͕͋Γ·͢ɻ • શମʹؔ͢Δઃఆ͸ main.cf ʹॻ͘

    • master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ͸ master.cf ʹॻ͘ • ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ
  37. master.cf ͷهड़ forward_to_rails unix - n n - - pipe

    flags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user} • forward_to_rails ͸໊শͳͷͰ೚ҙ • unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ • ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no • flags: X͸ίϚϯυ͕࠷ऴ഑৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱq͸Ξυ Ϩεͷѻ͍
  38. transport_maps, virtual_alias_maps /etc/postfix/transportʹ server_name forward_to_rails: Λهड़ɺ·ͨ /etc/postfix/virtual_alias_maps ʹ @server_name existing_user_name@server_name

    Λهड़ɻ
  39. transport_maps, virtual_alias_maps main.cf ʹ transport_maps = hash:/etc/postfix/transport virtual_alias_maps = hash:/etc/postfix/virtual_aliases

    ͱهड़͠ɺҎԼΛ࣮ߦ $ sudo postmap /etc/postfix/transport $ sudo postmap /etc/postfix/virtual_aliases
  40. ͜Ε͸Կʁ • virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂ౰Λߦ͏ • ͜͜Ͱ͸ɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰό ΢ϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠഑৴Λߦ͏ Α͏ʹ͍ͯ͠Δ •

    transport_maps: ഑ૹܦ࿏ࢦఆΛߦ͏ • ͜͜Ͱ͸ɺserver_name (ࣗ෼ࣗ਎) ʹདྷͨ΋ͷΛ͢΂ͯ master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢
  41. forward.shͷத਎ #!/bin/sh HOME=/home/sylph01 PATH=/usr/local/bin:$PATH cd /home/sylph01/keine && bin/rails action_mailbox:ingress:postfix ...

    ͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻ ͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ
  42. ͜͜·Ͱ΍ͬͯϝʔϧ ͕ड͚औΕ·͢ rails sͯ͠଴ͪड͚ͯΈ·͠ΐ͏

  43. ϩʔΧϧʹରͯ͠ϝʔϧͷς ετΛ͢Δ৔߹ mail ίϚϯυΛ࢖͏ͷ͕Α͍Ͱ͢ɻ UbuntuͰ͸ mailutils ͷapt-get͕ඞཁɻ echo "hogefuga" |

    mail -s "Subject" -r from- address@yourdomain destination-address@server EOF͕͸͖ͬΓ͢ΔͷͰecho͋Δ͍͸catͰຊจΛ౉͢͜ͱΛ͓͢ ͢Ί͠·͢ɻ
  44. None
  45. ActiveRecordͷΦϒ δΣΫτ͕ੜ੒͞Ε ͨʂ Α͘ݟΔͱActiveStorageʹ AttachmentΛ഑ஔͨ͠Γ΋͍ͯ͠Δ

  46. ࣮ࡍͲΜͳ΋ͷ͕Ͱ͖ͯΔ͔ rails c ͯ͠ m = ActionMailbox::InboundEmail.all ͱ͢Δͱड৴ͨ͠ϝʔϧ͢΂͕ͯऔΕ·͢ɻ ActionMailbox::InboundEmail ͕

    ActiveRecord ͳΫϥεͳͷͰ ͍ͭ΋ͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻ த਎Λݟͯॲཧ͢ΔͨΊʹ͸ Mail::Message ΦϒδΣΫτ͕ཉ͠ ͍ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ
  47. None
  48. dev؀ڥͩͱConductor ͱ͍͏ศརͳUI͕Ϛ΢ ϯτ͞Ε·͢ /rails/conductor/ action_mailbox/inbound_emails

  49. ϝʔϧ͔Β৘ใΛಘΔ • ૹ৴ݩΞυϨε: mail.from ʢ഑ྻͰ༩͑ΒΕΔ͜ͱʹ஫ҙʣ • ૹ৴ઌΞυϨε: mail.to ʢ͜Ε΋഑ྻʹͳΔʣ •

    ϔομ: mail.header['header_name'].value • Mail::Field ͕ฦͬͯ͘ΔͷͰ#value Ͱੜͷ஋ΛಘΔ • ໊݅: mail.subject • multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?
  50. ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτ ͷϝʔϧʹ͍ͭͯ ݱ୅Ͱ͸ISO-2022-JPΛݟ͔͚Δස౓͸ݮͬͯUTF-8ͰΤϯίʔυ ͞Εͨϝʔϧ͕େଟ਺ͱ͸ࢥ͍·͕͢… mail.decoded ͷ୅ΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCII ͱղऍ͞Ε)จࣈԽ͚͢Δݱ৅͕͋Γ·͢ɻ charset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱ৅ͩͱࢥ͏ͷͰMail

    gemʹҙਤత ͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ
  51. None
  52. None
  53. multipartͷϝʔϧͷ৔߹ ʢཁ͢Δʹఴ෇ϑΝΠϧ͕͋Δ৔߹ʣ mail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ

  54. ड৴ଆͷSPF/DKIM/DMARC ຊ౰͸MTAͷଆͰϑΟϧλ͢Δ΂͖Ͱ͕͢ɺ΍Ζ͏ͱࢥ͑͹Rails ͷଆͰ΋ೝূ݁Ռͷ৘ใ͸ಘΒΕ·͢ʢMTAͷΈ͕஌ͬͯΔ৘ใ ͕ඞཁͳͷͰ൑ఆ͸Ͱ͖ͳ͍ʣɻ • SPFͷ৔߹ postfix-policyd-spf-python Λ௨͢ • DKIMͷ৔߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ

    ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ෇༩͢Δ ͜ͱ͕Ͱ͖ɺͦͷத਎Λݟͯ൑ఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ
  55. None
  56. ࡞ͬͨ΋ͷͷ঺հɺಈ ࡞ͷ঺հ GH: sylph01/keine ͚ʔͶͳͷ͸Dark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓ Ϧ͕mokou͔ͩΒͰ͢

  57. None
  58. UUIDͷຒΊࠐΈ͸਺গ ͳ͍ͷͰखಈ ͿͬͪΌ͚γΣϧεΫϦϓτͱ͔Ͱࣗ ಈԽͯ͠΋͍͍

  59. None
  60. ΊΜͲ͍ͷͰϑϩϯτ Τϯυ͢Β࡞ͬͯͳ͍

  61. application_mailbox.rb class ApplicationMailbox < ActionMailbox::Base routing /^(UUIDͷਖ਼نදݱ)@/i => :registration routing

    /^sylph01@/i => :greet end ଞʹ΋ɺ routing -> { (inbound_email) do_something } => :mailbox Έ͍ͨʹϝʔϧͷத਎Λ࢖ͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ
  62. greet_mailbox.rb class GreetMailbox < ApplicationMailbox def process GreetMailer.greet(inbound_email).deliver_now end end

    process ͷதʹॲཧΛॻ͘ɻ Action MailerͰฦ͚ͩ͢ɻ
  63. registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid:

    local_part_of_to) if user != nil if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user: user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end else bounce_with BounceMailer.bounce(inbound_email) end
  64. registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid:

    local_part_of_to) ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λ࢖ͬͯొ࿥͞Ε͍ͯΔ UserΛ୳͢ɻ
  65. registration_mailbox.rb (user == nilͷ৔߹) else bounce_with BounceMailer.bounce(inbound_email) end bounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏৘ใΛ࢒ͭͭ͠Ԡ౴͢Δ

    ͜ͱ͕Ͱ͖·͢ɻ
  66. registration_mailbox.rb if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user:

    user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end Ҿ͖׵͑ࡁΈ͔Ͳ͏͔ͷ൑ఆɺҾ͖׵͑ͨΒϝʔϧΞυϨεΛอ ଘɻ
  67. None
  68. None
  69. Έͳ͞Μʹ΋͓ࢼ͍͠ ͚ͨͩ·͢ ϋογϡλάʹ༗ޮͳUUIDΛ10ݸ͹ Β·͖·͢ʢૣ͍΋ͷউͪʣ "Dark Depths of SMTP"͕खʹೖΓ· ͢ɻҾ͖׵͑ظݶ͸ࠓ೔·ͰͰ͢

  70. None
  71. ·ͱΊ • ϝʔϧʹؔ͢Δ֤छ֓೦ͷ঺հΛ͠·ͨ͠ • ͕࣌ؒ͋Ε͹΋ͬͱ͜ͷ࿩͕Ͱ͖Δ • PostfixͰAction Mailbox࢖͏ํ๏ͷ঺հΛ͠·ͨ͠ • ૬౰ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ

    • Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷ঺հΛ͠ ·ͨ͠
  72. Welcome to SMTPপ

  73. Special Thanks: ٕज़ग़൛αʔΫϧ Cryptic Command ͷϝϯόʔʹϝʔϧͷಋ௨ςετ΍จ ࣈԽ͚ؔ܎ͷτϥϒϧγϡʔτͰ͓ ੈ࿩ʹͳΓ·ͨ͠

  74. Questions/Comments? plz send to @s01