Action Mailbox in Action

@ Kaigi on Rails, 2020/10/03

Transcript

1. Action Mailbox in Action Ryo Kajiwara (sylph01) @ Kaigi on Rails, 2020/10/03
2. ୭ʁ sylph01 / ֿݪ ཾ Twitter: @s01 ҉߸ͱ͔Ͱ͖·͢ Elixirͱ͔Ͱ͖·͢ Rails·ΔͰΘ͔ΒΜ
4. Rails(మಓ) ʹ͸Α͘৐Γ·͢
5. ΋͏ͪΐͬͱਅ໘໨ʹ • ϓϩάϥϚɻ11݄͔Β໺ੜʹͳΓ·͢ • "Dark Depths of SMTP" (2017) ͱ͍͏ SMTPͷബ͍ຊΛॻ͖·ͨ͠ • W3C, IETFͳͲͰηΩϡϦςΟدΓͷ ϓϩτίϧͷඪ४Խͷ͓ख఻͍Λͯ͠ ͍·ͨ͠ • HTTPS in Local Network, Messaging Layer SecurityͳͲ • 2020೥͔ΒISOC-JPͷofficerΛ΍ͬͯ ·͢
6. εϥΠυ͸ͪ͜ΒͰݟΕ·͢ https:/ /speakerdeck.com/sylph01/action- mailbox-in-action
8. TL;DR
9. SMTP Λ΍ΊΖ
10. ͪΌΜͱͨ͠಺༰঺հ ࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ ͜ͱͰϝʔϧϓϩτίϧͷجૅͱ Railsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜ ͱΛ໨తͱ͠·͢
11. ͪΌΜͱͨ͠಺༰঺հ • Action Mailboxͱ͸Կ͔ • ϝʔϧʹؔ܎͢Δ֤छ֓೦ • ಛʹɺSPF, DKIM, DMARC • (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙ • αϯϓϧΞϓϦ • ϝʔϧΛ࢖ͬͨిࢠॻ੶ͷ"social DRM"
12. લ൒͸աڈʹ͠Ό΂ͬ ͨεϥΠυͱࣅͯ·͢ https://bit.ly/ 2S4SKEG
14. Action Mailboxͱ͸Կ͔ • Rails 6Ͱ௥Ճ͞Εͨ৽ػೳ • Action Mailer͸ϝʔϧΛ ૹ৴͢Δ ͨΊͷ΋ͷ͕ͩɺAction Mailbox͸RailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ • ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม׵ • ҰఆظؒܦͬͨΒࣗಈম٫ʢ࡟আʣ
16. Rails Guidesʹ͸MTAʮ΁ͷʯೖΓޱ Λඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ
17. ݸਓͰϝʔϧ΍Δͷ͸͓͢͢ Ί͠·ͤΜ • ໎࿭ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬ख ʹಧ͖·ͤΜ • IMAPαʔόʔΛཱͯΔͱετϨʔδ஍ࠈʹؕΓ·͢ • SMTPΛ΍ΊΖ
18. Guidesʹॻ͍ͯ͋Δ αʔϏεΛ࢖͓͏ Mailgun, Mandrill, Postmark, SendGrid, Amazon SES...
20. ֓೦ͷ঺հ MTA: ڱٛͷʮϝʔϧαʔόʔʯͱ ͍ͬͨΒ͜Εͷ͜ͱ ͖ͬ͞ڍ͛ͨWebαʔϏε͸MTAͷ໾ׂΛ΍ͬͯ͘ΕΔ MUA: ϢʔβʔΤʔδΣϯτʢϝʔϧ ιϑτʣ
22. ओʹMTA͕͠Ό΂Δͷ ͕SMTP ϝʔϧϘοΫεͱMUA͕͠Ό΂Δͷ ͕POP3΍IMAP
23. SPF, DKIM ͜ͷϝʔϧ͸ͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔ όʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻ ͲͪΒ΋DNSͷTXTϨίʔυʹهड़Λߦ͏ɻ • SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ • DKIM: ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔ͸ൿີ伴Λར༻͠ ͯϝοηʔδʹॺ໊͢Δɻ
27. DMARC • ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROM ίϚϯυͰ౉͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛ औΔ • Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ ໊ͷҰகΛऔΔ ͱ͍͏௥ՃͷೝূΛ͢Δɻࣦഊͨ͠৔߹ʹυϝΠϯΦʔφʔʹ໰ ୊ͷ͋ΔϝʔϧΛใࠂͰ͖Δ࢓૊Έ΋͋Δɻ
30. ࣮ࡍͷઃఆ $ dig TXT kaigionrails.s01.ninja @8.8.8.8 $ dig TXT mail._domainkey.kaigionrails.s01.ninja @8.8.8.8 $ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8 ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ஋͕ݟΕ·͢ɻ
31. ઃఆखॱ: ૹ৴ଆ ҎԼͷ৘ใΛυϝΠϯϨίʔυʹॻ͘ɻ • SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε • DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴 • Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞ Λ࡞Δඞཁ͕͋Δ • DMARC: DMARCͷೝূʹࣦഊͨ͠৔߹ͷڍಈ ड৴ଆͷઃఆ͸ޙͰઆ໌͠·͢ɻ
33. Action Mailbox in Action : The Hard Way ͋Δ͍͸ɺࣗ෼ͰMTA(Postfix)Λӡ༻ ͯ͠Action Mailboxʹͭͳ͙࿩
34. ͜͜·ͰRails Guidesͷ௨Γ • config.action_mailbox.ingress = :relay • rails credentials:edit ͯ͠ ingress_password Ληοτ • Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ ͕ΔΑ͏ʹͳΔϋζʂ • ͳΓ·ͤΜ
35. bin/rails action_mailbox:ingress:postfix ͱ͸Կ͔ • ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ • ༩͑ΒΕͨ INGRESS_PASSWORD Λ࢖ͬͯೝূ͠ • Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ౉͢ ΋ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔ ͷΛ଴ͪଓ͚·͢
36. Postfixଆͷઃఆ هड़͢΂͖ઃఆϑΝΠϧͱͯ͠ओͳ΋ͷʹ main.cf, master.cf ͱ ͍͏΋ͷ͕͋Γ·͢ɻ • શମʹؔ͢Δઃఆ͸ main.cf ʹॻ͘ • master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ͸ master.cf ʹॻ͘ • ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ
37. master.cf ͷهड़ forward_to_rails unix - n n - - pipe flags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user} • forward_to_rails ͸໊শͳͷͰ೚ҙ • unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ • ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no • flags: X͸ίϚϯυ͕࠷ऴ഑৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱq͸Ξυ Ϩεͷѻ͍
38. transport_maps, virtual_alias_maps /etc/postfix/transportʹ server_name forward_to_rails: Λهड़ɺ·ͨ /etc/postfix/virtual_alias_maps ʹ @server_name existing_user_name@server_name Λهड़ɻ
39. transport_maps, virtual_alias_maps main.cf ʹ transport_maps = hash:/etc/postfix/transport virtual_alias_maps = hash:/etc/postfix/virtual_aliases ͱهड़͠ɺҎԼΛ࣮ߦ $ sudo postmap /etc/postfix/transport $ sudo postmap /etc/postfix/virtual_aliases
40. ͜Ε͸Կʁ • virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂ౰Λߦ͏ • ͜͜Ͱ͸ɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰό ΢ϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠഑৴Λߦ͏ Α͏ʹ͍ͯ͠Δ • transport_maps: ഑ૹܦ࿏ࢦఆΛߦ͏ • ͜͜Ͱ͸ɺserver_name (ࣗ෼ࣗ਎) ʹདྷͨ΋ͷΛ͢΂ͯ master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢
41. forward.shͷத਎ #!/bin/sh HOME=/home/sylph01 PATH=/usr/local/bin:$PATH cd /home/sylph01/keine && bin/rails action_mailbox:ingress:postfix ... ͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻ ͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ
42. ͜͜·Ͱ΍ͬͯϝʔϧ ͕ड͚औΕ·͢ rails sͯ͠଴ͪड͚ͯΈ·͠ΐ͏
43. ϩʔΧϧʹରͯ͠ϝʔϧͷς ετΛ͢Δ৔߹ mail ίϚϯυΛ࢖͏ͷ͕Α͍Ͱ͢ɻ UbuntuͰ͸ mailutils ͷapt-get͕ඞཁɻ echo "hogefuga" | mail -s "Subject" -r from- address@yourdomain destination-address@server EOF͕͸͖ͬΓ͢ΔͷͰecho͋Δ͍͸catͰຊจΛ౉͢͜ͱΛ͓͢ ͢Ί͠·͢ɻ
45. ActiveRecordͷΦϒ δΣΫτ͕ੜ੒͞Ε ͨʂ Α͘ݟΔͱActiveStorageʹ AttachmentΛ഑ஔͨ͠Γ΋͍ͯ͠Δ
46. ࣮ࡍͲΜͳ΋ͷ͕Ͱ͖ͯΔ͔ rails c ͯ͠ m = ActionMailbox::InboundEmail.all ͱ͢Δͱड৴ͨ͠ϝʔϧ͢΂͕ͯऔΕ·͢ɻ ActionMailbox::InboundEmail ͕ ActiveRecord ͳΫϥεͳͷͰ ͍ͭ΋ͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻ த਎Λݟͯॲཧ͢ΔͨΊʹ͸ Mail::Message ΦϒδΣΫτ͕ཉ͠ ͍ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ
48. dev؀ڥͩͱConductor ͱ͍͏ศརͳUI͕Ϛ΢ ϯτ͞Ε·͢ /rails/conductor/ action_mailbox/inbound_emails
49. ϝʔϧ͔Β৘ใΛಘΔ • ૹ৴ݩΞυϨε: mail.from ʢ഑ྻͰ༩͑ΒΕΔ͜ͱʹ஫ҙʣ • ૹ৴ઌΞυϨε: mail.to ʢ͜Ε΋഑ྻʹͳΔʣ • ϔομ: mail.header['header_name'].value • Mail::Field ͕ฦͬͯ͘ΔͷͰ#value Ͱੜͷ஋ΛಘΔ • ໊݅: mail.subject • multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?
50. ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτ ͷϝʔϧʹ͍ͭͯ ݱ୅Ͱ͸ISO-2022-JPΛݟ͔͚Δස౓͸ݮͬͯUTF-8ͰΤϯίʔυ ͞Εͨϝʔϧ͕େଟ਺ͱ͸ࢥ͍·͕͢… mail.decoded ͷ୅ΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCII ͱղऍ͞Ε)จࣈԽ͚͢Δݱ৅͕͋Γ·͢ɻ charset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱ৅ͩͱࢥ͏ͷͰMail gemʹҙਤత ͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ
53. multipartͷϝʔϧͷ৔߹ ʢཁ͢Δʹఴ෇ϑΝΠϧ͕͋Δ৔߹ʣ mail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ
54. ड৴ଆͷSPF/DKIM/DMARC ຊ౰͸MTAͷଆͰϑΟϧλ͢Δ΂͖Ͱ͕͢ɺ΍Ζ͏ͱࢥ͑͹Rails ͷଆͰ΋ೝূ݁Ռͷ৘ใ͸ಘΒΕ·͢ʢMTAͷΈ͕஌ͬͯΔ৘ใ ͕ඞཁͳͷͰ൑ఆ͸Ͱ͖ͳ͍ʣɻ • SPFͷ৔߹ postfix-policyd-spf-python Λ௨͢ • DKIMͷ৔߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ෇༩͢Δ ͜ͱ͕Ͱ͖ɺͦͷத਎Λݟͯ൑ఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ
56. ࡞ͬͨ΋ͷͷ঺հɺಈ ࡞ͷ঺հ GH: sylph01/keine ͚ʔͶͳͷ͸Dark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓ Ϧ͕mokou͔ͩΒͰ͢
58. UUIDͷຒΊࠐΈ͸਺গ ͳ͍ͷͰखಈ ͿͬͪΌ͚γΣϧεΫϦϓτͱ͔Ͱࣗ ಈԽͯ͠΋͍͍
60. ΊΜͲ͍ͷͰϑϩϯτ Τϯυ͢Β࡞ͬͯͳ͍
61. application_mailbox.rb class ApplicationMailbox < ActionMailbox::Base routing /^(UUIDͷਖ਼نදݱ)@/i => :registration routing /^sylph01@/i => :greet end ଞʹ΋ɺ routing -> { (inbound_email) do_something } => :mailbox Έ͍ͨʹϝʔϧͷத਎Λ࢖ͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ
62. greet_mailbox.rb class GreetMailbox < ApplicationMailbox def process GreetMailer.greet(inbound_email).deliver_now end end process ͷதʹॲཧΛॻ͘ɻ Action MailerͰฦ͚ͩ͢ɻ
63. registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid: local_part_of_to) if user != nil if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user: user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end else bounce_with BounceMailer.bounce(inbound_email) end
64. registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid: local_part_of_to) ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λ࢖ͬͯొ࿥͞Ε͍ͯΔ UserΛ୳͢ɻ
65. registration_mailbox.rb (user == nilͷ৔߹) else bounce_with BounceMailer.bounce(inbound_email) end bounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏৘ใΛ࢒ͭͭ͠Ԡ౴͢Δ ͜ͱ͕Ͱ͖·͢ɻ
66. registration_mailbox.rb if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user: user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end Ҿ͖׵͑ࡁΈ͔Ͳ͏͔ͷ൑ఆɺҾ͖׵͑ͨΒϝʔϧΞυϨεΛอ ଘɻ
69. Έͳ͞Μʹ΋͓ࢼ͍͠ ͚ͨͩ·͢ ϋογϡλάʹ༗ޮͳUUIDΛ10ݸ͹ Β·͖·͢ʢૣ͍΋ͷউͪʣ "Dark Depths of SMTP"͕खʹೖΓ· ͢ɻҾ͖׵͑ظݶ͸ࠓ೔·ͰͰ͢
71. ·ͱΊ • ϝʔϧʹؔ͢Δ֤छ֓೦ͷ঺հΛ͠·ͨ͠ • ͕࣌ؒ͋Ε͹΋ͬͱ͜ͷ࿩͕Ͱ͖Δ • PostfixͰAction Mailbox࢖͏ํ๏ͷ঺հΛ͠·ͨ͠ • ૬౰ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ • Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷ঺հΛ͠ ·ͨ͠
72. Welcome to SMTPপ
73. Special Thanks: ٕज़ग़൛αʔΫϧ Cryptic Command ͷϝϯόʔʹϝʔϧͷಋ௨ςετ΍จ ࣈԽ͚ؔ܎ͷτϥϒϧγϡʔτͰ͓ ੈ࿩ʹͳΓ·ͨ͠
74. Questions/Comments? plz send to @s01