@ Kaigi on Rails, 2020/10/03
Action Mailbox inActionRyo Kajiwara (sylph01)@ Kaigi on Rails, 2020/10/03
View Slide
୭ʁsylph01 / ֿݪ ཾTwitter: @s01҉߸ͱ͔Ͱ͖·͢Elixirͱ͔Ͱ͖·͢Rails·ΔͰΘ͔ΒΜ
Rails(మಓ)ʹΑ͘Γ·͢
͏ͪΐͬͱਅ໘ʹ• ϓϩάϥϚɻ11݄͔ΒੜʹͳΓ·͢• "Dark Depths of SMTP" (2017) ͱ͍͏SMTPͷബ͍ຊΛॻ͖·ͨ͠• W3C, IETFͳͲͰηΩϡϦςΟدΓͷϓϩτίϧͷඪ४Խͷ͓ख͍Λ͍ͯ͠·ͨ͠• HTTPS in Local Network, MessagingLayer SecurityͳͲ• 2020͔ΒISOC-JPͷofficerΛͬͯ·͢
εϥΠυͪ͜ΒͰݟΕ·͢https://speakerdeck.com/sylph01/action-mailbox-in-action
TL;DR
SMTPΛΊΖ
ͪΌΜͱͨ͠༰հࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ͜ͱͰϝʔϧϓϩτίϧͷجૅͱRailsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜ͱΛతͱ͠·͢
ͪΌΜͱͨ͠༰հ• Action MailboxͱԿ͔• ϝʔϧʹؔ͢Δ֤छ֓೦• ಛʹɺSPF, DKIM, DMARC• (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙• αϯϓϧΞϓϦ• ϝʔϧΛͬͨిࢠॻ੶ͷ"social DRM"
લաڈʹ͠ΌͬͨεϥΠυͱࣅͯ·͢https://bit.ly/2S4SKEG
Action MailboxͱԿ͔• Rails 6ͰՃ͞Εͨ৽ػೳ• Action MailerϝʔϧΛ ૹ৴͢Δ ͨΊͷͷ͕ͩɺActionMailboxRailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ• ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม• ҰఆظؒܦͬͨΒࣗಈম٫ʢআʣ
Rails GuidesʹMTAʮͷʯೖΓޱΛඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ
ݸਓͰϝʔϧΔͷ͓͢͢Ί͠·ͤΜ• ໎ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬खʹಧ͖·ͤΜ• IMAPαʔόʔΛཱͯΔͱετϨʔδࠈʹؕΓ·͢• SMTPΛΊΖ
Guidesʹॻ͍ͯ͋ΔαʔϏεΛ͓͏Mailgun, Mandrill, Postmark,SendGrid, Amazon SES...
֓೦ͷհMTA: ڱٛͷʮϝʔϧαʔόʔʯͱ͍ͬͨΒ͜Εͷ͜ͱ͖ͬ͞ڍ͛ͨWebαʔϏεMTAͷׂΛͬͯ͘ΕΔMUA: ϢʔβʔΤʔδΣϯτʢϝʔϧιϑτʣ
ओʹMTA͕͠ΌΔͷ͕SMTPϝʔϧϘοΫεͱMUA͕͠ΌΔͷ͕POP3IMAP
SPF, DKIM͜ͷϝʔϧͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔόʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻͲͪΒDNSͷTXTϨίʔυʹهड़Λߦ͏ɻ• SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ• DKIM: ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔൿີ伴Λར༻ͯ͠ϝοηʔδʹॺ໊͢Δɻ
DMARC• ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROMίϚϯυͰ͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛऔΔ• Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ໊ͷҰகΛऔΔͱ͍͏ՃͷೝূΛ͢Δɻࣦഊͨ͠߹ʹυϝΠϯΦʔφʔʹͷ͋ΔϝʔϧΛใࠂͰ͖ΔΈ͋Δɻ
࣮ࡍͷઃఆ$ dig TXT kaigionrails.s01.ninja @8.8.8.8$ dig TXT mail._domainkey.kaigionrails.s01.ninja@8.8.8.8$ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ͕ݟΕ·͢ɻ
ઃఆखॱ: ૹ৴ଆҎԼͷใΛυϝΠϯϨίʔυʹॻ͘ɻ• SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε• DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴• Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞΛ࡞Δඞཁ͕͋Δ• DMARC: DMARCͷೝূʹࣦഊͨ͠߹ͷڍಈड৴ଆͷઃఆޙͰઆ໌͠·͢ɻ
Action Mailbox inAction : The Hard Way͋Δ͍ɺࣗͰMTA(Postfix)Λӡ༻ͯ͠Action Mailboxʹͭͳ͙
͜͜·ͰRails Guidesͷ௨Γ• config.action_mailbox.ingress = :relay• rails credentials:edit ͯ͠ ingress_password Ληοτ• Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ͕ΔΑ͏ʹͳΔϋζʂ• ͳΓ·ͤΜ
bin/railsaction_mailbox:ingress:postfixͱԿ͔• ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ• ༩͑ΒΕͨ INGRESS_PASSWORD Λͬͯೝূ͠• Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ͢ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔͷΛͪଓ͚·͢
Postfixଆͷઃఆهड़͖͢ઃఆϑΝΠϧͱͯ͠ओͳͷʹ main.cf, master.cf ͱ͍͏ͷ͕͋Γ·͢ɻ• શମʹؔ͢Δઃఆ main.cf ʹॻ͘• master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ master.cf ʹॻ͘• ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ
master.cf ͷهड़forward_to_rails unix - n n - - pipeflags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user}• forward_to_rails ໊শͳͷͰҙ• unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ• ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no• flags: XίϚϯυ͕࠷ऴ৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱqΞυϨεͷѻ͍
transport_maps,virtual_alias_maps/etc/postfix/transportʹserver_name forward_to_rails:Λهड़ɺ·ͨ /etc/postfix/virtual_alias_maps ʹ@server_name [email protected]_nameΛهड़ɻ
transport_maps,virtual_alias_mapsmain.cf ʹtransport_maps = hash:/etc/postfix/transportvirtual_alias_maps = hash:/etc/postfix/virtual_aliasesͱهड़͠ɺҎԼΛ࣮ߦ$ sudo postmap /etc/postfix/transport$ sudo postmap /etc/postfix/virtual_aliases
͜ΕԿʁ• virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂΛߦ͏• ͜͜Ͱɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰόϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠৴Λߦ͏Α͏ʹ͍ͯ͠Δ• transport_maps: ૹܦ࿏ࢦఆΛߦ͏• ͜͜Ͱɺserver_name (ࣗࣗ) ʹདྷͨͷΛͯ͢master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢
forward.shͷத#!/bin/shHOME=/home/sylph01PATH=/usr/local/bin:$PATHcd /home/sylph01/keine &&bin/rails action_mailbox:ingress:postfix ...͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ
͜͜·Ͱͬͯϝʔϧ͕ड͚औΕ·͢rails sͯͪ͠ड͚ͯΈ·͠ΐ͏
ϩʔΧϧʹରͯ͠ϝʔϧͷςετΛ͢Δ߹mail ίϚϯυΛ͏ͷ͕Α͍Ͱ͢ɻUbuntuͰ mailutils ͷapt-get͕ඞཁɻecho "hogefuga" | mail -s "Subject" -r from-[email protected] [email protected]EOF͕͖ͬΓ͢ΔͷͰecho͋Δ͍catͰຊจΛ͢͜ͱΛ͓͢͢Ί͠·͢ɻ
ActiveRecordͷΦϒδΣΫτ͕ੜ͞ΕͨʂΑ͘ݟΔͱActiveStorageʹAttachmentΛஔͨ͠Γ͍ͯ͠Δ
࣮ࡍͲΜͳͷ͕Ͱ͖ͯΔ͔rails c ͯ͠m = ActionMailbox::InboundEmail.allͱ͢Δͱड৴ͨ͠ϝʔϧ͕ͯ͢औΕ·͢ɻActionMailbox::InboundEmail ͕ ActiveRecord ͳΫϥεͳͷͰ͍ͭͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻதΛݟͯॲཧ͢ΔͨΊʹ Mail::Message ΦϒδΣΫτ͕ཉ͍͠ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ
devڥͩͱConductorͱ͍͏ศརͳUI͕Ϛϯτ͞Ε·͢/rails/conductor/action_mailbox/inbound_emails
ϝʔϧ͔ΒใΛಘΔ• ૹ৴ݩΞυϨε: mail.from ʢྻͰ༩͑ΒΕΔ͜ͱʹҙʣ• ૹ৴ઌΞυϨε: mail.to ʢ͜ΕྻʹͳΔʣ• ϔομ: mail.header['header_name'].value• Mail::Field ͕ฦͬͯ͘ΔͷͰ#value ͰੜͷΛಘΔ• ໊݅: mail.subject• multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?
ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτͷϝʔϧʹ͍ͭͯݱͰISO-2022-JPΛݟ͔͚ΔසݮͬͯUTF-8ͰΤϯίʔυ͞Εͨϝʔϧ͕େଟͱࢥ͍·͕͢…mail.decoded ͷΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCIIͱղऍ͞Ε)จࣈԽ͚͢Δݱ͕͋Γ·͢ɻcharset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱͩͱࢥ͏ͷͰMail gemʹҙਤత͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ
multipartͷϝʔϧͷ߹ʢཁ͢ΔʹఴϑΝΠϧ͕͋Δ߹ʣmail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ
ड৴ଆͷSPF/DKIM/DMARCຊMTAͷଆͰϑΟϧλ͢Δ͖Ͱ͕͢ɺΖ͏ͱࢥ͑RailsͷଆͰೝূ݁ՌͷใಘΒΕ·͢ʢMTAͷΈ͕ͬͯΔใ͕ඞཁͳͷͰఆͰ͖ͳ͍ʣɻ• SPFͷ߹ postfix-policyd-spf-python Λ௨͢• DKIMͷ߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ༩͢Δ͜ͱ͕Ͱ͖ɺͦͷதΛݟͯఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ
࡞ͬͨͷͷհɺಈ࡞ͷհGH: sylph01/keine͚ʔͶͳͷDark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓϦ͕mokou͔ͩΒͰ͢
UUIDͷຒΊࠐΈগͳ͍ͷͰखಈͿͬͪΌ͚γΣϧεΫϦϓτͱ͔ͰࣗಈԽ͍͍ͯ͠
ΊΜͲ͍ͷͰϑϩϯτΤϯυ͢Β࡞ͬͯͳ͍
application_mailbox.rbclass ApplicationMailbox < ActionMailbox::Baserouting /^(UUIDͷਖ਼نදݱ)@/i => :registrationrouting /^[email protected]/i => :greetendଞʹɺrouting -> { (inbound_email) do_something } => :mailboxΈ͍ͨʹϝʔϧͷதΛͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ
greet_mailbox.rbclass GreetMailbox < ApplicationMailboxdef processGreetMailer.greet(inbound_email).deliver_nowendendprocess ͷதʹॲཧΛॻ͘ɻAction MailerͰฦ͚ͩ͢ɻ
registration_mailbox.rbfrom_address = mail.from.firstlocal_part_of_to = mail.to.first.split("@").firstuser = User.find_by(uuid: local_part_of_to)if user != nilif user.redeemed? == falseuser.email = from_addressuser.saveIssuerMailer.with(user: user).issue(inbound_email).deliver_nowelseif user.email == from_addressIssuerMailer.with(user: user).issue(inbound_email).deliver_nowelsebounce_with BounceMailer.already_redeemed(inbound_email)endendelsebounce_with BounceMailer.bounce(inbound_email)end
registration_mailbox.rbfrom_address = mail.from.firstlocal_part_of_to = mail.to.first.split("@").firstuser = User.find_by(uuid: local_part_of_to)ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λͬͯొ͞Ε͍ͯΔUserΛ୳͢ɻ
registration_mailbox.rb(user == nilͷ߹)elsebounce_with BounceMailer.bounce(inbound_email)endbounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏ใΛͭͭ͠Ԡ͢Δ͜ͱ͕Ͱ͖·͢ɻ
registration_mailbox.rbif user.redeemed? == falseuser.email = from_addressuser.saveIssuerMailer.with(user: user).issue(inbound_email).deliver_nowelseif user.email == from_addressIssuerMailer.with(user: user).issue(inbound_email).deliver_nowelsebounce_with BounceMailer.already_redeemed(inbound_email)endendҾ͖͑ࡁΈ͔Ͳ͏͔ͷఆɺҾ͖͑ͨΒϝʔϧΞυϨεΛอଘɻ
Έͳ͞Μʹ͓ࢼ͍͚ͨͩ͠·͢ϋογϡλάʹ༗ޮͳUUIDΛ10ݸΒ·͖·͢ʢૣ͍ͷউͪʣ"Dark Depths of SMTP"͕खʹೖΓ·͢ɻҾ͖͑ظݶࠓ·ͰͰ͢
·ͱΊ• ϝʔϧʹؔ͢Δ֤छ֓೦ͷհΛ͠·ͨ͠• ͕࣌ؒ͋Εͬͱ͜ͷ͕Ͱ͖Δ• PostfixͰAction Mailbox͏ํ๏ͷհΛ͠·ͨ͠• ૬ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ• Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷհΛ͠·ͨ͠
Welcome toSMTPপ
Special Thanks:ٕज़ग़൛αʔΫϧ Cryptic Commandͷϝϯόʔʹϝʔϧͷಋ௨ςετจࣈԽ͚ؔͷτϥϒϧγϡʔτͰ͓ੈʹͳΓ·ͨ͠
Questions/Comments?plz send to @s01