Action Mailbox in Action

by sylph01

Slide 1

Slide 1 text

Action Mailbox in Action Ryo Kajiwara (sylph01) @ Kaigi on Rails, 2020/10/03

Slide 2

Slide 2 text

୭ʁ sylph01 / ֿݪ ཾ Twitter: @s01 ҉߸ͱ͔Ͱ͖·͢ Elixirͱ͔Ͱ͖·͢ Rails·ΔͰΘ͔ΒΜ

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Rails(మಓ) ʹ͸Α͘৐Γ·͢

Slide 5

Slide 5 text

΋͏ͪΐͬͱਅ໘໨ʹ • ϓϩάϥϚɻ11݄͔Β໺ੜʹͳΓ·͢ • "Dark Depths of SMTP" (2017) ͱ͍͏ SMTPͷബ͍ຊΛॻ͖·ͨ͠ • W3C, IETFͳͲͰηΩϡϦςΟدΓͷ ϓϩτίϧͷඪ४Խͷ͓ख఻͍Λͯ͠ ͍·ͨ͠ • HTTPS in Local Network, Messaging Layer SecurityͳͲ • 2020೥͔ΒISOC-JPͷofficerΛ΍ͬͯ ·͢

Slide 6

Slide 6 text

εϥΠυ͸ͪ͜ΒͰݟΕ·͢ https:/ /speakerdeck.com/sylph01/actionmailbox-in-action

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

TL;DR

Slide 9

Slide 9 text

SMTP Λ΍ΊΖ

Slide 10

Slide 10 text

ͪΌΜͱͨ͠಺༰঺հ ࣗલͰϝʔϧαʔόʔ(MTA)ΛཱͯΔ ͜ͱͰϝʔϧϓϩτίϧͷجૅͱ Railsʹ͓͚Δϝʔϧͷѻ͍ΛֶͿ͜ ͱΛ໨తͱ͠·͢

Slide 11

Slide 11 text

ͪΌΜͱͨ͠಺༰঺հ • Action Mailboxͱ͸Կ͔ • ϝʔϧʹؔ܎͢Δ֤छ֓೦ • ಛʹɺSPF, DKIM, DMARC • (ϝΠϯσΟογϡ) PostfixΛAction Mailboxʹͭͳ͙ • αϯϓϧΞϓϦ • ϝʔϧΛ࢖ͬͨిࢠॻ੶ͷ"social DRM"

Slide 12

Slide 12 text

લ൒͸աڈʹ͠Ό΂ͬ ͨεϥΠυͱࣅͯ·͢ https://bit.ly/ 2S4SKEG

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Action Mailboxͱ͸Կ͔ • Rails 6Ͱ௥Ճ͞Εͨ৽ػೳ • Action Mailer͸ϝʔϧΛ ૹ৴͢Δ ͨΊͷ΋ͷ͕ͩɺAction Mailbox͸RailsͰϝʔϧΛ ड৴͢Δ ͜ͱ͕Ͱ͖Δ • ड৴ϝʔϧΛActiveRecordͷΦϒδΣΫτʹม׵ • ҰఆظؒܦͬͨΒࣗಈম٫ʢ࡟আʣ

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Rails Guidesʹ͸MTAʮ΁ͷʯೖΓޱ Λඋ͍͑ͯΔͱॻ͍ͯ͋Δ͕ɺͲͪΒ ͔ͱ͍͏ͱMTAʮ͔ΒͷʯೖΓޱ

Slide 17

Slide 17 text

ݸਓͰϝʔϧ΍Δͷ͸͓͢͢ Ί͠·ͤΜ • ໎࿭ϝʔϧରࡦ͕ඇৗʹ͠ΜͲ͍ɻઃఆϛεΔͱϝʔϧ͕૬ख ʹಧ͖·ͤΜ • IMAPαʔόʔΛཱͯΔͱετϨʔδ஍ࠈʹؕΓ·͢ • SMTPΛ΍ΊΖ

Slide 18

Slide 18 text

Guidesʹॻ͍ͯ͋Δ αʔϏεΛ࢖͓͏ Mailgun, Mandrill, Postmark, SendGrid, Amazon SES...

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

֓೦ͷ঺հ MTA: ڱٛͷʮϝʔϧαʔόʔʯͱ ͍ͬͨΒ͜Εͷ͜ͱ ͖ͬ͞ڍ͛ͨWebαʔϏε͸MTAͷ໾ׂΛ΍ͬͯ͘ΕΔ MUA: ϢʔβʔΤʔδΣϯτʢϝʔϧ ιϑτʣ

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

ओʹMTA͕͠Ό΂Δͷ ͕SMTP ϝʔϧϘοΫεͱMUA͕͠Ό΂Δͷ ͕POP3΍IMAP

Slide 23

Slide 23 text

SPF, DKIM ͜ͷϝʔϧ͸ͪΌΜͱ͜ͷυϝΠϯΛॴ༗͍ͯ͠Δਓʢͷαʔ όʔʣ͔Βདྷͯ·͢Αɺͱ͍͏͜ͱΛ͍ࣔͨ͠ɻ ͲͪΒ΋DNSͷTXTϨίʔυʹهड़Λߦ͏ɻ • SPF: ڐՄ͢ΔIPΞυϨεΛࢦఆɻ • DKIM: ެ։伴ΛTXTϨίʔυʹઃఆɻαʔόʔ͸ൿີ伴Λར༻͠ ͯϝοηʔδʹॺ໊͢Δɻ

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

DMARC • ϔομʹࣔ͞ΕΔૹ৴ऀͷυϝΠϯ(Header-From)ͱMAIL FROM ίϚϯυͰ౉͞ΕΔૹ৴ऀͷυϝΠϯ(Envelope-From)ͷҰகΛ औΔ • Header-FromͷυϝΠϯ໊ͱDKIMͷ"d="Ͱ༩͑ΒΕΔυϝΠϯ ໊ͷҰகΛऔΔ ͱ͍͏௥ՃͷೝূΛ͢Δɻࣦഊͨ͠৔߹ʹυϝΠϯΦʔφʔʹ໰ ୊ͷ͋ΔϝʔϧΛใࠂͰ͖Δ࢓૊Έ΋͋Δɻ

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

࣮ࡍͷઃఆ $ dig TXT kaigionrails.s01.ninja @8.8.8.8 $ dig TXT mail._domainkey.kaigionrails.s01.ninja @8.8.8.8 $ dig TXT _dmarc.kaigionrails.s01.ninja @8.8.8.8 ͰͦΕͧΕSPF, DKIM, DMARCͷઃఆ஋͕ݟΕ·͢ɻ

Slide 31

Slide 31 text

ઃఆखॱ: ૹ৴ଆ ҎԼͷ৘ใΛυϝΠϯϨίʔυʹॻ͘ɻ • SPF: ڐՄ/ෆڐՄ͢ΔIPΞυϨε • DKIM: DKIM signer͕༻͍Δ伴ϖΞͷެ։伴 • Αͬͯɺૹ৴ଆMTAʹDKIM signerΛΠϯετʔϧ͠ɺ伴ϖΞ Λ࡞Δඞཁ͕͋Δ • DMARC: DMARCͷೝূʹࣦഊͨ͠৔߹ͷڍಈ ड৴ଆͷઃఆ͸ޙͰઆ໌͠·͢ɻ

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Action Mailbox in Action : The Hard Way ͋Δ͍͸ɺࣗ෼ͰMTA(Postfix)Λӡ༻ ͯ͠Action Mailboxʹͭͳ͙࿩

Slide 34

Slide 34 text

͜͜·ͰRails Guidesͷ௨Γ • conﬁg.action_mailbox.ingress = :relay • rails credentials:edit ͯ͠ ingress_password Ληοτ • Α͠ɺ࣍ʹॻ͍ͯ͋Δbin/railsͷίϚϯυଧͬͨΒͳΜ͔ܨ ͕ΔΑ͏ʹͳΔϋζʂ • ͳΓ·ͤΜ

Slide 35

Slide 35 text

bin/rails action_mailbox:ingress:postﬁx ͱ͸Կ͔ • ඪ४ೖྗʹ༩͑ΒΕͨϝʔϧຊจʢϔομؚΉʣΛ • ༩͑ΒΕͨ INGRESS_PASSWORD Λ࢖ͬͯೝূ͠ • Rails͕ղऍͰ͖ΔΑ͏ʹࢦఆͨ͠ URL ʹ౉͢ ΋ͷɻ͜Ε͚ͩΛίϚϯυϥΠϯͰ࣮ߦ͢Δͱඪ४ೖྗ͕ऴΘΔ ͷΛ଴ͪଓ͚·͢

Slide 36

Slide 36 text

Postfixଆͷઃఆ هड़͢΂͖ઃఆϑΝΠϧͱͯ͠ओͳ΋ͷʹ main.cf, master.cf ͱ ͍͏΋ͷ͕͋Γ·͢ɻ • શମʹؔ͢Δઃఆ͸ main.cf ʹॻ͘ • master ϓϩηεͷಈ࡞ʹؔ͢Δઃఆ͸ master.cf ʹॻ͘ • ͍͍ͩͨʮ௨৴ͷ੍ޚʯʮϝʔϧͷϧʔςΟϯάʯ

Slide 37

Slide 37 text

master.cf ͷهड़ forward_to_rails unix - n n - - pipe ﬂags=Xhq user=sylph01:sylph01 argv=/home/sylph01/keine/forward.sh ${nexthop} ${user} • forward_to_rails ͸໊শͳͷͰ೚ҙ • unix: UNIXυϝΠϯιέοτʹϝʔϧΛྲྀ͢ɻ • ͦͷޙͷྻ: unpriv ͕noʢಛݖ͕ඞཁʣɺchroot ͕no • ﬂags: X͸ίϚϯυ͕࠷ऴ഑৴ઌͰ͋Δ͜ͱΛࣔ͢ɻhͱq͸Ξυ Ϩεͷѻ͍

Slide 38

Slide 38 text

transport_maps, virtual_alias_maps /etc/postﬁx/transportʹ server_name forward_to_rails: Λهड़ɺ·ͨ /etc/postﬁx/virtual_alias_maps ʹ @server_name existing_user_name@server_name Λهड़ɻ

Slide 39

Slide 39 text

transport_maps, virtual_alias_maps main.cf ʹ transport_maps = hash:/etc/postfix/transport virtual_alias_maps = hash:/etc/postfix/virtual_aliases ͱهड़͠ɺҎԼΛ࣮ߦ $ sudo postmap /etc/postfix/transport $ sudo postmap /etc/postfix/virtual_aliases

Slide 40

Slide 40 text

͜Ε͸Կʁ • virtual_alias_maps: ΤΠϦΞεʢผ໊ʣͷׂ౰Λߦ͏ • ͜͜Ͱ͸ɺ@server_name ʹདྷͨϝʔϧ͕ϢʔβʔෆࡏͰό ΢ϯε͠ͳ͍Α͏ɺଘࡏ͢ΔϢʔβʔ໊ʹରͯ͠഑৴Λߦ͏ Α͏ʹ͍ͯ͠Δ • transport_maps: ഑ૹܦ࿏ࢦఆΛߦ͏ • ͜͜Ͱ͸ɺserver_name (ࣗ෼ࣗ਎) ʹདྷͨ΋ͷΛ͢΂ͯ master.cf ʹॻ͍ͨ forward_to_rails: ʹྲྀ͢

Slide 41

Slide 41 text

forward.shͷத਎ #!/bin/sh HOME=/home/sylph01 PATH=/usr/local/bin:$PATH cd /home/sylph01/keine && bin/rails action_mailbox:ingress:postﬁx ... ͜͜ʹઌ΄ͲͷingressίϚϯυΛॻ͘ɻ ͪΌΜͱRubyͱRails͕ݟ͔ͭΔΑ͏ʹͯ͋͛͠Δඞཁ͕͋Δɻ

Slide 42

Slide 42 text

͜͜·Ͱ΍ͬͯϝʔϧ ͕ड͚औΕ·͢ rails sͯ͠଴ͪड͚ͯΈ·͠ΐ͏

Slide 43

Slide 43 text

ϩʔΧϧʹରͯ͠ϝʔϧͷς ετΛ͢Δ৔߹ mail ίϚϯυΛ࢖͏ͷ͕Α͍Ͱ͢ɻ UbuntuͰ͸ mailutils ͷapt-get͕ඞཁɻ echo "hogefuga" | mail -s "Subject" -r from- address@yourdomain destination-address@server EOF͕͸͖ͬΓ͢ΔͷͰecho͋Δ͍͸catͰຊจΛ౉͢͜ͱΛ͓͢ ͢Ί͠·͢ɻ

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

ActiveRecordͷΦϒ δΣΫτ͕ੜ੒͞Ε ͨʂ Α͘ݟΔͱActiveStorageʹ AttachmentΛ഑ஔͨ͠Γ΋͍ͯ͠Δ

Slide 46

Slide 46 text

࣮ࡍͲΜͳ΋ͷ͕Ͱ͖ͯΔ͔ rails c ͯ͠ m = ActionMailbox::InboundEmail.all ͱ͢Δͱड৴ͨ͠ϝʔϧ͢΂͕ͯऔΕ·͢ɻ ActionMailbox::InboundEmail ͕ ActiveRecord ͳΫϥεͳͷͰ ͍ͭ΋ͷ ActiveRecord ͷૢ࡞ͰϝʔϧΛऔΓग़ͤ·͢ɻ த਎Λݟͯॲཧ͢ΔͨΊʹ͸ Mail::Message ΦϒδΣΫτ͕ཉ͠ ͍ͷͰɺ #mail ΛݺΜͰऔΓ·͢ɻ

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

dev؀ڥͩͱConductor ͱ͍͏ศརͳUI͕Ϛ΢ ϯτ͞Ε·͢ /rails/conductor/ action_mailbox/inbound_emails

Slide 49

Slide 49 text

ϝʔϧ͔Β৘ใΛಘΔ • ૹ৴ݩΞυϨε: mail.from ʢ഑ྻͰ༩͑ΒΕΔ͜ͱʹ஫ҙʣ • ૹ৴ઌΞυϨε: mail.to ʢ͜Ε΋഑ྻʹͳΔʣ • ϔομ: mail.header['header_name'].value • Mail::Field ͕ฦͬͯ͘ΔͷͰ#value Ͱੜͷ஋ΛಘΔ • ໊݅: mail.subject • multipartͰ͋Δ͔Ͳ͏͔: mail.multipart?

Slide 50

Slide 50 text

ϝʔϧຊจͷॲཧɺಛʹϚϧνόΠτ ͷϝʔϧʹ͍ͭͯ ݱ୅Ͱ͸ISO-2022-JPΛݟ͔͚Δස౓͸ݮͬͯUTF-8ͰΤϯίʔυ ͞Εͨϝʔϧ͕େଟ਺ͱ͸ࢥ͍·͕͢… mail.decoded ͷ୅ΘΓʹ mail.body.decoded ͱ͢Δͱ(US-ASCII ͱղऍ͞Ε)จࣈԽ͚͢Δݱ৅͕͋Γ·͢ɻ charset͕Ҿ͖ܧ͕Ε͍ͯͳ͍ݱ৅ͩͱࢥ͏ͷͰMail gemʹҙਤత ͔Ͳ͏͔֬ೝ͢ΔissueΛཱͯͨɻ

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

multipartͷϝʔϧͷ৔߹ ʢཁ͢Δʹఴ෇ϑΝΠϧ͕͋Δ৔߹ʣ mail.text_part.decoded ͱ͢Δͱຊจ͕ಘΒΕ·͢ɻ

Slide 54

Slide 54 text

ड৴ଆͷSPF/DKIM/DMARC ຊ౰͸MTAͷଆͰϑΟϧλ͢Δ΂͖Ͱ͕͢ɺ΍Ζ͏ͱࢥ͑͹Rails ͷଆͰ΋ೝূ݁Ռͷ৘ใ͸ಘΒΕ·͢ʢMTAͷΈ͕஌ͬͯΔ৘ใ ͕ඞཁͳͷͰ൑ఆ͸Ͱ͖ͳ͍ʣɻ • SPFͷ৔߹ postﬁx-policyd-spf-python Λ௨͢ • DKIMͷ৔߹OpenDKIMΛverifierϞʔυͰಈ࡞ͤ͞Δ ͜ͱʹΑͬͯ Authentication-Results ͱ͍͏ϔομΛ෇༩͢Δ ͜ͱ͕Ͱ͖ɺͦͷத਎Λݟͯ൑ఆ͢Δ͜ͱ͕Ͱ͖·͢ɻ

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

࡞ͬͨ΋ͷͷ঺հɺಈ ࡞ͷ঺հ GH: sylph01/keine ͚ʔͶͳͷ͸Dark Depths of SMTPॻ͍ͨݩωλͷPhoenixͷΞϓ Ϧ͕mokou͔ͩΒͰ͢

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

UUIDͷຒΊࠐΈ͸਺গ ͳ͍ͷͰखಈ ͿͬͪΌ͚γΣϧεΫϦϓτͱ͔Ͱࣗ ಈԽͯ͠΋͍͍

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

ΊΜͲ͍ͷͰϑϩϯτ Τϯυ͢Β࡞ͬͯͳ͍

Slide 61

Slide 61 text

application_mailbox.rb class ApplicationMailbox < ActionMailbox::Base routing /^(UUIDͷਖ਼نදݱ)@/i => :registration routing /^sylph01@/i => :greet end ଞʹ΋ɺ routing -> { (inbound_email) do_something } => :mailbox Έ͍ͨʹϝʔϧͷத਎Λ࢖ͬͯϚονϯά͢Δ͜ͱ͕Ͱ͖·͢ɻ

Slide 62

Slide 62 text

greet_mailbox.rb class GreetMailbox < ApplicationMailbox def process GreetMailer.greet(inbound_email).deliver_now end end process ͷதʹॲཧΛॻ͘ɻ Action MailerͰฦ͚ͩ͢ɻ

Slide 63

Slide 63 text

registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid: local_part_of_to) if user != nil if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user: user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end else bounce_with BounceMailer.bounce(inbound_email) end

Slide 64

Slide 64 text

registration_mailbox.rb from_address = mail.from.first local_part_of_to = mail.to.first.split("@").first user = User.find_by(uuid: local_part_of_to) ϝʔϧΞυϨεͷϩʔΧϧύʔτ(UUID)Λ࢖ͬͯొ࿥͞Ε͍ͯΔ UserΛ୳͢ɻ

Slide 65

Slide 65 text

registration_mailbox.rb (user == nilͷ৔߹) else bounce_with BounceMailer.bounce(inbound_email) end bounce_with ͢Δͱʮࣦഊͨ͠ʯͱ͍͏৘ใΛ࢒ͭͭ͠Ԡ౴͢Δ ͜ͱ͕Ͱ͖·͢ɻ

Slide 66

Slide 66 text

registration_mailbox.rb if user.redeemed? == false user.email = from_address user.save IssuerMailer.with(user: user).issue(inbound_email).deliver_now else if user.email == from_address IssuerMailer.with(user: user).issue(inbound_email).deliver_now else bounce_with BounceMailer.already_redeemed(inbound_email) end end Ҿ͖׵͑ࡁΈ͔Ͳ͏͔ͷ൑ఆɺҾ͖׵͑ͨΒϝʔϧΞυϨεΛอ ଘɻ

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

Έͳ͞Μʹ΋͓ࢼ͍͠ ͚ͨͩ·͢ ϋογϡλάʹ༗ޮͳUUIDΛ10ݸ͹ Β·͖·͢ʢૣ͍΋ͷউͪʣ "Dark Depths of SMTP"͕खʹೖΓ· ͢ɻҾ͖׵͑ظݶ͸ࠓ೔·ͰͰ͢

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

·ͱΊ • ϝʔϧʹؔ͢Δ֤छ֓೦ͷ঺հΛ͠·ͨ͠ • ͕࣌ؒ͋Ε͹΋ͬͱ͜ͷ࿩͕Ͱ͖Δ • PostfixͰAction Mailbox࢖͏ํ๏ͷ঺հΛ͠·ͨ͠ • ૬౰ͳ͜ͱ͕ͳ͍ݶΓ͓͢͢Ί͠·ͤΜ • Action MailboxͱMail::MessageΦϒδΣΫτͷѻ͍ํͷ঺հΛ͠ ·ͨ͠

Slide 72

Slide 72 text

Welcome to SMTPপ

Slide 73

Slide 73 text

Special Thanks: ٕज़ग़൛αʔΫϧ Cryptic Command ͷϝϯόʔʹϝʔϧͷಋ௨ςετ΍จ ࣈԽ͚ؔ܎ͷτϥϒϧγϡʔτͰ͓ ੈ࿩ʹͳΓ·ͨ͠

Slide 74

Slide 74 text

Questions/Comments? plz send to @s01