ϝϧΧϦUS/UKͰͷࣄྫͱ
όϯεϝʔϧॲཧͷհ
Masahiro Nagano @kazeburo
SendGrid Night #7
Mercari US/UK and Bounced mail processing
Slide 2
Slide 2 text
Me
• Masahiro Nagano
• @kazeburo
• Mercari, Inc
Principal Engineer
Site Reliability Engineering (SRE) Team
• BASE, Inc Technical Advisor
Slide 3
Slide 3 text
Agenda
• About Mercari
• Mercari Global Infrastructure
• Introducing of our bounce-mail processing
Slide 4
Slide 4 text
Mercari
1. The largest Mobile
Marketplace app in
Japan
2. Instant listing
3. Safe and secure payment
Slide 5
Slide 5 text
Global Service
JP US UK
2017/03/15
Launch
2016/08
3rd place
in AppStore
Slide 6
Slide 6 text
Global Infrastructure
Ishikari DC
Dedicated Server Cloud Cloud
JP US UK
Slide 7
Slide 7 text
Email Infrastructure
Ishikari DC
Dedicated Server Cloud Cloud
JP US UK
on Mechanical Turk
emand Workforce
Human Intelligence
Tasks (HIT)
Assignment/
Task
Requester
Workers
azon
nical Turk
Service Specific
Amazon Mechanical Turk
On-Demand Workforce
Human Intelligence
Tasks (HIT)
Assignment/
Task
Workers
Amazon
Mechanical Turk
Non-Service Specific
ssignment/
Task
Requester
Workers
otherMTA
Email content(US)
• Transaction Emails
• Buy, Sell, Register
• New arrival items (Daily) =>
• with automated schedule
• Promotion Emails
• From Self-made CRM / AB tests
Total: Tens of millions to over 100 million / month
Slide 10
Slide 10 text
Bounce mail processing
Slide 11
Slide 11 text
A while ago
We were using SES.
Slide 12
Slide 12 text
SES bounce processing
App
Worker
Postfix
Amazon SES
BOUNCE
Amazon SNS
MySQL
WebHook
email: failed
User Users Client Multimedia Corporate
data center
Tra
s
Mobile Client
Amazon Mechanical Turk
On-Demand Workforce
Human Intelligence
Tasks (HIT)
Assignment/
Task
Requester
Workers
Amazon
Mechanical Turk
Non-Service Specific NOTIFY
Oh!
Slide 13
Slide 13 text
We currently use SendGrid
Slide 14
Slide 14 text
SendGrid has
Event WebHook
Slide 15
Slide 15 text
Make SendGrid
Event Receiver
App
Worker
Postfix
Event
WebHook
User Users Client Multimedia
d
Mobile Client
Amazon Mechanical Turk
On-Demand Workforce
Human Intelligence
Tasks (HIT)
Assignment/
Task
Requester
Workers
Amazon
Mechanical Turk
Non-Service Specific NOTIFY
Oh!
All
Action
Event
Receiver
fluentd
Amazon S3
WebHook
Mackerel
When
BOUNCE
Slide 16
Slide 16 text
SendGrid Event Receiver
• 120 lines Perl App server + fluentd
• Receive all SendGrid Event and log them
• Log SES/SNS compatible JSON when event_type is
`bounced` or `dropped`
• fluentd send it to API server asynchronously
• Did not need modify API code
• Other logs are used for monitoring and analysis
Slide 17
Slide 17 text
Monitoring with Mackerel
Something
Happened?
Visualizing + Alerting
Slide 18
Slide 18 text
Conclusion
• Mercari US/UK is using SendGrid
• Thank you for Great Service. Delivering Emails has
been made simple and reliable.
• We realize bounce-mail processing by EventHook