Slide 1

Slide 1 text

ϝϧΧϦUS/UKͰͷࣄྫͱ ό΢ϯεϝʔϧॲཧͷ঺հ Masahiro Nagano @kazeburo
 SendGrid Night #7 Mercari US/UK and Bounced mail processing

Slide 2

Slide 2 text

Me • Masahiro Nagano • @kazeburo • Mercari, Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team • BASE, Inc Technical Advisor

Slide 3

Slide 3 text

Agenda • About Mercari • Mercari Global Infrastructure • Introducing of our bounce-mail processing

Slide 4

Slide 4 text

Mercari 1. The largest Mobile Marketplace app in Japan 2. Instant listing 3. Safe and secure payment

Slide 5

Slide 5 text

Global Service JP US UK 2017/03/15 Launch 2016/08 3rd place in AppStore

Slide 6

Slide 6 text

Global Infrastructure Ishikari DC Dedicated Server Cloud Cloud JP US UK

Slide 7

Slide 7 text

Email Infrastructure Ishikari DC Dedicated Server Cloud Cloud JP US UK on Mechanical Turk emand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers azon nical Turk Service Specific Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific ssignment/ Task Requester Workers otherMTA

Slide 8

Slide 8 text

Email Architecture ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. sers Client Multimedia Corporate data center Traditional server Mobile Client nagement nsole IAM Add-on Example: IAM Add-on an Intelligence asks (HIT) Assignment/ Task Requester Workers App App Queue = Q4M BUY Worker Worker Postfix Using Postfix for 1. Queuing messages 2. Faster than PHP 3. Connection aggregation
 (But Postfix can’t reuse ssl connection ) PHP PHP PHP PHP SMTP SMTP+TLS

Slide 9

Slide 9 text

Email content(US) • Transaction Emails • Buy, Sell, Register • New arrival items (Daily) => • with automated schedule • Promotion Emails • From Self-made CRM / AB tests Total: Tens of millions to over 100 million / month

Slide 10

Slide 10 text

Bounce mail processing

Slide 11

Slide 11 text

A while ago We were using SES.

Slide 12

Slide 12 text

SES bounce processing App Worker Postfix Amazon SES BOUNCE Amazon SNS MySQL WebHook email: failed User Users Client Multimedia Corporate data center Tra s Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific NOTIFY Oh!

Slide 13

Slide 13 text

We currently use SendGrid

Slide 14

Slide 14 text

SendGrid has Event WebHook

Slide 15

Slide 15 text

Make SendGrid
 Event Receiver App Worker Postfix Event WebHook User Users Client Multimedia d Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific NOTIFY Oh! All Action Event Receiver fluentd Amazon S3 WebHook Mackerel When BOUNCE

Slide 16

Slide 16 text

SendGrid Event Receiver • 120 lines Perl App server + fluentd • Receive all SendGrid Event and log them • Log SES/SNS compatible JSON when event_type is `bounced` or `dropped` • fluentd send it to API server asynchronously • Did not need modify API code • Other logs are used for monitoring and analysis

Slide 17

Slide 17 text

Monitoring with Mackerel Something Happened? Visualizing + Alerting

Slide 18

Slide 18 text

Conclusion • Mercari US/UK is using SendGrid • Thank you for Great Service. Delivering Emails has been made simple and reliable. • We realize bounce-mail processing by EventHook

Slide 19

Slide 19 text

end