Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SendGrid Night 7 メルカリUS-UKでの事例とバウンスメール処理/Mercari US-UK and Bounced mail processing

kazeburo
March 29, 2017

SendGrid Night 7 メルカリUS-UKでの事例とバウンスメール処理/Mercari US-UK and Bounced mail processing

kazeburo

March 29, 2017
Tweet

More Decks by kazeburo

Other Decks in Technology

Transcript

  1. Me • Masahiro Nagano • @kazeburo • Mercari, Inc
 Principal

    Engineer
 Site Reliability Engineering (SRE) Team • BASE, Inc Technical Advisor
  2. Mercari 1. The largest Mobile Marketplace app in Japan 2.

    Instant listing 3. Safe and secure payment
  3. Email Infrastructure Ishikari DC Dedicated Server Cloud Cloud JP US

    UK on Mechanical Turk emand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers azon nical Turk Service Specific Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific ssignment/ Task Requester Workers otherMTA
  4. Email Architecture ©2011 Amazon Web Services LLC or its affiliates.

    All rights reserved. sers Client Multimedia Corporate data center Traditional server Mobile Client nagement nsole IAM Add-on Example: IAM Add-on an Intelligence asks (HIT) Assignment/ Task Requester Workers App App Queue = Q4M BUY Worker Worker Postfix Using Postfix for 1. Queuing messages 2. Faster than PHP 3. Connection aggregation
 (But Postfix can’t reuse ssl connection ) PHP PHP PHP PHP SMTP SMTP+TLS
  5. Email content(US) • Transaction Emails • Buy, Sell, Register •

    New arrival items (Daily) => • with automated schedule • Promotion Emails • From Self-made CRM / AB tests Total: Tens of millions to over 100 million / month
  6. SES bounce processing App Worker Postfix Amazon SES BOUNCE Amazon

    SNS MySQL WebHook email: failed User Users Client Multimedia Corporate data center Tra s Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific NOTIFY Oh!
  7. Make SendGrid
 Event Receiver App Worker Postfix Event WebHook User

    Users Client Multimedia d Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific NOTIFY Oh! All Action Event Receiver fluentd Amazon S3 WebHook Mackerel When BOUNCE
  8. SendGrid Event Receiver • 120 lines Perl App server +

    fluentd • Receive all SendGrid Event and log them • Log SES/SNS compatible JSON when event_type is `bounced` or `dropped` • fluentd send it to API server asynchronously • Did not need modify API code • Other logs are used for monitoring and analysis
  9. Conclusion • Mercari US/UK is using SendGrid • Thank you

    for Great Service. Delivering Emails has been made simple and reliable. • We realize bounce-mail processing by EventHook
  10. end