DMARC Adaption Case Study JPAAWG 2nd General Meeting Nov 15, 2019 Kenji Kitaura Data Science & AI Department Rakuten, Inc. 

2 

3 Agenda § Email Sending in Rakuten § DMARC Adoption Best Practice § DMARC Adoption Case Study 

4 Email Sending in Rakuten 

5 Senders ü +70 Rakuten Services ü +48,000 Merchants(Ichiba) ü +2,900 Facility(Travel) ü Many Partners Rakuten Services Merchants, Hotels & Partners Customers 

6 Platform to send email Cloud Service 3rd Party tool Rakuten DC ü Email Service Provider ü AWS, Azure, GCP ü CRM Solutions(Salesforce etc) ü Common Email Platform ü Service Specific Email System The majority of emails are sent from on-premise common email platforms. Rakuten also support merchant email sending. 

7 Email Sending in Rakuten Various combinations of sender, platform, and purpose complicate the analysis of email sending. Managing all emails are very difficult. Sender ü Common Email Platform in Rakuten DC ü Service specific Email Server ü Cloud Email Service and 3rd party tools(Marketing tool, CRM such as Salesforce etc…) ü +70 Services ü +48K Merchants ü +2.9K Facilities ü Many Partners Sending Platform 

8 Our Challenges 

9 Damage caused by Spoofed email Unfortunately, spoofed emails disguised as Rakuten are sent continuously. We take this situation seriously and are working on countermeasures. 

10 Spoofed email Sample Spoofing emails were sent which can’t be distinguished from fake and real. 

11 Spoofed email Sample Spoofing emails were sent which can’t be distinguished from fake and real. [【ご注意ください】楽天カードを装った不審なメール(カード利⽤お知らせメール) ・ https://ichiba.faq.rakuten.net/detail/000007165・2019/10/29] Highlight links to suspicious files 

12 Risks from Spoofing emails § Damage brand image § Refrain from using services because of anxiety § Increased inquiries 

13 Anti spoofing email project 

14 Project structure Information Security Email Platform Network Operation Domain Management Cyber Security Brand Marketing Under information security team initiatives, project members were selected from each organization. Each member was their org leader. Once they aligned with the direction, each member committed their responsibility as org leader. 

15 Objective Taking full advantage of the sender authentication technology so that we protect users from email spoofing. Scope ü All Japanese Business Domains ü 100% adaption of DKIM(with First Party Signature) ü Publish DMARC with p=reject ü SPF is optional(as much as possible) 

16 DMARC Adoption Best Practice 

17 Phases definition 1. Publish DMARC & Understand your email sending 2. Adaption DKIM & SPF 3. Policy Ramp-up & Monitoring Let’s review the general DMARC Adaption steps. 

18 1. Publish DMARC & Understand your email sending ü Publish DMARC recode with p=none ü Start receiving DMARC report (rua) ü Understand the From Domain used in your organization ü Understand all email delivery routes • DMARC Record Creation • DNS Record Lookup and Parsing • Report Parsing and Visualization Tools for this Phase https://dmarc.org/resources/deployment-tools/ : 2019.11.06 There is no risk just by issuing a DMARC record. However, we have to carefully grasp the current situation. 

19 2. Adaption of DKIM & SPF ü Implement DKIM and / or SPF. ü The SPF and DKIM domains are aligned with the domain where DMARC was declared. ü Check the authentication result in the mail header. Confirm Authentication-Results, located in email header. ü Confirm that all the emails sent from your organization pass DMARC in the DMARC report. ü Policy change decision making. • Report Parsing and Visualization • Message Validation Tools for this Phase Adaption of DKIM and SPF on the confirmed email sending infrastructure at the previous step. 

20 3. Policy Ramp-up & Monitoring ü Change to p=quarantine with small pct(%) ü Increase pct to 100 step by step ü Change to p=reject with small pct ü Increase pct to 100 step by step ü Keep monitoring • Report Parsing and Visualization Tools for this Phase Policies will become stricter(none→quarantine→reject) and wider(pct=5 → 10 …. 100 ) gradually. 

21 DMARC Adoption Case Study In Rakuten 

22 Importance of sender authentication 

23 Major Email Box Provider are ready Ratio of email to Global Email Box Provider(EBP) such as @gmail.com has doubled in 6 years. Major EBPs are actively using sender authentication. They have been involved in it since the specification discussion. 

24 DMARC Adaption steps in Rakuten 

25 DMARC Adaption steps Publish initial record with p=none Collect and analyze DMARC report DKIM & SPF adaption Policy ramp-up decision Initial Auditing Phase Policy Ramp-up Phase Ongoing monitoring Phase p=reject(quarantine and pct are option) Realtime email open rate monitoring in 2 days after changing Verify DMARC report Confirm Number of inquiries to call center DMARC success rate in sending platform Find unknown platform and check DMARC success rate Steps for introducing DMARC. At Rakuten, the goal is basically p=reject. 

26 DKIM & SPF Adaption Cloud Service 3rd Party tool Rakuten DC ü DKIM: Must be First Party Signature ü SPF: Align with DMARC domain(relaxed) ü DKIM: Must be First Party Signature ü SPF: Align with DMARC domain is optional Pass DMARC based on DKIM. SPF alignment is also supported in Rakuten DC. SPF alignment on 3rd party tool makes maintenance complex. 

27 DMARC report analysis All domain reports are collected and analyzed in one system. Domain owners are able to see the detail DMARC, DKIM and SPF success rate for each platform. 

28 Grasp the point to analyze DMARC report ü Analyze by subdomain ü Group source IPs by organization domain. (It may be divided by smaller meaningful groups such as transactional IPs and promotional IPs .) ü Analyze the pass rate of DMARC, DKIM, and SPF separately. ü Check if DKIM and SPF are aligned with DMARC domain. ü Check if DMARC failed emails pass ARC. I recommend using the DMARC report analysis tool. However, it is better to use it after understanding the important points for you. 

29 Decision making & Promotion 

30 Top down and Bottom up with Guidance Since it is necessary to implement with many services, we asked for top-down and invited the operators to briefing session. CISO CxO (Executives) Service Director Operator Service Tech External tool Vendors Project Team Technical Assistant Guidance/Manual Seminar/Consultation Describe directly 

31 Result 

32 DMARC Implemented Domains ü DMARC is published in almost all domains. Stricter policy for some domains. ü On a volume basis, more than half of the emails have already been p=reject. ü All of new business domain will be applied p=reject as default 7.2% p=reject (Domain) Email sent by p=reject 56% ※ As of 2019/10/29 100% DMARC adaption 

33 DMARC pass ratio trend In the second half of last year, we focused on DKIM adaption. DMARC pass by SPF. 

34 Spoofing emails trend from DMARC report There were occasional large phishing campaigns. There is always a certain amount of unauthorized emails in normal times. One of large service Domain: p=reject 100% Other Major Domains p=reject 100% 

35 Email delivery status become clear by DMARC report An analysis of the DMARC report revealed email statistics. ü Your email sending platform ü Number of emails ü DKIM and SPF implementation status ü Number of spoofing emails 

36 Number of inbound phone call decreased in call center The number of phone calls regarding phishing has temporarily decreased by about 50% since the adaptation of DMARC with p=reject. DMARC p=reject implemented on 2017/11 Reject rate against unauthorized email 

37 Blocker and Solution 

38 DKIM Signature Failure 1. Body line length > 998 bytes → Using Base64 Encoding 2. Subject: header line length > 989 bytes → Actually this wasn’t a problem. 3. From(RFC5322.From) : header line length > 257 bytes → We made announcement to shorten it. 4. Subject: header first line is empty or single space → We changed email library at client side. In one of our environment(Postfix + OpenDKIM) had DKIM signature failure cases. Subject: =?ISO-2022-JP?B?GyRCJDMkcyRKGyhC?= =?ISO-2022-JP?B?GyRCJCskcyQ4JEckORsoQg==?= Verification Failure Verification Failure Verification Failure No Signature ※ This is only confirmed for specific version combinations. Newer versions may not have the same problem. 

39 Forwarded Email did not pass DMARC 1) Partners which forward email through Mailing List 2) Many Forwarded email from Mail Box Provider Gmail & etc DMARC Report < Mailing List Mail Box Provider 1) 2) Rakuten has a lot of forwarded email because major business model are B to B to C models. For 1), the Merchant was identified from the SPF Authentication Result. Then we asked them to stop forwarding. For 2), we didn’t encounter several failure cases. 

40 Deliverability Monitoring In order to maintain Deliverability, it is necessary to monitor the number of emails that have not arrived unexpectedly due to a defect in sender authentication. At present, there is no way to detect it early, so it is possible to monitor with user engagement KPI such as open rate. Trend of email opening in a campaign 

41 Other Initiatives 

42 Yahoo! Mail Brand Image Display Yahoo! Mail is one of the most popular email services for customers in Japan. Last year, Yahoo! Japan and Rakuten began displaying brand images using DKIM as a measure against spoofing emails. [楽天サービスに対する不正対策・https://corp.rakuten.co.jp/security/anti-fraud/・2019/10/29] Brand Symbol 

43 M3AAWG/JPAAWG ü Collect information on email & email security technologies and trends. ü Implement best practices in Rakuten services. Rakuten is actively working to improve email security as a large email sender. As part of that activity, I participate in major conferences in this field. 

No content