20 Jan 2017 / This is Mercari, This is a SRE - Mercari DAY

Transcript

1. This is Mercari, This is an SRE Mercari DAY 2017

   @AcademyHills Kenichi Sasaki

2. Introduce myself • Kenichi Sasaki (@siroken3) • Mercari, Inc (JOIN

   at 2014/7)
 Site Reliability Engineering (SRE) Team • αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ • ։ൃ؀ڥͳͲͷ੔උ

3. Agenda • Site Reliability Engineering (SRE) ͱ͸ • MercariͱMercariͷSRE •

   Mercari SREͷࣄྫ • ·ͱΊ

4. SRE ͱ͸

5. 4JUF
   3FMJBCJMJUZ
   &OHJOFFSJOH w (PPHMF
   ͕ఏএ
   w (PPHMF
   ͷ༷ʑͳϓϩμΫτɾ αʔϏεΛԣஅͯ͠ɺαΠτ ͷ৴པੑΛ޲্ͤ͞Δ
   4PGUXBSF
   &OHJOFFS5FBN

6. SRE ͷ࠾༻(US) Facebook, Twitter, Apple, eBay, Microsoft.. 3,662

7. SRE in JP • 2015/11 Mercari Tech Blog ʹͯ঺հͯ͠Ҏ߱஫ ໨͕ू·Δ

   • “ΠϯϑϥνʔϜվΊ Site Reliability Engineering (SRE) νʔϜʹͳΓ·ͨ͠”
 http://tech.mercari.com/entry/2015/11/18/153421 • 2016/06 SRE Tech Talk#1 • https://connpass.com/event/34825/

8. SRE Tech Talk#2! • https://eventdots.jp/event/609456 • 2017/01/23 12:00 క੾

9. SREͷ࠾༻(JP) 40!

10. Mercari & Mercari SRE

11. Infrastructure UK ४උத JP US ͘͞ΒΠϯλʔωοτ
 ੴङDC AWS GCP

12. Architecture US nginx nginx nginx ©2011 Amazon Web Services LLC

    or its affiliates. All rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers DNS-RR App App App App App App MySQL MySQL memcached memcached nginx nginx nginx ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia C d Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific DNS-RR App App App App App App MySQL MySQL memcached memcached EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 EC2 JP Ϋϥ΢υͰ΋
    ಉ͡ߏ੒ util util cloud cloud util util EC2 EC2

13. Mercari SRE Team

14. Mercari SRE • 2015/11 ͔Β SRE ʹมߋ
 ͦΕ·Ͱ͸ΠϯϑϥνʔϜ • ϝϯόʔ͸5ਓ

    • શһ͕౦ژΦϑΟεۈ຿

15. SREʹมߋͨ͠ཧ༝ • ʮΠϯϑϥʯ͔ΒΠϝʔδ͢Δۀ຿ͷ࿮Λ௒͑Δ • ϝϧΧϦΛ5೥ɾ10೥ͱ௕͘࢖͍ͬͯͨͩͨ͘Ίͷʮ৴པੑʯ • εϚϗͰ͸24࣌ؒ޷͖ͳ࣌ʹɺ޷͖ͳ৔ॴ͔ΒΞΫηε͕དྷ Δɻ͍ͭͰ΋շదʹΞΫηε͕Ͱ͖ΔʮՄ༻ੑʯͱ
 ʮύϑΥʔϚϯεʯ •

    ιϑτ΢ΣΞΤϯδχΞϦϯάͰ͜ΕΒΛղܾ͍ͯ͘͠νʔϜ • (ւ֎/USͰ΋௨͡Δ໊শ)

16. Mercari SREͷۀ຿ • APIαʔόɺϛυϧ΢ΣΞͷՄ༻ੑɺύϑΥʔϚϯεͷ
 ҡ࣋ɾ޲্ • OnCall (ো֐ରԠͷ౰൪) • ϩάऩूɾ෼ੳج൫ͷߏஙɺӡ༻

    • αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ • ηΩϡϦςΟͷ୲อ

17. Mercari SREࣄྫ • Gaurun - push notification server • ngx_luaʹΑΔBalancer֦ு

    • PHP daemon class preload • URL Shorten Service • ChatOpsˍZero DownTime Deploy • etc…

18. ChatOps Deploy

19. ChatOps Deploy • ϝϧΧϦ • SCMͱͯ͠GitHubΛ࠾༻ • IssueTrackerͰͷ࢓༷ݕ౼ • GitHub

    PullRequestͰͷϐΞϨϏϡʔ

20. Google Calendar + ChatOps

21. Inside ChatOps @jp-go-bold: yes check PullRequest check Issue Tracker exec

    Ansible merge PR git clone git clone composer install deploy application preprocessing ϐΞϨϏϡʔࡁ Ϛωʔδϟঝೝࡁ

22. ·ͱΊ

23. ·ͱΊ • ϝϧΧϦ͸C2CϚʔέοτϓϨΠεɺ৴པੑॏཁ • SRE͸৴པੑΛγεςϜ໘͔Βࢧ͑Δ • ۀ຿͸Πϯϑϥʹཹ·Βͳ͍ɻ৴པੑΛ௥ٻ