Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Failures In Implementing Reliability

Jaakko Pallari
January 16, 2020
Programming
0
74

Failures In Implementing Reliability

Stories on trying to do something reliably but failing. Presented at SRE Finland meetup.

Jaakko Pallari

January 16, 2020
Tweet

More Decks by Jaakko Pallari

See All by Jaakko Pallari
Kubernetes security
jkpl
0
93
DevOps & SRE: Two Cultures, Coming Together, Facing Each Other
jkpl
0
200
Staging reactive data pipelines using Kafka as the backbone
jkpl
0
1.1k

Other Decks in Programming

See All in Programming
Reimagining text fields in Compose (dcNYC 23)
zachklipp
0
250
たのしいドメイン駆動設計: 序 / Enjoy domain driven design : ZYO
jpt_corp_official
8
4.2k
Build Backwards-Compatible REST APIs with Rolling Versions
subomi
0
140
PHP8.2にバージョンアップして もっと型表現を豊かにしよう
akitotsukahara
0
170
ZennにみるCloudRunとBigQueryによるアプリケーション構築 / zenn-cloudrun-bigquery-serverless
wadayusuke
13
3.6k
テーブル駆動テストと状態
hazumirr
1
290
締切とはなにか、どういう効果があるのか #scrummikawa
murabayashi
0
330
Astro 3.0入門
nozaki
0
200
The impact of API Platform on Open-Source
loic425
0
120
WebViewと向き合う
mkeeda
1
150
AWS Lambdaから始める Devチームの小さなDevOps改善 〜QCDどれも諦めない運用を目指して〜 / Start to improving small DevOps with AWS Lambda by Dev Team
cocoeyes02
0
590
Chatwork プロダクト本部 紹介資料（エンジニア向け）
chatwork_hr
1
3.2k

Featured

See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
239
20k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
17
1.3k
Learning to Love Humans: Emotional Interface Design
aarron
265
38k
Testing 201, or: Great Expectations
jmmastey
26
6k
Making the Leap to Tech Lead
cromwellryan
119
8.1k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
271
12k
Web Components: a chance to create the future
zenorocha
304
41k
Building a Modern Day 
E-commerce SEO Strategy
aleyda
12
5.7k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
10
1.2k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
112
17k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
39
3.9k
The Art of Programming - Codeland 2020
erikaheidi
39
12k

Transcript

  1. Welcome to SRE Finland!

    View Slide

  2. Slack channel
    Sign up @ devopsfinland.org
    Join #sre-finland

    View Slide

  3. View Slide

  4. The best DevOps company *
    * according to us

    View Slide

  5. The best DevOps company *
    * according to us
    we also do SRE, Dataops
    and DevSecOps

    View Slide

  6. Jaakko Pallari
    SRE Finland co-organiser
    Lead SRE Consultant @ Polar Squad
    Background in SW dev and DevOps

    View Slide

  7. FAILURES IN
    IMPLEMENTING
    RELIABILITY

    View Slide

  8. Disclaimer
    No SRE involved

    View Slide

  9. View Slide

  10. View Slide

  11. SCALE!

    View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. SCALE?

    View Slide

  16. SCALE?
    SCALE!

    View Slide

  17. SCALE!

    View Slide

  18. SCAL
    SCALE!

    View Slide

  19. View Slide

  20. “use the right
    tool for the
    right job lol”

    View Slide

  21. View Slide

  22. Mission:
    Zero downtime
    upgrades

    View Slide

  23. Azure
    Ansible
    Kubernetes
    Go
    PostgreSQL
    Monorepo

    View Slide

  24. View Slide

  25. Azure
    Kubernetes
    Go
    PostgreSQL
    Monorepo

    View Slide

  26. Azure ARM
    Ansible
    Kubernetes
    Go
    PostgreSQL
    Monorepo

    View Slide

  27. View Slide

  28. View Slide

  29. What are we
    even installing
    here?

    View Slide

  30. What is even
    installed in
    production?

    View Slide

  31. View Slide

  32. 1.0.4, 1.0.5 ... 1.6.1, 1.6.2
    in prod latest

    View Slide

  33. PRODUCTION
    PRODUCTION v2

    View Slide

  34. View Slide

  35. THE INFRA

    View Slide

  36. THE INFRA

    View Slide

  37. EMPOWER AND TRUST

    View Slide

  38. THE INFRA

    View Slide

  39. THE INFRA

    View Slide

  40. What was
    learned

    View Slide

  41. It’s OK to trust the
    tools you know

    View Slide

  42. Focus on
    the right issue

    View Slide

  43. EMPOWER AND TRUST
    the teams you work with

    View Slide

  44. thank you

    View Slide