sneak into Auschwitz. His mission: he would allow himself to get arrested, and once there, he would organize with other Polish soldiers, coordinate a mutiny, and break out of the prison camp. www.yurynino.com
never going to happen! In a night, he cut an alarm wire and escaped! Pilecki’s story is the single most heroic thing I’ve ever come across in my life. www.yurynino.com
Communists decided to make an example of him. In 1948, they held a show trial and charged Pilecki with everything from falsifying documents and violating curfew to engaging in espionage and treason. A month later, he was found guilty and sentenced to death. On the final day of the trial, Pilecki was allowed to speak. He stated that his allegiance had always been to Poland and its people, that he had never harmed or betrayed any Polish citizen, and that he regretted nothing. He concluded his statement with “I have tried to live my life such that in the hour of my death I would feel joy rather than fear.” As Pilecki we, who work in DevOps as seen as Villains! However we shouldn’t be super heroes www.yurynino.com
software with little-to-no direct interaction with the development teams. Why are we the Villains? Developers package an application with documentation. The QA teams install and test the application.
philosophical component to it. we are so desperate for a hero today: not because things are necessarily so bad, but because we’ve lost the capacity to take responsability. www.yurynino.com
that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. DEVOPS https://aws.amazon.com/devops/what-is-devops/ www.yurynino.com
Communists decided to make an example of him. In 1948, they held a show trial and charged Pilecki with everything from falsifying documents and violating curfew to engaging in espionage and treason. A month later, he was found guilty and sentenced to death. On the final day of the trial, Pilecki was allowed to speak. He stated that his allegiance had always been to Poland and its people, that he had never harmed or betrayed any Polish citizen, and that he regretted nothing. He concluded his statement with “I have tried to live my life such that in the hour of my death I would feel joy rather than fear.” SRE Principles www.yurynino.com
system. Goals we set for how much availability we expect out of a system. Contracts with what happens if the system doesn’t meet its SLOs. Service Level Indicator Service Level Objective Service Level Agreements
a system: counts and types, error counts and processing times. A Google SRE team with 10–12 members typically has one members whose primary assignment is to build and maintain monitoring systems for their service.
that never fail. However, increasing reliability is worse for a service rather than better! Extreme reliability comes at a cost! Embrace the Risk! No sé me gusta la adrenalina
Things! • User account creation. • Software or hardware installation preparation and decommissioning. • Rollouts of new software versions. • Runtime configuration changes. Automate the Resilience!
Communists decided to make an example of him. In 1948, they held a show trial and charged Pilecki with everything from falsifying documents and violating curfew to engaging in espionage and treason. A month later, he was found guilty and sentenced to death. On the final day of the trial, Pilecki was allowed to speak. He stated that his allegiance had always been to Poland and its people, that he had never harmed or betrayed any Polish citizen, and that he regretted nothing. He concluded his statement with “I have tried to live my life such that in the hour of my death I would feel joy rather than fear.” SRE PRACTICES www.yurynino.com
orchestrate a real-time, cross-functional response. Alert Noise Reduction Group related alerts into a single incident, minimizing alert fatigue while centralizing critical context to accelerate triage. Enriched Incident Context Include graphs, images, runbook links, or links to conference calls directly in the incident details. Multiple Alert Types Send automated notifications via SMS message, mobile app push notification, phone call, or email. Todo está bien!
graphs, images, and more directly with email notifications, enabling your team to shave time off the response workflow. Dynamic Notifications Customize notification channels and behavior based on event payloads, service, or time of day. Incident History Audit Keep an audit trail of all notifications and status updates directly in the incident, including confirmation of notification delivery to devices. Todo está bien!
perform operations on production within 5 minutes for user-facing for highly time-critical and 30 minutes for less time-sensitive. • The company provides the page-receiving device, which is typically a phone. • Google has flexible alert delivery systems that can dispatch pages via multiple mechanisms (email, SMS, robot call, app) across multiple devices.
how a system is supposed to work. Expertise is gained by investigating why a system doesn't work. Brian Redman Ways in which things go right are special cases of the ways in which things go wrong. John Allspaw www.yurynino.com Effective Troubleshooting
be executed upon in the event of an IT incident. • An incident response process is something you hope to never need. LEARN Incident post-mortems are a great way for teams to continuously learn ASSESS Collaborate with subject matter experts and work with your incident commander RESOLVE Once a plan of attack has been formulated, incident resolution begins. Response Emergency Response
constant iteration Analysis & Learning • Understand what happened and build new action plans and training • Siloed apps for reporting and analysis make it hard to summarize incidents, delaying post mortems Detection & Alerting • Monitoring systems send alerts for issues which need to be reviewed and escalated • Alerting systems generate noise which is lost in email, unclear where to escalate Remediation • Resolve the issues • Multiple tools, people and processes need to be coordinated for the implementation of long-term solutions for incidents. Process Overview & Challenges Containment • Data must be reviewed and the current situation assessed • Damage has to be contained quickly and efficiently to minimize the impact. Incident Management
from it? A postmortem is an artifact with a detailed description of exactly what went wrong in an incident. A postmortem is a written record of an incident, its impact, the actions taken to mitigate it, the root cause, and the follow-up actions to prevent the incident.
Communists decided to make an example of him. In 1948, they held a show trial and charged Pilecki with everything from falsifying documents and violating curfew to engaging in espionage and treason. A month later, he was found guilty and sentenced to death. On the final day of the trial, Pilecki was allowed to speak. He stated that his allegiance had always been to Poland and its people, that he had never harmed or betrayed any Polish citizen, and that he regretted nothing. He concluded his statement with “I have tried to live my life such that in the hour of my death I would feel joy rather than fear.” Muchas gracias! @yurynino www.yurynino.com