Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Heartbleed at Acquia

Avatar for Marc Seeger Marc Seeger
May 20, 2014
Technology
0
15k

Heartbleed at Acquia

A quick presentation on how we handled Heartbleed at Acquia. Held at a DevOps Boston meetup.

Avatar for Marc Seeger

Marc Seeger

May 20, 2014
Tweet

More Decks by Marc Seeger

See All by Marc Seeger
Security in DECT
Avatar for Marc Seeger rb2k
2
170
The DIRAC video codec
Avatar for Marc Seeger rb2k
1
83
Communitygetriebene Android Systemerweiterungen
Avatar for Marc Seeger rb2k
1
54
Alternative infrastructure
Avatar for Marc Seeger rb2k
1
180
NoSQL Lunch and Learn
Avatar for Marc Seeger rb2k
9
8.5k
Lunch and Learn: Cucumber and Capybara
Avatar for Marc Seeger rb2k
7
21k

Other Decks in Technology

See All in Technology
身近なCSVを活用する！AWSのデータ分析基盤アーキテクチャ
Avatar for Kotaro.S koosun
0
3.9k
How We Built a Secure Sandbox Platform for AI
Avatar for GMO Flatt Security flatt_security
1
110
Building AI Applications with Java, LLMs, and Spring AI
Avatar for Thomas Vitale thomasvitale
1
230
大規模プロダクトで実践するAI活用の仕組みづくり
Avatar for daiki k1tikurisu
5
1.8k
AI時代の戦略的アーキテクチャ 〜Adaptable AI をアーキテクチャで実現する〜 / Enabling Adaptable AI Through Strategic Architecture
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
PRO
15
9.6k
The Complete Android UI Testing Landscape: From Journey to Traditional Approaches
Avatar for Alex Zhukovich alexzhukovich
1
100
技術広報のOKRで生み出す 開発組織への価値 〜 カンファレンス協賛を通して育む学びの文化 〜 / Creating Value for Development Organisations Through Technical Communications OKRs — Nurturing a Culture of Learning Through Conference Sponsorship —
Avatar for Pauli pauli
5
520
Kubernetesと共にふりかえる! エンタープライズシステムのインフラ設計・テストの進め方大全
Avatar for 高棹大樹 daitak
0
440
Redux → Recoil → Zustand → useSyncExternalStore： 状態管理の10年とReact本来の姿
Avatar for ZOZO Developers zozotech
PRO
21
9k
.NET 10のEntity Framework Coreの新機能
Avatar for tkym htkym
0
100
AI駆動開発2025年振り返りとTips集
Avatar for KNR knr109
1
100
[CV勉強会@関東 ICCV2025 読み会] World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model (Zheng+, ICCV 2025)
Avatar for abemii_ abemii
0
250

Featured

See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
132
19k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1371
200k
Code Review Best Practice
Avatar for Trisha Gee trishagee
72
19k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.9k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8k
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
8
1.1k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
127
54k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
1.8k
Navigating Team Friction
Avatar for Lara Hogan lara
190
16k
How Fast Is Fast Enough? [PerfNow 2025]
Avatar for Tammy Everts tammyeverts
3
340
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
62k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k

Transcript

  1. Marc Seeger (@rb2k)
 Boston Devops Meetup
 May 20th 2014 at

  2. Act 1: Technology

  3. How it all started 7:24 PM

  4. How it all started 7:30 PM

  5. How it all started 7:26 PM

  6. How it all started 7:33 PM

  7. How it all started

  8. Quick risk assessment Lucid: [00:35:27] [email protected]:~# openssl version OpenSSL 0.9.8k

    25 Mar 2009 ! Precise: [00:34:37] [email protected]:~# openssl version OpenSSL 1.0.1 14 Mar 2012

  9. Where’s Waldo OpenSSL 8000 EC2 Machines: - 99.9% of them

    puppetized - Candidates: - Balancers - SVN Servers - Appliances - ELBs - 3rd party AMIs - Unique little snowflakes
 (Jira, Crucible,…)

  10. Let the patching begin

  11. Rollout Australia: ! Con: - Spiders - Snakes ! Pro:

    - Ops is awake

  12. Rollout

  13. Scan www

  14. Waiting on ELBs…

  15. Internal Certificates

  16. Suddenly: “reverse” Heartbleed

  17. Act 2: Communication

  18. Internal • Pre-determined chat rooms • Dial-in conference bridges •

    A communication plan Thanks SSAE-16, PCI and FedRAMP… I guess :)

  19. Statuspage + Twitter * Powered by StatusPage.io *

  20. Documentation https://docs.acquia.com/articles/heartbleed-acquia-cloud

  21. Proactive communication Phone calls by Acquia support, TAMs, …

  22. Since then: Post mortem

  23. Since then: Incident Commander (shamelessly stolen from Heroku) http://en.wikipedia.org/wiki/Incident_command_system

  24. Since then: Dedicated resource to vet security threats

  25. Since then: Clean up intranet docs

  26. Since then: Additional tooling

  27. We’re hiring (shameless self promotion) bit.ly/acquiajobs