Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Heartbleed at Acquia

Avatar for Marc Seeger Marc Seeger
May 20, 2014
Technology
16k
0

Heartbleed at Acquia

A quick presentation on how we handled Heartbleed at Acquia. Held at a DevOps Boston meetup.

Avatar for Marc Seeger

Marc Seeger

May 20, 2014

More Decks by Marc Seeger

See All by Marc Seeger
Security in DECT
Avatar for Marc Seeger rb2k
2
210
The DIRAC video codec
Avatar for Marc Seeger rb2k
1
100
Communitygetriebene Android Systemerweiterungen
Avatar for Marc Seeger rb2k
1
64
Alternative infrastructure
Avatar for Marc Seeger rb2k
1
200
NoSQL Lunch and Learn
Avatar for Marc Seeger rb2k
9
8.6k
Lunch and Learn: Cucumber and Capybara
Avatar for Marc Seeger rb2k
7
22k

Other Decks in Technology

See All in Technology
【５分でわかる】セーフィー エンジニア向け会社紹介
Avatar for Safie safie_recruit
0
55k
みてねにおけるAI-DLC導入活動とAIドリブン開発の現在地/JAWS-UG AI-DLC #2
Avatar for Isao Shimizu isaoshimizu
2
330
[RSJ26] Building a VLA Model Based on Self-Distilled Classification
Avatar for Semantic Machine Intelligence Lab., Keio Univ. keio_smilab
PRO
0
170
Genie Code ワークショップ 応用編 / Genie-Code-Workshop-advanced
Avatar for Databricks Japan databricksjapan
PRO
0
270
APIセキュリティを組織で実現するには~注力する点と設計・実装に入れたい対策~
Avatar for RiiiM(りむ) riiimparm
3
900
Genie Code ワークショップ 基礎編 / Genie-Code-Workshop-fundamental
Avatar for Databricks Japan databricksjapan
PRO
0
310
Bet AI Day 2026丨バクラク Autopilot、業務システムの再設計
Avatar for LayerX layerx
PRO
1
910
Kiro Crewしか勝たん！？
Avatar for Kazuki Miura miu_crescent
PRO
0
160
データエンジニアリングワークショップ：Auto LoaderとSparkで学ぶデータパイプライン構築
Avatar for Databricks Japan databricksjapan
PRO
0
110
Nav2、Nav3 ... はたまた自作？〜 作って理解する Nav3 の設計意図 〜 / Nav2, Nav3 ... or Build Your Own? — Understanding Nav3's design intent by building it from scratch
Avatar for Yuki Anzai yanzm
0
180
AI-DLCって実際どう？ 〜聞きたいこと全部聞いてみる〜
Avatar for 荒木 ARK news_it_enj
0
180
Amazon S3 Tablesに全部任せてみた結果——コンパクション/スナップショット管理は本当に手放せるか
Avatar for ShigeruOda shigeruoda
0
200

Featured

See All Featured
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.7k
Unlocking the hidden potential of vector embeddings in international SEO
Avatar for Frank van Dijk frankvandijk
0
910
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.9k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
3k
Designing for Timeless Needs
Avatar for Cassini Nazir cassininazir
1
470
My Coaching Mixtape
Avatar for mlcsv mlcsv
0
300
Leo the Paperboy
Avatar for Maya Tellez mayatellez
8
2.2k
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
249
1.3M
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
3k
How to build an LLM SEO readiness audit: a practical framework
Avatar for Nick Samuel nmsamuel
1
880
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k

Transcript

  1. Marc Seeger (@rb2k)
 Boston Devops Meetup
 May 20th 2014 at

  2. Act 1: Technology

  3. How it all started 7:24 PM

  4. How it all started 7:30 PM

  5. How it all started 7:26 PM

  6. How it all started 7:33 PM

  7. How it all started

  8. Quick risk assessment Lucid: [00:35:27] [email protected]:~# openssl version OpenSSL 0.9.8k

    25 Mar 2009 ! Precise: [00:34:37] [email protected]:~# openssl version OpenSSL 1.0.1 14 Mar 2012

  9. Where’s Waldo OpenSSL 8000 EC2 Machines: - 99.9% of them

    puppetized - Candidates: - Balancers - SVN Servers - Appliances - ELBs - 3rd party AMIs - Unique little snowflakes
 (Jira, Crucible,…)

  10. Let the patching begin

  11. Rollout Australia: ! Con: - Spiders - Snakes ! Pro:

    - Ops is awake

  12. Rollout

  13. Scan www

  14. Waiting on ELBs…

  15. Internal Certificates

  16. Suddenly: “reverse” Heartbleed

  17. Act 2: Communication

  18. Internal • Pre-determined chat rooms • Dial-in conference bridges •

    A communication plan Thanks SSAE-16, PCI and FedRAMP… I guess :)

  19. Statuspage + Twitter * Powered by StatusPage.io *

  20. Documentation https://docs.acquia.com/articles/heartbleed-acquia-cloud

  21. Proactive communication Phone calls by Acquia support, TAMs, …

  22. Since then: Post mortem

  23. Since then: Incident Commander (shamelessly stolen from Heroku) http://en.wikipedia.org/wiki/Incident_command_system

  24. Since then: Dedicated resource to vet security threats

  25. Since then: Clean up intranet docs

  26. Since then: Additional tooling

  27. We’re hiring (shameless self promotion) bit.ly/acquiajobs