Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Securing Containers the Netflix Way

Bryan Payne
January 25, 2017

Securing Containers the Netflix Way

This talk was presented at the Container Security Summit 2017.

Security has been a first principle at Netflix as we created our internal container service, known as Titus. Along the way we learned many places where our security needs aligned with others in the community. But, perhaps most interestingly, we also learned about where we differed. This talk will take you on a tour of how we have handled container security at Netflix from threat modeling to container isolation and identity. You’ll see how we built containers to integrate seamlessly into our development and production ecosystem. Finally, we discuss what we see as the open security challenges for large-scale container deployments.

Bryan Payne

January 25, 2017

More Decks by Bryan Payne

Other Decks in Programming


  1. Netflix by the Numbers • 93M+ members • 1000+ developers

    • 190+ countries • > 1/3 NA internet download traffic • 500+ microservices • 100,000+ virtual machines • 3+ AWS regions
  2. Why Containers? Velocity! • Improved developer experience • Improved system

    resource management • Improved deployment time Speeding Up Deployments Measuring CPU usage once a minute makes no sense for containers… Coping with rate of change is a big challenge for monitoring tools. Datacenter Snowflakes • Deploy in months • Live for years Virtualized and Cloud • Deploy in minutes • Live for weeks Container Deployments • Deploy in seconds • Live for minutes/hours AWS Lambda Events • Respond in milliseconds • Live for seconds Monitoring Microservice and Containers: A Challenge by Adrian Cockcroft @ Gluecon 2015
  3. Container Use Cases @ Netflix • Media Encoding - R&D

    Cycle • Using VMs: 1 month • Using Containers: 1 week • Continuous Integration • Build all Netflix codebase in hours • Saves countless hours of debugging
  4. Services Add Complexity • Resizing & Lifecycle Duration • Autoscaling

    • Harder to upgrade underlying host • Have More State • Want to use existing tool chains
  5. Netflix Culture —> Netflix Security Values are what we Value

    High Performance Freedom & Responsibility Context, not Control Highly Aligned, Loosely Coupled http://www.slideshare.net/reed2001/culture-1798664
  6. Scumblr https://github.com/Netflix/Scumblr • Sync from data sources • Github •

    Route53 DNS • Manual upload • Analysis on these results • Search • Curl • Static analysis
  7. Stethoscope • Help employees stay safe with BYOD • Education

    • Self service • Personalized • Actionable • User Focused Security • No forced updates • No company wide emails • No information overload
  8. Titus Security = Titus Security =Container Security Understanding and Hardening

    Linux Containers by Aaron Grattafiori (NCC Group) https://www.nccgroup.trust/us/our-research/understanding-and-hardening-linux-containers/
  9. Titus Security = Titus Security = Container App Security API

    Security Identity AWS Security Monitoring Registry Security Control Plane Security
  10. AWS Security • Metadata proxy service • IAM roles per

    container • Security groups per container • Dedicated AWS account • Limit IAM perms for host
  11. API Security • mTLS for all API access • Security

    assessment for all components • Threat modeling • AppSec assessment
  12. Monitoring • Insight for security patches • Check human access

    using Stethoscope • Forensic audit trail
  13. Control Plane Security • TLS + authN for all UI

    access • Container isolation • Namespaces • Capabilities • MAC • Etc • General host OS hardening