Securing the DevOps Lifecycle

Traditional security techniques don't work well in the high-cycle times of Agile teams practicing DevOps methodologies. Here we explore ways in which application security can be built into modern development processes.

1. SECURING THE DEVOPS LIFECYCLE Jarret Raim, BSides Austin, April 2012
2. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! WHO AM I? Jarret Raim – Security Intrapreneur 2! DEVELOPER ACADEMIC APPLICATION SECURITY SECURITY CONSULTANT SECURITY PRODUCTS jarret.raim@rackspace.com!
3. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! RACKSPACE® HOSTING The Service Leader in Cloud Computing 3! 172,000+ CUSTOMERS 4,000+ RACKERS 9 GLOBAL DATA CENTERS 120 + COUNTRIES 2008, 2010, 2011 & 2012 LEADER IN GARTNER'S MAGIC QUADRANT FOR MANAGED HOSTING 40%   FORTUNE® 100 OF THE WE SERVE RAX
4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 4! OUR VISION To be recognized as one of the World’s greatest service companies. “ ”
5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! RAX CLOUD APPROACH Open source orchestration, management & provisioning cloud platform !
6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! Applications OPENSTACK™ Management Layer that Adds Automation and Control Creates Pools of Resources Automates The Network USERS ADMINS CLOUD OPERATING SYSTEM Connects to apps via APIs Self-service Portals for users Stage II Cloud Data Center Stage III Cloud Federation Stage I Server Virtualization
7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! §  Scalable Web Apps §  Test, Development, Production §  SaaS/Resellers/Web Developers Dedicated AND Cloud Two independent applications DEDICATED •  Customizable •  Compliance •  High Performance APP 1 CLOUD HOSTING •  Flexible •  Elastic •  Pay-as-you-go APP 2 HYBRID HOSTING RAX HYBRID CLOUD SOLUTIONS
8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! PROVIDER DC! CUSTOMER SITE! RACKSPACE LOCATIONS! 8! DEDICATED! PUBLIC CLOUD! PRIVATE CLOUD! PRIVATE CLOUD! PUBLIC CLOUD! Rackspace Provides! The Fanatical Support! •  One Control Panel across OpenStack connected clouds! •  One Fanatical Support Team! •  Our Cloud, Your Cloud, Partner Hosted OpenStack Cloud! •  Global Reach! THE FUTURE: FANATICAL SUPPORT ANYWHERE 8
9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 9! SECURING THE DEVOPS LIFECYCLE
10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 10! • Cycle time for software is getting shorter! • Continuous delivery is a goal! • Scanning windows are not viable! ! THE PROBLEM 10 "DevOps" is an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/ infrastructure) professionals.
11. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 11! • Automated software testing! • Automated operational infrastructure! • Automated security testing! THE SOLUTION 11
12. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 12! AUTOMATING 12 • Declarative configuration language! • Plain-text configuration in source control! • Fully programmatic, no manual interactions!
13. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 13! CHEF 13 1.  Solo! 2.  Server! 3.  Hosted! 4.  Private Hosted! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Node! Racker! Server / Hosted / Private!
14. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 14! COOKBOOKS 14 • Most major software packages have cookbooks! • You will have to write your own / customize! • Good place to spend security cycles! - Merge patches upstream for extra points.!
15. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 15! GROUPING & TAGGING 15 • Tagging your servers applies the required set of recipes! • A base set of recipes is common! • Each server will have multiple tags set at bootstrap time! Node! Node! Node! Node! DB! Node! Node! Node! Node! Cache! Node! Node! Node! Node! Web! Apache! Monitoring! MySql! Memcache!
16. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 16! LIMITATIONS 16 • Focus on single machines! • A multi-box configuration is based on copying existing configurations! • No support for implicit application or environment configuration! • Applications include more than just servers! • Images have security issues! Web! Web! Web! Cloud Load Balancer! Memcached! Database as a Service! Web! Cloud Files! CDN!
17. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 17! CHECKMATE 17 A system to build generic application configurations! Architect   • Templates   • Ques2ons   Contractor   • Decomposi2on   • Orchestra2on   Inspector   • Verifica2on   • Due  Diligence  
18. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 18! ARCHITECTURE 18 Message   Queue   Architect   Contractor   Inspector   Checkmate   Web   Message   Queue   Compute   Storage   Load   Balancer   Database   Hadoop   Caching   • Components communicate through a common queue! • Each provisioning component is independent!
19. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 19! ARCHITECT 19 Template! Generic Provider Definitions! Architecture Questions! Scaling Factors! base:      name:  wordpress  large      environment-­‐name:  {tenantId}-­‐ wordpress-­‐large     providers:      -­‐  rackspace:          -­‐  compute:  &rax-­‐cloud-­‐servers              endpoint:  https://...          -­‐  loadbalancer:  &rax-­‐lbaas              endpoint:  https://...          -­‐  database:  &rax-­‐dbaas              endpoint:  https://...          -­‐  common:          vendor:  rackspace          credentials:              -­‐  token:  {token}  
20. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 20! ARCHITECT 20 Template! Generic Provider Definitions! Architecture Questions! Scaling Factors! • Requests per hour?! • Budget! • High availability! • Disaster resistant! • SSL! • Backup! • CDN! …!
21. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 21! ARCHITECT 21 Template! Generic Provider Definitions! Architecture Questions! Scaling Factors! tiers:   -­‐  name:  web      resource:  &loadbalancer          min-­‐occur:  1          type:  loadbalancer          connection:  public              port:  [80,  443]              allow:  all              isolation:  none      resource:  &webheads          min-­‐occur:  2          type:  compute          os:  Ubuntu  11.10          memory-­‐min:  2Gb          memory-­‐max:  4Gb          configs:          -­‐  wordpress-­‐mp              attributes:              -­‐  role:  web              connection:  *database  
22. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 22! ARCHITECT 22
23. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 23! CONTRACTOR 23 • Takes Architect’s plan and builds it! • Task Decomposition! - Uses standard workflow patterns! • Orchestration / Ordering! • Status Reporting! • Farms out tasks to sub- contractors! Our current implementation uses an open source Python workflow engine, SpiffWorkflow.!
24. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 24! INSPECTOR 24 • Takes Architect’s plan & contractor’s output! • Focuses on checking for code compliance! - Not perfection, bare minimums! • Can include multiple facets! - Security! - Scalability! - Compliance! Our current implementation includes WP Scan for WordPress and the Nikto vulnerability scanner.!
25. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 25! INSPECTOR 25 +  Server:  Apache/2.2.12  (Ubuntu)   +  No  CGI  Directories  found  (use  '-­‐C  all'  to  force  check  all   possible  dirs)   +  Apache/2.2.12  appears  to  be  outdated  (current  is  at  least  Apache/ 2.2.17).  Apache  1.3.42  (final  release)  and  2.0.64  are  also  current.   +  ETag  header  found  on  server,  inode:  12534048,  size:  317,  mtime:   0x4b9436dbea280   +  Allowed  HTTP  Methods:  GET,  HEAD,  POST,  OPTIONS     +  OSVDB-­‐3268:  /icons/:  Directory  indexing  found.   +  OSVDB-­‐3233:  /icons/README:  Apache  default  file  found.   +  6448  items  checked:  0  error(s)  and  5  item(s)  reported    
26. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! 26! INSPECTOR 26 [!]  The  WordPress  "http://-­‐-­‐-­‐.com/readme.html"  file  exists.   [!]  WordPress  version  3.1  identified  from  meta  generator.     [+]  Enumerating  installed  plugins...Checking  for  2394  total   plugins   [+]  We  found  2  plugins:   Name:  disqus-­‐comment-­‐systemLocation:     Name:  wordpress-­‐popular-­‐postsLocation:       [+]  There  were  1  vulnerabilities  identified  from  the  plugin   names:     [!]  ["WordPress  Plugin  Disqus  Comment  System  <=  2.68  Reflected   Cross-­‐Site  Scripting  (XSS)"]*    
27. RACKSPACE® HOSTING | WWW.RACKSPACE.COM ! Architect   • Templates   • Ques2ons   Contractor   • Decomposi2on   • Orchestra2on   Inspector   • Verifica2on   • Due  Diligence   Monitor   • Trending   • Thresholding   27! FUTURE WORK 27
28. RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218! US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM! RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM! ANY QUESTIONS?