How can you achieve web-scale datacenter automation?

CeBIT keynote 2016, see also http://www.dcdconverged.com/conferences/cebit-2016/speakers/michael-hausenblas and demo at https://www.youtube.com/watch?v=u4fEeIrLJZ4

Transcript

1. HOW CAN YOU ACHIEVE WEB-SCALE DATA CENTER AUTOMATION? Michael Hausenblas, Mesosphere Inc
2. © 2016 Mesosphere, Inc. All Rights Reserved. HOW CAN YOU ACHIEVE WEB- SCALE DATA CENTER AUTOMATION? 2 Michael Hausenblas, Developer & Cloud Advocate | 2016-03-15 | CeBIT
3. © 2016 Mesosphere, Inc. All Rights Reserved. SDDC CHALLENGES 3
4. © 2016 Mesosphere, Inc. All Rights Reserved. SDDC CHALLENGES 4
5. © 2016 Mesosphere, Inc. All Rights Reserved. SDDC CHALLENGES 5
6. © 2016 Mesosphere, Inc. All Rights Reserved. SDDC CHALLENGES 6
7. © 2016 Mesosphere, Inc. All Rights Reserved. SDDC CHALLENGES 7 • Software is eating the world, need to release features faster (and more of them) than your competitor • Mobile devices, constraint devices, IoT • Distributed applications
8. © 2016 Mesosphere, Inc. All Rights Reserved. IT'S TIME FOR A NEW SORT OF OPERATING SYSTEM 8
9. © 2016 Mesosphere, Inc. All Rights Reserved. AN OPERATING SYSTEM … 9 … provides abstraction of resources … exposes common services … enables faster app development
10. © 2016 Mesosphere, Inc. All Rights Reserved. RESOURCE ABSTRACTION 10 counter = 1
11. © 2016 Mesosphere, Inc. All Rights Reserved. COMMON SERVICES 11 • memory management • job scheduling • filesystem • networking stack • access control • user management • package manager
12. © 2016 Mesosphere, Inc. All Rights Reserved. COMMON SERVICES 12 int socket(int domain, int type, int protocol); int bind(int sockfd, struct sockaddr *my_addr, int addrlen); int connect(int sockfd, struct sockaddr *serv_addr, int addrlen); int listen(int sockfd, int backlog); ping netstat ethtool traceroute ifconfig
13. © 2016 Mesosphere, Inc. All Rights Reserved. MEET THE DATACENTER
 OPERATING
 SYSTEM
 13
14. © 2016 Mesosphere, Inc. All Rights Reserved. A LITTLE HISTORY 14 PHYSICAL (x86) VIRTUAL HYPERSCALE MAINFRAME SERVER VIRTUAL MACHINE PARTITION (LPAR) UNIT OF INTERACTION " ERP, CRM, PRODUCTIVITY, MAIL & WEB SERVER " LINUX, WINDOWS " DATA / TRANSACTION PROCESSING " UNIX, IBM OS/360 DEFINITIVE APPS AND OS " ERP, CRM, PRODUCTIVITY, MAIL & WEB SERVER " HYPERVISOR + GUEST OS " BIG DATA, INTERNET OF THINGS, MOBILE APPS " ??? ??? DATACENTER " BIG DATA, INTERNET OF THINGS, MOBILE APPS " DATACENTER OPERATING SYSTEM (DCOS)
15. © 2016 Mesosphere, Inc. All Rights Reserved. TRADITIONAL VS. DCOS APPROACH 15 • Many “snowflakes” • Management nightmare • Lengthy cycles to deploy code • Low utilization Mesosphere DCOS DCOS APPROACH Container
 Apps
 (All) Big Data Analytics 
 (All) PaaS 
 (All) Stateful Service
 (All) • High performance and resource isolation • Easy scalability and multi-tenancy • Fault tolerant and highly available • Highly efficient with highest utilization proven in mission- critical deployments on-prem or in the cloud TRADITIONAL APPROACH PaaS 
 1 Container
 App 
 1 Big Data Analytics 1 Big Data Analytics 2 PaaS 
 2 Container
 App 
 2 Stateful Service
 1 Stateful Service
 2 workload portability
16. © 2016 Mesosphere, Inc. All Rights Reserved. 16
17. © 2016 Mesosphere, Inc. All Rights Reserved. BENEFITS 17 DCOS • Run stateless services (Web server, app server, etc.) and Big Data services like Kafka, Spark, or Cassandra together on one cluster • Dynamic partitioning of your cluster, depending on your business requirements • Increased utilization (10% → 80%++)
18. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER OPERATIONS 18
19. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER OPERATIONS 19 • CI/CD (delivery, versioning, smoke tests) • container orchestration (failover, support for multiple workload types, stateful services) • troubleshooting (monitoring/logging/debugging/ tracing) • high utilization • security (isolation, authz/authn)
20. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER LIFE CYCLE 20
21. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER ORCHESTRATION 21
22. © 2016 Mesosphere, Inc. All Rights Reserved. WHAT DOES A SCHEDULER DO? 22
23. © 2016 Mesosphere, Inc. All Rights Reserved. DEMO 23 https://www.youtube.com/watch?v=u4fEeIrLJZ4
24. © 2016 Mesosphere, Inc. All Rights Reserved. TAKE- AWAYS 24 • To tackle SDDC challenges, a new approach is necessary: a distributed operating system • Portable workloads across environments
 (on-premises/hybrid/public cloud) are possible • Container operations is becoming mainstream
25. © 2016 Mesosphere, Inc. All Rights Reserved. Q & A 25 • @mhausenblas • mhausenblas.info • @mesosphere • mesosphere.com