Exoscale, an inside view of a (Swiss) cloud provider

Transcript

1. EXOSCALE, AN INSIDE VIEW OF A SWISS CLOUD PROVIDER
2. #WHOAMI LOIC LAMBIEL Head of Operations @loiclambiel loic.lambiel@exoscale.ch
3. Simple and efficent cloud hosting platform Perfect for devs, sysadmins & SaaS companies Full compatibility with automation tools Hosted in a safe jurisdiction
4. CLOUD BUILDING BLOCKS Services Infrastructure So ware People
5. SERVICES Compute (IaaS) Object Storage (S3 compatible) DNS Status pages More to come !
6. DATACENTERS LOCATIONS Geneva (Equinix GV2) Attinghausen, Uri (Deltalis DK2)
8. SERVICE: COMPUTE Apache Cloudstack based Basic networking Local storage Linux KVM hypervisor: Smart OS inspired
9. CLOUDSTACK Great extensibility, easy to plug into. Maintaining our own fork
10. NETWORKING One public IP per VM Security groups Security groups are hypervisor controlled layer 2 & 3 firewall rules. Provides all the flexibility of a traditionnal firewall, completely API controlled.
11. LOCAL STORAGE No SAN ! FastI/O, persistent disks All SSD, RAID10
12. LINUX KVM HYPERVISOR Bestin class hypervisor. Diskless and netboot approach. Avoids resource waste, facilitates upgrades
13. SERVICE: OBJECT STORAGE S3 Api: Pithos, our in house frontend (OSS) Backed by Apache Cassandra for data storage
14. INFRASTRUCTURE DOG FOODING A Significant part of our infrastructure components are actually self hosted, Exoscale being one of it's own customer.
15. INFRASTRUCTURE THE GOOD CITIZEN CONTRACT Configuration management Visibility Build factory Remote execution
16. THE GOOD CITIZEN CONTRACT new machines have roles role defines converged configuration as sum of components each component has an expected normal state and reports it no local intervention needed
17. CONFIGURATION MANAGEMENT: GIT code is a greatway to define infrastructure ensures homogeneity ability to iterate fast great source of change tracking avoids fear of change
18. CONFIGURATION MANAGEMENT: PUPPET battle tested tool simple declarative DSL to express configuration fits our component approach well
19. VISIBILITY FROM THE MAP TO THE TERRITORY logs Metrics Alerts Dashboards
20. WHY FOCUS ON VISIBILITY distributed systems with lots of moving parts, high node volatility
21. LOGS All application and system logs sent over the wire Logstash disects and extracts metadata Elasticsearch indexes for easy retrieval Simple correlation
22. METRICS All application and system metrics sent over the wire by collectd Cyanite aggregates and produces appropriate rollups Cyanite is a time series system backed by Cassandra and compatible with Graphite
23. METRICS if itmoves, graph it. if it doesn't, graph it in case it starts moving
24. ALERTS Unbounded stream of log and metric data Passive approach bodes wellwith node volatility Riemann takes decisions based on stream content Ability to extract meaningful information
25. DASHBOARDS Extensive use of TV dashboards for metrics display May allow an early spot of an incubating issue
26. BUILD FACTORY Continuous integration Package repositories
27. CONTINUOUS INTEGRATION Over 100 build jobs Ties into our code hosting platform Handled by jenkins
28. PACKAGE REPOSITORIES Generates valid and signed Debian repositories Ensures fast upgrades Simplifies configuration management
29. REMOTE EXECUTION A simple pubsub system IRC and HTTP interaction Recurrent commands stored as scenarios
30. A SIMPLE PUBSUB SYSTEM each node runs an agent responsible for carrying out commands. commands are sent to groups of nodes (by predicates such as role).
31. IRC AND HTTP INTERACTION most of our production environment are controlled through our chatrooms
32. RECURRENT COMMANDS STORED AS SCENARIOS intricate workflows can be expressed through a simple DSL
33. SOFTWARE FILLING IN THE GAPS Customer management Real-time metering and billing Integrated console Afew other things
34. CUSTOMER MANAGEMENT Keeping track of our users Support services (ticket management, coupons, emails)
35. REAL-TIME METERING AND BILLING Can't be tied to a cloudstack only solution Cloudstack emits useful data Ties into our customer management Relying on Apache Kafka for all our stream processing
36. INTEGRATED CONSOLE Integrated experience across our services Build with simplicity in mind
38. A FEW OTHER THINGS We have a strong commitment to open source Pithos Cyanite Warp Collectd modules CS (Cloudstack python client) Are some of the Exoscale projects
39. A FEW OTHER THINGS We also love to contribute: Apache Cloudstack Collectd Riemann Apache Kafka Docker machine And many more.
40. PEOPLE EFFICIENT WORK. QUIET NIGHTS Small SRE team Avoiding deploy anxiety
41. SMALL SRE TEAM Our platform must be simple to operate, additional moving parts must provide business value or help operations
42. AVOIDING DEPLOY ANXIETY Our so ware and infrastructure helps ensure we have good tools to ensure quiet nights and easily caught errors
43. LINKS https://github.com/exoscale/ http://pithos.io/ http://cyanite.io/ (IRC command and control) https://github.com/pyr/warp http://riemann.io/ https://collectd.org/
44. THANKS !