Slide 1

Slide 1 text

MAKING CLOUDY PEANUT BUTTER CUPS: APACHE CLOUDSTACK + RIAK CS John Burwell ([email protected] | @john_burwell) Thursday, June 27, 13

Slide 2

Slide 2 text

Who Am I • Consulting Engineer @ Basho Technologies • Apache CloudStack PMC Member • Ran operations and designed automated provisioning for hybrid analytic/virtualization clouds • Led architectural design and server-side development of a SaaS physical security platform Thursday, June 27, 13

Slide 3

Slide 3 text

Thursday, June 27, 13

Slide 4

Slide 4 text

Hopefully not ... Thursday, June 27, 13

Slide 5

Slide 5 text

Thursday, June 27, 13

Slide 6

Slide 6 text

Thursday, June 27, 13

Slide 7

Slide 7 text

Thursday, June 27, 13

Slide 8

Slide 8 text

Composable Infrastructure • Resource Pooling • Rapid Elasticity • Self Service • Metering/Charge Back Thursday, June 27, 13

Slide 9

Slide 9 text

Doesn’t vSphere/ XCP/XenServer already do that? Thursday, June 27, 13

Slide 10

Slide 10 text

Traditional Workload Management Network Hypervisor Cluster Hypervisor Cluster Shared

Slide 11

Slide 11 text

Cloud Workload • Reliable • Fault Tolerant • Commodity Hardware • Self-Service provisioning • Dynamic scale out model SDN Rack Rack Object

Slide 12

Slide 12 text

Traditional Workload Cloud Workload The Chicken Or The Egg? Thursday, June 27, 13

Slide 13

Slide 13 text

Got It ... Why Go Private? • Leverage existing infrastructure investment • Customization and Control • Regulatory/Security Requirements • Cost Thursday, June 27, 13

Slide 14

Slide 14 text

Open Source Private Cloud + Compute Services Object Storage Services Thursday, June 27, 13

Slide 15

Slide 15 text

Make infrastructure boring. Thursday, June 27, 13

Slide 16

Slide 16 text

What Is Apache CloudStack? §Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. §CloudStack is a Top-Level Project at the Apache Software Foundation. Thursday, June 27, 13

Slide 17

Slide 17 text

Apache CloudStack Users Thursday, June 27, 13

Slide 18

Slide 18 text

Infrastructure Model • Management Server Farm • Management and provisioning tasks • Zone • Collection of pods, network offerings and secondary storage • Pod • Collection of clusters in the same failure boundary • Cluster • A grouping of hosts and their associated storage • Hosts • Servers onto which services will be provisioned • Primary Storage • VM disk storage • Network • Logical network associated with service offerings • Secondary Storage • Template, snapshot and ISO storage Zone CloudStack Pod Cluster Host Host Network Primary Storage VM VM CloudStack Pod Cluster Secondary Storage Thursday, June 27, 13

Slide 19

Slide 19 text

Management Server Farm • MS is stateless. MS can be deployed as physical server or VM • Single MS node can manage up to 10K hosts. Multiple nodes can be deployed for scale or redundancy • RHEL/CentOS (5.4+), Ubuntu (10.0.4+), Fedora (16+) User

Slide 20

Slide 20 text

Got Scale? • Running in production at >30,000 physical hosts, supported by only 4 management server instances • Even greater scale by using CloudStack in a Regional model • Remember: The separation of control, management and data planes is critical for cloud platform scale. Thursday, June 27, 13

Slide 21

Slide 21 text

Deployment Architecture • Hypervisor is the basic unit of scale. • Cluster consists of one ore more hosts of same hypervisor • All hosts in cluster have access to shared (primary) storage • Pod is one or more clusters, usually with L2 switches. • Availability Zone has one or more pods, has access to secondary storage. • One or more zones represent a cloud                                                                                                           Pod  1 …. Cluster  N L2 Host 2 Cluster  1 Host 1 Primary Storage Zone  1 …. L3 Secondary Storage Pod  N Management Server Farm Internet Thursday, June 27, 13

Slide 22

Slide 22 text

Hypervisors • Xen / XCP (Ubuntu 12.04.2) • XenServer (6.1) • KVM (Ubuntu 12.04.2 / CentOS 6) • VMWare vCenter (4.1 / 5.1) Thursday, June 27, 13

Slide 23

Slide 23 text

Storage • ISCSI • NFS • Local Filesystem Storage • Ceph • S3-backed Secondary Storage • Swift-backed Secondary Storage Thursday, June 27, 13

Slide 24

Slide 24 text

Networking • VLAN and Security Group isolation • Load Balancer Support • Juniper SRX, F5, Nicira NVP and Open vSwitch integration Thursday, June 27, 13

Slide 25

Slide 25 text

4.1: Hot Off The Presses • AWS style regions • Event pub-sub framework (RabbitMQ implementation) • Advanced search within the UI • API Server request throttling • API Discoverer Service • Self-Service Password Reset • Users directly changing their API keys • EC2 query API • Cloudmonkey CLI • Autoscale (F5-specific) Much, much more .... Thursday, June 27, 13

Slide 26

Slide 26 text

4.2: On-Deck • Midonet and BigSwitch SDN integrations • Enhanced Baremetal provisioning • VM I/O Throttling • Hyper-V 2012 Support • LXC Support • Cisco VSG integration • Cisco ASA 1000V • VM Affinity Rules • VM Affinity Rules • Zone-wide primary storage • Security group isolation in Advanced Network zones • Dedicating resources to domains and accounts • IP Address reservation w/o a vNIC allocation • vSphere DVS support • UI Plugin framework Thursday, June 27, 13

Slide 27

Slide 27 text

First A Little Riak • Dynamo-inspired, masterless key-value store • Apache License • Always available for writes • Highly available reads • Secondary Indexes/Search • MapReduce • Operations Friendly Thursday, June 27, 13

Slide 28

Slide 28 text

... Then A Little Riak CS • Built on top Riak • Apache Licensed • S3-compatible API • Large object support and multipart upload • Multi-tenancy and per-user reporting • Operations Friendly Thursday, June 27, 13

Slide 29

Slide 29 text

How It Works Large

Slide 30

Slide 30 text

Riak CS Roadmap • Keystone integration • OpenStack Storage API v1 (aka Swift) support • Performance improvements around large bucket operations Thursday, June 27, 13

Slide 31

Slide 31 text

The Complete Private Cloud Management Network Hypervisor Cluster Hypervisor Cluster Shared

Slide 32

Slide 32 text

Apache CloudStack + Riak CS Roadmap • Now • S3-backed Secondary Storage • 4.2 • Partial support for using object stores as secondary storage • 4.3 • Integrated authentication • Complete support for using object stores as secondary storage Thursday, June 27, 13

Slide 33

Slide 33 text

Join Our Community • Learn more @ http://cloudstack.apache.org • Grab it @ http://cloudstack.apache.org/downloads.html • Follow us @cloudstack • Help from the [email protected] mailing list or on Freenode @ #cloudstack • Contribute on [email protected] mailing list or on Freenode @ #cloudstack-dev Thursday, June 27, 13

Slide 34

Slide 34 text

Up And Running With Riak CS • Learn more @ http://basho.com/riak-cloud-storage • Grab a distribution @ http://docs.basho.com/riakcs/latest/ riakcs-downloads/ • Follow us @basho • Help from the [email protected] mailing list or on FreeNode @ #riak • Contribute @ http://github.com/basho/riak_cs Thursday, June 27, 13

Slide 35

Slide 35 text

Thoughts? Questions? Thursday, June 27, 13

Slide 36

Slide 36 text

Thank you! Slides available @ http://speakerdeck.com/jburwell Thursday, June 27, 13