BETTER, FASTER, CHEAPER INFRASTRUCTURE: APACHE CLOUDSTACK + RIAK CS John Burwell ([email protected] | @john_burwell) 

Who Am I • Senior Software Engineer @ Basho Technologies • Member, Apache CloudStack PMC • Ran operations and designed automated provisioning for hybrid analytic/virtualization cloud infrastructures • Led architectural design and server-side development of a SaaS physical security platform 

... but I submitted a ticket. 

Better adjust to shifting requirements Deliver resources and recover from failure faster Utilize cheaper commodity components without compromising reliability 

1administrator per 10000 servers 

Composable Infrastructure • Resource Pooling • Rapid Elasticity • Self Service Automated late binding of responsibilities to hardware 

Doesn’t vSphere/ XCP/XenServer already do that? 

Software Is Eating Hardware • Moore’s Law has pulled fault tolerance up the stack • Commodity Hardware • Reduced cost • Services wider range of workloads • Dynamic resource allocation Specialized Hardware Reduces Agility 

Traditional Workload Management Network Hypervisor

Cloud Workload • Reliable • Fault Tolerant • Commodity Hardware • Self-Service provisioning • Control Plane -> Dynamic scale out model SDN Rack Rack Object

Traditional Workload Cloud Workload Reality 

Got It ... Why Go Private? • Leverage existing infrastructure investment • Customization and Control • Regulatory/Security Requirements • Cost 

Complete Private Cloud + Compute Services Object Storage Services 

What Is Apache Cloudstack? §Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. ! §CloudStack is a Top-Level Project at the Apache Software Foundation. 

~300: Acknowledged production implementations ~43,000: Hosts in a single CloudStack instance ~100: Project committers 

Anatomy Of CloudStack • Management Server Farm • Management and provisioning tasks • Zone • Collection of pods, network offerings and secondary storage • Pod • Collection of clusters in the same failure boundary • Cluster • A grouping of hosts and their associated storage • Hosts • Servers onto which services will be provisioned • Primary Storage • VM disk storage • Network • Logical network associated with service offerings • Secondary Storage • Template, snapshot and ISO storage Zone CloudStack Pod Cluster ! Host ! Host Network Primary Storage VM VM CloudStack Pod Cluster Secondary Storage 

Primary Vs Secondary Storage • Random access read/write • Changes often • Zone Scoped • Focused on throughput • Write once/read many • Append only • Zone or Region Scoped • Focused on reliability Secondary Storage is the compute lifeboat. 

Secondary Storage Vm • Required for secondary storage operation • Separates data transfer operations from the control plane • Transfers data and manipulates secondary storage as directed by the management server • Dynamically scaled by the management server • An SSVM template is required and its installation is an additional step 

What Is Object Storage? • Stores large, immutable, chunks of unstructured data referenced through a unique key • Objects live in a unique global namespace (e.g. bucket) • Simple interface that provides uniform access to distributed storage • Typically provide redundancy and replication in software allowing the use commodity JBOD storage 

What Is Riak CS? • Simple, reliable storage of large objects accessed using the S3 or Swift API • Multi-tenant security model with per object ACLs • Usage Accounting and Reporting • Open Source (Apache2 Licensed) Your Own Private S3 

Anatomy Of Riak CS • S3/Swift API Proxy + Riak key-value store • Peer-to-Peer Design: Any node can service any read or accept any write request • Maintains multiple replicas (default 3) spread across multiple nodes • Ops Friendly Horizontally scalable storage on commodity hardware 

Riak CS Use Cases • Secondary Storage for VM snapshot and template data • Backup Storage • Serving Static Content • Application Log Storage 

Using Riak Cs With CloudStack • Create a user account and bucket for CloudStack — the user’s ACLs should be restricted to the CloudStack bucket • In CloudStack, define an S3 secondary storage resource using the Riak CS cluster’s end point, the CloudStack account’s credentials, and the CloudStack bucket name • Create an NFS share to stage files sized to meet peak snapshot and VM creation load. This share is scratch space to support hypervisors that can not directly stream objects. 

Achieving Commodity Reliability • Create a zone per power source/backbone network connection • Establish independent primary storage in each zone with block- level replication between zones • Utilize Riak CS for secondary storage — evenly distributing nodes across zones • Distribute application instances across all zones 

A Complete Private Cloud Management Network Hypervisor

Up And Running With Riak CS • Learn more @ http://basho.com/riak-cloud-storage • Grab it @ http://docs.basho.com/riakcs/latest/riakcs- downloads/ • Follow us @basho • Help from the [email protected] mailing list or on FreeNode @ #riak • Contribute @ http://github.com/basho/riak_cs 

Join Our Community • Learn more @ http://cloudstack.apache.org • Grab it @ http://cloudstack.apache.org/downloads.html • Follow us @cloudstack • Help from the [email protected] mailing list or on Freenode @ #cloudstack • Contribute on the [email protected] mailing list or on Freenode @ #cloudstack-dev 

Thoughts? Questions? 

Thank You Deck http://slideshare.net/JohnBurwell1 Follow @ john_burwell Email [email protected]