Scaling with Heat

Horizontal and Vertical Scaling using OpenStack Heat (and Ceilometer)
Authors: Jordan Ayala, Nabil Chamas, Guillermo Peralta

Transcript

1. OpenStack Heat Horizontal and Vertical Scaling
2. Agenda - OpenStack - Overview - Services and Components - DevStack - Installation - Heat - Overview - Installation - Scaling - Hands-On 2
3. OpenStack OpenStack is a set of software tools for building and managing cloud computing platforms for public and private clouds. Backed by some of the biggest companies in software development and hosting. Many think that OpenStack is the future of cloud computing. OpenStack is managed by the OpenStack Foundation, a non-profit that oversees both development and community-building around the project. It’s Open Source. 3
4. Openstack vs Devstack vs Packstack Devstack is a tool for deploying Openstack on Ubuntu. Packstack is a tool for deploying Openstack on CentOS, Fedora. Openstack is cloud software which may be packaged in different ways ( RH vs Debian) and also installed in different ways. In particular, via the most stable puppet technology-utility packstack ( RH approach ). 4
6. Nova Is the primary computing engine behind OpenStack. It is used for deploying and managing large numbers of virtual machines and other instances to handle computing tasks. 6
7. Swift Is a storage system for objects and files. Rather than the traditional idea of a referring to files by their location on a disk drive, developers can instead refer to a unique identifier referring to the file or piece of information and let OpenStack decide where to store this information. This makes scaling easy. 7
8. Cinder Is a block storage component, which is more analogous to the traditional notion of a computer being able to access specific locations on a disk drive. This more traditional way of accessing files might be important in scenarios in which data access speed is the most important consideration. 8
9. Neutron Provides the networking capability for OpenStack. It helps to ensure that each of the components of an OpenStack deployment can communicate with one another quickly and efficiently. 9
10. Horizon Is the dashboard behind OpenStack. It is the only graphical interface to OpenStack, so for users wanting to give OpenStack a try, this may be the first component they actually “see.” Developers can access all of the components of OpenStack individually through an application programming interface (API), but the dashboard provides system administrators a look at what is going on in the cloud, and to manage it as needed. 10
11. Keystone Provides identity services for OpenStack. It is essentially a central list of all of the users of the OpenStack cloud, mapped against all of the services provided by the cloud, which they have permission to use. It provides multiple means of access, meaning developers can easily map their existing user access methods against Keystone. 11
12. Glance Provides image services to OpenStack. In this case, "images" refers to images (or virtual copies) of hard disks. Glance allows these images to be used as templates when deploying new virtual machine instances. 12
13. Ceilometer Provides telemetry services, which allow the cloud to provide billing services to individual users of the cloud. It also keeps a verifiable count of each user’s system usage of each of the various components of an OpenStack cloud. Think metering and usage reporting. 13
14. Heat Is the orchestration component of OpenStack, which allows developers to store the requirements of a cloud application in a file that defines what resources are necessary for that application. In this way, it helps to manage the infrastructure needed for a cloud service to run. 14
15. Agenda - OpenStack - Overview - Services and Components - DevStack - Overview - Installation - Heat - Overview - Installation - Scaling - Hands-On 15
16. What is DevStack? DevStack is a script to quickly create an OpenStack development environment. It can also be used to demonstrate starting/running OpenStack services and provide examples of using them from a command line. 16
17. Installing DevStack (1) 1. Install Linux a. Ubuntu 14.04/16.04, Fedora 23/24, CentOS/RHEL 7, as well as Debian and OpenSUSE. 2. Download DevStack $ git clone https://git.openstack.org/openstack-dev/devstack 3. Create local.conf $ cp devstack/sample/local.conf devstack 4. Add Stack User $ devstack/tools/create-stack-user.sh; su stack 17
18. Installing DevStack (2) 5. Start the install $ ./devstack/stack.sh a. Grab some popcorn (it’s gonna take a while) 6. ???? 7. PROFIT! a. Now you have a working DevStack. Go ahead and celebrate. 18
19. Agenda - OpenStack - Overview - Services and Components - DevStack - Overview - Installation - Heat - Overview - Installation - Scaling - Hands-On 19
20. OpenStack Heat Heat is the main project in the OpenStack Orchestration program. It’s a service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds. An orchestration engine to launch multiple composite cloud applications, based on human-readable templates. 20
21. Why ‘Heat’? It makes the clouds rise! 21
22. How it works Infrastructure is described in Heat templates. Resources such as servers, floating IPs, volumes, user, etc, can be described in a template. Relationships between resources can also be specified within templates. To change your infrastructure, simply modify the template with your editor of choice (vim, emacs, nano, etc). Horizontal and vertical scaling can be achieved using a Scaling Groups and Policies (with some help of our dear friend Ceilometer). 22
23. Architecture heat: CLI tool, communicates with heat-api. heat-api: provides an OpenStack-native REST API that processes API requests by sending them to the heat-engine over RPC. heat-api-cfn: AWS-style Query API that is compatible with AWS CloudFormation. heat-engine: does the main work of orchestrating the launch of templates and providing events back to the API consumer. 23
24. Installing Heat Prerequisites: working DevStack installation. 1. Modify local.conf 2. That’s it, now run DevStack $ devstack/stack.sh [[local|localrc]] #Enable heat services enable_service h-eng h-api h-api-cfn h-api-cw IMAGE_URL_SITE="http://download.fedoraproject.org" IMAGE_URL_PATH="/pub/fedora/linux/releases/21/Cloud/Images/x86_64/" IMAGE_URL_FILE="Fedora-Cloud-Base-20141203-21.x86_64.qcow2" IMAGE_URLS+=","$IMAGE_URL_SITE$IMAGE_URL_PATH$IMAGE_URL_FILE 24
25. Heat Orchestration Template (HOT) HOT is a template format supported by the heat, along with the other template format, i.e. the Heat CloudFormation-compatible format (CFN). 25 heat_template_version: 2016-10-14 description: # a description of the template parameter_groups: # a declaration of input parameter groups and order parameters: # declaration of input parameters resources: # declaration of template resources outputs: # declaration of output parameters conditions: # declaration of conditions
26. AutoScalingGroup OS::Heat::AutoScalingGroup An autoscaling group that can scale arbitrary resources. Allows the creating of a desired count of similar resources. • max_size: maximum number of resources in the group. • min_size: minimum number of resources in the group. • resource: resources in the group. 26
27. ScalingPolicy OS::Heat::ScalingPolicy A resource to manage scaling of AutoScalingGroup, defining which metric should be scaled and by how much. • adjustment_type: type of adjustment. • auto_scaling_group_id: AutoScaling group to apply policy to. • scaling_adjustment: size of adjustment. 27
28. heat_template_version: 2014-10-16 description: A simple auto scaling group. resources: servers: type: OS::Heat::AutoScalingGroup properties: cooldown: 60 max_size: 5 min_size: 1 resource: type: OS::Nova::Server::Cirros scaleup_policy: type: OS::Heat::ScalingPolicy properties: adjustment_type: change_in_capacity auto_scaling_group_id: { get_resource: servers } cooldown: 60 scaling_adjustment: 1 cpu_alarm_high: type: OS::Ceilometer::Alarm properties: meter_name: cpu_util statistic: avg period: 60 evaluation_periods: 1 threshold: 50 alarm_actions: - {get_attr: [scaleup_policy, alarm_url]} comparison_operator: gt 28 AutoScaling Servers
29. parameters: flavor: type: string label: Flavor description: Type of instance to be used. default: m1.small cluster_size: type: number label: Cluster size description: Number of instances in cluster. default: 2 $ heat stack-update tiny -f heat_4b.yaml -e lib/env.yaml -P "cluster_size=5;flavor=m1.large" 29 AutoScaling Servers resources: tiny_cluster: type: OS::Heat::ResourceGroup properties: count: { get_param: cluster_size } resource_def: type: Lib::MSG::Tiny properties: image: { get_param: image } flavor: { get_param: flavor } key: { get_param: key } private_network: { get_param: private_network }
30. References - DevStack installation: - http://docs.openstack.org/developer/devstack/ - OpenStack Resource Types: - http://docs.openstack.org/developer/heat/template_guide/openstack.html - Heat Templates: - https://github.com/openstack/heat-templates 30