Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWS and Ansible | Andrew Kurinnyi
Search
Matt Chung
February 11, 2015
Technology
1
100
AWS and Ansible | Andrew Kurinnyi
AWS and Ansible integration using CloudFormation.
Matt Chung
February 11, 2015
Tweet
Share
More Decks by Matt Chung
See All by Matt Chung
Serverless Architecture with AWS Lambda
itsmemattchung
0
210
@mwhuge - Tools don't matter @OCDevOps
itsmemattchung
0
100
Scale13X DevOps for the enterprise | Fox | Matt Chung
itsmemattchung
0
71
DevOps Journey for Fox | Matt Chung
itsmemattchung
0
110
Other Decks in Technology
See All in Technology
Culture Deck
optfit
0
330
偶然 × 行動で人生の可能性を広げよう / Serendipity × Action: Discover Your Possibilities
ar_tama
1
740
30分でわかる『アジャイルデータモデリング』
hanon52_
9
2.2k
All you need to know about InnoDB Primary Keys
lefred
0
120
Ask! NIKKEIの運用基盤と改善に向けた取り組み / NIKKEI TECH TALK #30
kaitomajima
1
450
アジャイル開発とスクラム
araihara
0
160
個人開発から公式機能へ: PlaywrightとRailsをつなげた3年の軌跡
yusukeiwaki
11
2.7k
君はPostScriptなウィンドウシステム 「NeWS」をご存知か?/sunnews
koyhoge
0
720
室長と気ままに学ぶマイクロソフトのビジネスアプリケーションとビジネスプロセス
ryoheig0405
0
320
データ資産をシームレスに伝達するためのイベント駆動型アーキテクチャ
kakehashi
PRO
2
230
データ基盤の成長を加速させる:アイスタイルにおける挑戦と教訓
tsuda7
3
650
データの品質が低いと何が困るのか
kzykmyzw
6
1k
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
7k
Become a Pro
speakerdeck
PRO
26
5.1k
RailsConf 2023
tenderlove
29
1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.5k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Optimising Largest Contentful Paint
csswizardry
34
3.1k
Building Flexible Design Systems
yeseniaperezcruz
328
38k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Building an army of robots
kneath
302
45k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Code Reviewing Like a Champion
maltzj
521
39k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Transcript
AWS!infrastructure!with!Ansible! and!CloudForma2on
Pieces&of&a&puzzle • Provisioning*of*resources*(VPC,*EC2,*ELB,*S3) • Machine*configura>on*(Ansible)
Provisioning)*)CloudForma2on An#AWS&specific#tool#which#makes#it#easier#to#deploy'the' collec+on'of'AWS'resources#you#need#to#run#your#applica9on# repeatedly'and'predictably#using#a#declara+ve'template
Pros • Infrastructure,as,a,code • Reusable,templates • Transac5onal,updates • Extensible
Cons • AWS%only • Not%always%up%to%date%with%latest%features • Wri7ng%custom%resources%is%a%bit%of%a%pain • Large%JSON%templates%could%be%hard%to%maintain Alterna(ve:+Terraform.io
Managing&mul*ple&stacks&with&Ansible --- - hosts: localhost connection: local gather_facts: False vars:
project: cftest environment: production region: us-east-1 tasks: - name: Create VPC cloudformation: stack_name={{ environment }}-{{ project }}-vpc state=present region="{{ region }}" template=vpc.json args: template_parameters: cidr: 10.8.0.0/16 register: vpc
- name: Create RDS cloudformation: stack_name={{ environment }}-{{ project }}-rds
state=present region="{{ region }}" template=rds.json args: template_parameters: vpcid: {{ vpc['stack_outputs']['VPCID'] }} subnets: {{ vpc['stack_outputs']['DbSubnets'] }} subnet_index: 0 register: rds
- name: Create WEB cloudformation: stack_name={{ environment }}-{{ project }}-web
state=present region="{{ region }}" template=web.json args: template_parameters: vpcid: {{ vpc['stack_outputs']['VPCID'] }} subnets: {{ vpc['stack_outputs']['PublicSubnets'] }} project: {{ project }} environment: {{ environment }} rds_endpoint: {{ rds['stack_outputs']['RDSEndpoint'] }} register: web
Machine(configura.on(/(Ansible Once%you%brought%up%your%fleet%you%need%to%configure%them Ansible(uses(SSH(to(connect(to(your(machines(and(run(commands
Ansible • Agentless)*)connect)to)servers)via)SSH • Tasks)are)sequen7al • Reusability)through)"roles"
Ansible(Inventory vpn.example.com [elasticsearch] foo.example.com bar.example.com [web] one.example.com two.example.com three.example.com
Ansible(Playbooks(.(Configura4on --- - hosts: elasticsearch sudo: true roles: - role:
elasticsearch - role: users - hosts: web sudo: true roles: - role: nginx - role: supervisord - role: users
Ansible(Playbooks(.(Deploy --- - hosts: web sudo: true roles: - role:
scout - role: celery - role: uwsgi
Ansible(Playbooks(.(Nginx(tasks --- - name: add nginx ppa apt_repository: repo=ppa:nginx/stable state=present
- name: install nginx apt: pkg=nginx state=present - name: nginx config copy: src=etc/nginx/nginx.conf dest=/etc/nginx/nginx.conf notify: restart nginx - name: log folder permissions file: path=/var/log/nginx/ state=directory owner=deploy group=adm - name: cache folder file: path=/var/cache/nginx/ state=directory owner=deploy group=adm - name: launch nginx service: name=nginx state=running enabled=yes
Ansible(Playbooks(.(Run $ ansible all -m ping $ ansible web -m
ping $ ansible-playbook -i ec2.py site.yml $ ansible-playbook -i ec2.py deploy.yml -f 10
Ansible(challenges • Managing'inventories'across'mul2ple'accounts • Provisioning'machines'across'mul2ple'VPCs • Provisioning'on'startup
Mul$ple'accounts 'ec2.py'(dynamic(inventory(works(with(a(single(account h"ps:/ /raw.github.com/ansible/ansible/devel/plugins/inventory/ ec2.py ansible -i ec2.py -u ubuntu
us-east-1d -m ping
Mul$ple'accounts 'ansible)inventory'/IAM/role/in/each/account/with/read)only/ permissions [ec2] iam_roles = arn:aws:iam::13000000001:role/ansible-inventory,arn:aws:iam::25000000002:role/ansible-inventory regions = us-east-1
regions_exclude = # us-gov-west-1,cn-north-1
Mul$ple'accounts • one%central%role%with%'sts:AssumeRole'%permissions • changed%'ec2.py'%to%iterate%through%role%ARNs,%and%run% 'assume_role'%on%each%one • once%we%have%all%the%instances,%generate%dynamic%inventory
ec2.py def get_all_instances(self, iam_role=None): for region in self.regions: kwargs =
{} if iam_role: conn = sts.connect_to_region(region) role = conn.assume_role(iam_role, "AnsibleInventory") kwargs = { 'aws_access_key_id': role.credentials.access_key, 'aws_secret_access_key': role.credentials.secret_key, 'security_token': role.credentials.session_token } self.get_instances_by_region(region, **kwargs) if self.rds_enabled: self.get_rds_instances_by_region(region, **kwargs)
Mul$ple'VPCs • VPC%peering • One%central%infrastructure%VPC%peered%with%individual%app%VPCs
Provisioning)on)startup • Keep%a%zip%with%Ansible%config%in%s3%bucket • Use%UserData%script%to%download%the%latest%ansible%config • Run%it%locally • You%s?ll%need%to%run%'ec2.py'%inventory%script%to%determine% instance%group%membership
Provisioning)on)startup #cloud-config # vim: syntax=yaml packages: - git-core - python-dev
- python-virtualenv - python-pip - awscli runcmd: - update-ca-certificates --fresh - aws --region us-east-1 s3 cp s3://<your_bucket_name>/ansible_config/$(aws --region us-east-1 s3 ls s3://<your_bucket_name>/ansible_config/ | awk '{print $4}' | tail -n1) /tmp/ansible_config.tar.gz - cd /tmp; tar -xzvf ansible_config.tar.gz - cd /tmp/ansible_config; pip install -r requirements.txt - cd /tmp/ansible_config; EC2_INI_PATH="/tmp/ansible_config/config/local.ini" ansible-playbook -i ec2.py site.yml --connection=local --limit `ec2metadata --local-ipv4`,localhost,127.0.0.1 - cd /tmp/ansible_config; EC2_INI_PATH="/tmp/ansible_config/config/local.ini" ansible-playbook -i ec2.py deploy.yml --connection=local --limit `ec2metadata --local-ipv4`,localhost,127.0.0.1 - cd /tmp/ansible_config; EC2_INI_PATH="/tmp/ansible_config/config/local.ini" ansible-playbook -i ec2.py notify.yml --connection=local --limit `ec2metadata --local-ipv4`,localhost,127.0.0.1
Other&challenges • Accessing)your)private)network)4)OpenVPN)(pritunl.com) • Service)discovery)4)Route53)Internal)HostedZones,)Consul • SSH)into)instances)for)developers)by)tags)4)Fabric)script)which) uses)jumpbox)as)a)gateway)host
Ques%ons?
Thank&you