website for creating courses • A platform for research and analytics • A community of learners and developers creating courses and extending the edX open source platform
edx.org • Login as staff@example.com / edx • This public sandbox runs on a single EC2 server that is recreated weekly • One of the many examples of Ansible automation @ edX
~ 700 Ansible tasks • ~ 150 forks of our configuration repo • edX Ansible plays are run every day at edX and around the globe to provision and deploy the platform
◦ Even the most simple tasks felt like we were constantly fighting the tool ◦ We wanted one tool for code updates and base OS configuration While we were having issues with Puppet Stanford was adopting the edX platform and having similar issues with Chef. edX started using Ansible (1.3) in September, 2013
are short, simple and often limited to calling a single role • Ansible roles are self-contained ◦ Role name prefix for variable namespace ◦ Defaults defined in the role, overridden using extra vars.
over the tools and the data • Isolate deployments, student privacy is extremely important • Operates in the cloud but strives to be cloud agnostic • Official releases of our configuration scripts with Vagrant images and public AMIs
Using Vagrant, code directories are exported from the host filesystem to the guest • Fullstack - Every service in production installed on a single server • A small number of multi-cluster redundant configurations (stage, production, loadtest)
same as production • One-click single instance provisioning for launching sandboxes • Ansible handles creation, termination and config • Official vagrant images and public AMIs
• python virtual environments for every service • pre-built images for AWS auto-scaling • AWS virtual private clouds used for multiple edX installations • VPC layout described using cloudformation • Utilizing AWS resources where possible (RDS, ELB, S3, Elasticcache, SES)
source repo contains all of the edX roles • A private repo contains roles internal to edX • A private repo containing one yaml file per environment for variable overrides • For code version updates, vars are set on the command-line
that connects to other VPCs and runs ansible plays. • Inventory is fed directly into Jenkins via an inventory script that gathers tag information • All Ansible tasks are run from Jenkins for auditing and repeatability
to ec2.py group names so tasks can be run on one server in a tag group • Use serial with pre and post tasks for rolling updates group names returned by ec2.py
easier to scale • Ansible was the perfect tool to make this work with very little additional effort • Netflix’s Asgard has proven extremely useful for managing autoscaling groups and, cut overs and rollbacks.
script launches an instance and runs ansible on bringup in user-data • Progress is sent back to the user via sqs using a custom Ansible callback plugin