Upgrade to Pro — share decks privately, control downloads, hide ads and more …

I get by with a little help from my friends (Hashidays AMS Day 2 Keynote 2018)

E5b21a91ebcda83d51e9d59fdc0883cc?s=47 Abby Fuller
June 27, 2018

I get by with a little help from my friends (Hashidays AMS Day 2 Keynote 2018)


Abby Fuller

June 27, 2018


  1. I get by with a little help from my friends

    Abby Fuller, @abbyfuller Developer Relations, AWS AMSTERDAM AMSTERDAM
  2. Hi, I’m Abby. I work in developer relations for AWS,

    and I talk about containers a lot.
  3. Containers are great, but with them comes a fair amount

    of work: scheduling, resource usage, service discovery, deployments, monitoring, observability, logging…
  4. None
  5. Even when it is *literally* your job to keep up

    with the container space, it’s still overwhelming!
  6. It seems like every day, there’s a new project …Or

    a blog …Or a new tool …Or a talk
  7. So how are you supposed to keep up?

  8. None
  9. You can get by with a little help from your

  10. I mean this two (2) ways: 1. Through your tooling

    and processes (tools are friends!) 2. And also through the community (person friends!)
  11. Let’s talk tools. In other words, how can you work

    smarter, not harder?
  12. Safety first!

  13. None
  14. If you’re working with AWS, this starts with roles In

    other words, how you can, and your services perform actions securely? All types of IAM roles: • Service roles • Controls what the service can access on your behalf( for example, access to DynamoDB) • User roles • Represent a person, and what they can access. Forexample,a role “abby” with administrator access • Groups • logical groupings of IAMUsers with shared permissions
  15. Sssh, I have secrets. Storing secret keys and passwords is

    worth a whole talk on its own. A few words of advice: • Rotate your keys, don’t share them between a) services, b) humans • Go for “principle of least access” • Where possible, store your keys somewhere else, and then control access (for example, store keys in KMS, and control access with IAM/service roles) • There are also third party tools out there (lots): for example, Vault
  16. There are also tools to help with the security of

    your containers themselves Lean on tools to check your images for vulnerabilities. A couple of options, both paid and open source: • Aqua MicroScanner (community edition) • Aqua continuous image assurance • Docker Security Scan with Trusted Registry • Clair from CoreOS
  17. None
  18. Now that we’re thinking about safety, let’s go from the

    bottom up.
  19. Good development starts locally • Standardized, easy to set up

    local environments help eliminate “well it worked on my machine” • Some common approaches to this: Vagrant and/or VirtualBox, EC2 box with local access • Any approach should have a few things in common: shared configuration between team members (i.e., running the environment uses the same versions of packages), and is as close as possible to the production environment.
  20. Some tools that might help you out • Vagrant •

    AWS Cloud9 • Cloud development environment that integrates a shell, AWS resources, and an editor • If you’re using Kubernetes, you can run it locally for development purposes with MiniKube
  21. From local environment to production, you probably interact with your

    containers through CLIs. Your CLI depends on your orchestration platform. We’ll look at two: Elastic Container Service (ECS) from Amazon, and Kubernetes.
  22. CLIs for ECS (and Fargate) • aws-cli: the official OG.

    Open source, includes most AWS services. • ecs-cli: also official, but just for ECS. Supports docker compose files. Some good unofficial options: • Fargate CLI • Coldbrew CLI
  23. (A few of the) CLIs for Kubernetes • kubectl: run

    commands against Kubernetes clusters. • kops: kubectl for clusters. Can use to manage production Kubernetes infrastructure in AWS. • kubeadm: bootstrap production-ready Kubernetes clusters, following best practices. Not production ready, but cool: • kubicorn: helps users manage their Kubernetes infrastructure
  24. So you’re interacting with your clusters with CLIs- now what?

  25. Automate!

  26. Say it with me: no artisanal containers • Infrastructure as

    code: automate and template everything from your containers to your infrastructure. This makes everything repeatable and portable! • Containers can be built directly from repositories as part of your CI/CD process. • After that, automate your AWS!
  27. A couple options for AWS automation Cloudformation: • Common language

    for describing, creating, and deleting AWS resources. • Created and maintained by Amazon. • Free to use, just pay for the resources created. Don’t like writing Cloudformation templates? There are samples and snippets here.
  28. Looks something like this!

  29. Terraform: • Created by Hashicorp (thanks y’all!) • Open source

    Either one works! Both focus on resource management. Cloudformation: planning and execution in one step. Meaning, just executes. Terraform: planning and execution in two steps (terraform plan, terraform apply) A less JSON-y option: Terraform!
  30. Here’s a Terraform example

  31. You’re all automated! You’re using CLIs! You thought about security!

    How do you know if everything is working?
  32. Monitoring! Observability! These help you a) debug, and b) understand

    what’s happening in your system
  33. Beyond just logs, you want actual, useable, helpful information about

    what’s happening with your systems.
  34. If it doesn’t make sense to you at 3am, it’s

    probably not helping much: a memoir.
  35. Just a few things to think about: • Reduce the

    noise • Page only on issues that require immediate, emergency attention • You need all of it- both monitoring AND observability (more than just logs!) • Make sure you’re asking and answering the right questions.
  36. You don’t have to go it alone! Lots of tools

    out there to help you make sense of what’s going on
  37. Here’s a caveat though: you have to use the RIGHT

    tool for the job.
  38. Don’t do everything yourself when you don’t have to, but

    don’t be afraid to get fancy on your own if there isn’t a tool that works for you!
  39. This might be things like tweaking your load balancer and

    autoscaling group settings for faster deployments
  40. None
  41. Or bringing a custom AMI to AWS

  42. The easiest way to use ECS is with the ECS-optimized

    AMI, but life is too short to not have custom AMIs. A few steps to registering your own AMI with ECS: 0. Make sure your AMI has the right requirements! 1. Install ecs-agent 2. Install Docker daemon (if it’s not already installed) 3. Register instance with cluster Most of these options can be set in EC2 user-data (more on that in a sec)
  43. With Amazon Linux, in /etc/ecs/ecs.config: ECS_ENABLE_TASK_IAM_ROLE=true ECS_ENABLE_TASK_IAM_ROLE_NETWORK_HOST=true ECS_LOGFILE=/log/ecs-agent.log ECS_AVAILABLE_LOGGING_DRIVERS=["json- file","awslogs"]

    ECS_LOGLEVEL=info ECS_CLUSTER=default Full instructions here.
  44. Or resorting to configuring everything with Bash in EC2 user-data

  45. User-data is good for everything from starting services, to configuring

    your environment, to enabling options and flags that aren’t supported in the UI. Scripts run at boot, and you can modify them through the EC2 Console.
  46. For example, I can pass in Docker flags that are

    unsupported by the AWS UI: $ echo ”WHATEVER_FLAG=hi” >> /etc/sysconfig/docker Here be dragons. And cruft.
  47. None
  48. OK, so remember when I said there were two ways

    you could get some help? Here’s #2
  49. Your best resource is the community.

  50. Kubernetes, AWS, Hashicorp services and tons of open source projects

    all have vibrant communities and resources here to help you.
  51. None
  52. Talk to peers, and colleagues, and internet friends. Write a

    blog, or give a talk. Come to conferences (just like this one).
  53. None
  54. If you don’t know how to get started, or you’re

    struggling: ask for help! I guarantee that someone has been in the same situation before.
  55. Some resources to get you started Find a Kubernetes meetup

    Join the TGIK community call with Kris Nova and Joe Beda! Or a SIG. Try an AWS User Group Or try one of the approximately 12957329057 Slack channels out there for a community that works for you!
  56. Thank you! @abbyfuller abbyfull@amazon.com AMSTERDAM