AWS Observability (without the Pain)

Good observability is a must-have for modern AWS applications. CloudWatch gives you the tools but leaves you with a lot of work to do. To accelerate this, we created a serverless plugin to quickly build best practice alarms and dashboards.

Transcript

1. AWS Observability
   without the pain
   Eóin Shanaghy - Luciano Mammino
   AWS Summer School - June 24th 2021
   

   View Slide

2. Hi! I’m Eoin 🙂
   CTO
   aiasaservicebook.com
   @eoins
   eoins
   ✉ Get in touch
   

   View Slide

3. 👋 Hello, I am Luciano
   Senior architect
   nodejsdesignpatterns.com
   Let’s connect:
   🌎 loige.co
   🐦 @loige
   🎥 loige
   🧳 lucianomammino
   

   View Slide

4. We are business focused technologists
   that deliver.
   Accelerated Serverless | AI as a Service | Platform Modernisation
   We are hiring! Let’s have a chat 🙂
   

   View Slide

5. fth.link/o11y-no-pain
   

   View Slide

6. Observability in the cloud
   a measure of how well internal states of a
   system can be inferred from knowledge of its
   external outputs
   🪵 🔍 📈 🚨
   Structured Logs Tracing Metrics Alarms
   “
   

   View Slide

7. AWS native o11y = CloudWatch
   Cloudwatch gives you:
   ➔ Logs with Insights
   ➔ Metrics
   ➔ Dashboards
   ➔ Alarms
   ➔ Canaries
   ➔ Distributed tracing (with X-Ray)
   

   View Slide

8. Outside AWS
   Established
   New entrants
   Roll your own (only for the brave)
   

   View Slide

9. CloudWatch out of the box
   😍 A toolkit you can use to build
   observability
   🤩 Metrics are automatically
   generated for all services!
   😟 Lots of dashboards, but by
   service and not by application!
   😢 Zero alarms out of the box!
   

   View Slide

10. ✅ Third-party services offer a better UX
    ❌ Data needs to be exported from
    AWS
    ❌ Uncertainty about future direction of
    new providers in this space
    ❌ You might be stuck in dual-mode
    Using third-parties
    

    View Slide

11. Getting the best out of Cloudwatch
    Cloudwatch can be your friend if you...
    📚 Research and understand available metrics
    📐 Decide thresholds
    📊 Write IaC for application dashboards
    ⏰ Write IaC for service metric alarms
    ⏪ Update every time your application changes
    📋 Copy and paste for each stack in your application
    (a.k.a. A LOT OF WORK!)
    

    View Slide

12. Best practices
    😇 AWS Well Architected Framework
    🏛 5 Pillars
    ⚙ Operational excellence pillar covers observability
    🧐 Serverless lens applies these pillars
    👍 Good guidance on metrics to observe
    👎 More reading and research + you still have to pick thresholds
    

    View Slide

13. CloudFormation for CloudWatch Alarms 😬
    "Type": "AWS::CloudWatch::Alarm",
    "Properties": {
    "ActionsEnabled": true,
    "AlarmActions": [
    "arn:aws:sns:eu-west-1:665863320777:FTSLICAlarms"
    ],
    "AlarmName": "LambdaThrottles_serverless-test-project-dev-hello",
    "AlarmDescription": "Throttles % for serverless-test-project-dev-hello ..",
    "EvaluationPeriods": 1,
    "ComparisonOperator": "GreaterThanThreshold",
    "Threshold": 0,
    "TreatMissingData": "notBreaching",
    "Metrics": [
    {
    "Id": "throttles_pc",
    "Expression": "(throttles / throttles + invocations) * 100",
    "Label": "% Throttles",
    "ReturnData": true
    },
    {
    "Id": "throttles",
    "MetricStat": {
    "Metric": {
    "Namespace": "AWS/Lambda",
    "MetricName": "Throttles",
    "Dimensions": [
    {
    "Name": "FunctionName",
    "Value": "serverless-test-project-dev-hello"
    }
    ]
    },
    "Period": 60,
    "Stat": "Sum"
    },
    "ReturnData": false
    },
    {
    "Id": "invocations",
    "MetricStat": {
    "Metric": {
    "Namespace": "AWS/Lambda",
    "MetricName": "Invocations",
    

    View Slide

14. Can we automate this?
    Magically
    generated alarms
    and dashboards for
    each application!
    

    View Slide

15. fth.link/slic-watch
    Introducing
    SLIC Watch
    

    View Slide

16. How SLIC Watch works 🛠
    Your app
    serverless.yml
    sls deploy
    CloudFormation stack
    very-big.json
    SLIC Watch
    👀 🛠
    CloudFormation stack ++
    even-bigger.json
    Deploy ☁
    📊📈
    

    View Slide

17. Before SLIC Watch
    

    View Slide

18. After SLIC Watch
    

    View Slide

19. After SLIC Watch
    

    View Slide

20. After SLIC Watch
    

    View Slide

21. After SLIC Watch
    

    View Slide

22. After SLIC Watch
    Check out SLIC Slack
    

    View Slide

23. Configuration
    🎀 SLIC Watch comes with sane defaults
    📝 You can configure what you don’t like
    🔌 Or disable specific dashboards or alarms
    

    View Slide

24. How to get started
    📣 Create an SNS Topic as the alarm destination
    📦 ❯ npm install serverless-slic-watch-plugin --save-dev
    ✍ Update serverless.yml
    ⚙ Configure (optional)
    🚢 ❯ sls deploy
    plugins:
    - serverless-slic-watch-plugin 💡 Check out
    the complete
    example project
    in the repo!
    

    View Slide

25. Wrapping up 🎁
    ★ CloudWatch is very valuable if you configure it right!
    ★ Automation takes away the pain
    ★ SLIC Watch gives you this automation
    ★ You still have control and flexibility
    🔬Try it out! 🗣 Give feedback! 🌈 Let’s make it better!
    

    View Slide

26. Thank you!
    fth.link/o11y-no-pain
    Cover picture by Javier Balseiro on Unsplash
    

    View Slide