Application Monitoring in Microsoft Azure

Session about Application Monitoring in Microsoft Azure. Inspecting the topics What's required for application monitoring, Setup application monitoring, Best practices and costs.

Transcript

1. APPLICATION MONITORING IN MICROSOFT AZURE A Developers’s Perspective
2. ABOUT ME Daniel Lindemann Enthusiastic .NET developer and consultant with a strange love for optimizing, automating and containerizing applications. What I do: ▪ Microsoft Azure ▪ Cloud-native & Serverless architectures ▪ Container technologies ▪ DevOps - Dev at night, Ops by day E-Mail: d.lindemann@abtis.de LinkedIn: https://linkedin.com/in/daniel-lindemann Web: https://dlindemann.de https://abtis.de
3. AGENDA ▪ What is Application Monitoring? ▪ Application Monitoring in Azure ▪ Application Monitoring with Application Insights ▪ Custom Queries with KQL ▪ Alerting in Microsoft Azure ▪ Monitoring Costs ▪ Monitoring vNext
4. BEFORE WE START Disclaimer ▪ Demos are built with .NET 7.0 (C#) ▪ VS Code Rulez ▪ Terminal ❤️
5. BEFORE WE START Visit the demo application https://dlmn.link/demo-aam
6. WHAT IS APPLICATION MONITORING?
7. APPLICATION MONITORING What is Application Monitoring? Application monitoring is the process of collecting log data in order to help developers and administrators track availability, bugs, resource usage, and changes to performance in applications that affect the end-user experience (UX).
8. APPLICATION MONITORING How does this help me do my job? Know what your apps are doing and how they are performing ▪ Incoming Requests ▪ Exceptions ▪ Dependency calls (DB queries, Http requests, …) ▪ Logs ▪ Metrics (CPU, Memory) ▪ Tracing and correlation
9. APPLICATION MONITORING Logging setup ▪ Logs show what is happening in an application ▪ Logs show important information and at what time they happened ▪ Reading logs is debugging in a production environment ▪ Best practices ▪ Use structured logging for storing extra information ▪ Stick to best practices for cloud-native apps (https://12factor.net/logs) ▪ Log in JSON-format, so the platform can scrap
10. APPLICATION MONITORING Distributed logging sytem ▪ Logs can be viewed in one place ▪ Logs are searchable ▪ Analyse logs for specific errors ▪ Please don’t log ▪ In databases ▪ In logfiles without cleanup
11. APPLICATION MONITORING Distributed Tracing Web Service A Service B
12. APPLICATION MONITORING Correlation Web Service B Service A Id:1 Id:2 Parent: 1 Id:3 Parent: 2
13. APPLICATION MONITORING Spans Trace Span Span Span Id: 00-a678b313d0b921195028ed37952d7a99-8ac83a9e17967ea6-01 ParentId: 00-a678b313d0b921195028ed37952d7a99-cbf8f5c99a26d994-01 RootId: a678b313d0b921195028ed37952d7a99 StartTimeUTC: 9/15/2021 7:07:01 AM Tags: - shoe.size: EU 44 - shoe.type: Sneaker - shoe.color: White
14. APPLICATION MONITORING IN AZURE
15. APPLICATION MONITORING Application Monitoring in Azure Application Insights is an extension of Azure Monitor and provides application performance monitoring (APM) features. Application Insights overview - Azure Monitor | Microsoft Learn Azure Monitor is a comprehensive monitoring solution for collecting, analyzing, and responding to telemetry from your cloud and on-premises environments. You can use Azure Monitor to maximize the availability and performance of your applications and services. Azure Monitor overview - Azure Monitor | Microsoft Learn Azure Monitor Application Insights
16. APPLICATION MONITORING Where can application monitoring be used Application types that can be monitored: ▪ Apps running on VMs ▪ Apps running on app services / function apps / logic apps ▪ Apps running in containers (ACA / AKS) ▪ Console Apps ▪ Worker Services ▪ Almost everything!
17. APPLICATION MONITORING Service integrations Log Analytics Workspace Application Insights Azure Monitor VMs Web Apps Functions AKS Container Apps ... VM Insights Container Insights +
18. APPLICATION MONITORING Diagnostic logs ▪ Should application monitoring include diagnostic logs? → It depends! ▪ Detailed information about ▪ Resource changes ▪ Infrastructure changes ▪ Azure events ▪ Diagnostic logs can be stored in different locations ▪ Examples for interesting monitoring information ▪ Scaling is not working correctly (App service) ▪ Pod killed and could not be recreated (AKS) ▪ Timeouts / Blocks / Deadlocks (SQL)
19. APPLICATION MONITORING WITH APPLICATION INSIGHTS
20. APPLICATION MONITORING Use DIKW model - Actions and decisions move from data to information, knowledge, and wisdom Cloud monitoring strategy - Cloud Adoption Framework | Microsoft Learn
21. APPLICATION MONITORING How to start ▪ Plan your monitoring strategy ▪ Centralized, Decentralized, Hybrid ▪ Enable resource diagnostic settings ▪ Initial alert rules ▪ Add basic configuration for monitoring ▪ Start extending the configuration based on your needs ▪ You don’t know all the parts where your application can break ▪ You don’t know what information you need
22. APPLICATION MONITORING Configure application insights ▪ Application insights configuration ▪ Add Application Insights library to project ▪ Set Application Insights connection string ▪ Add application insights to frontend and backend ▪ Application insights is available on many platforms including .NET, Java, JavaScript, NodeJS, Python Add Application Insight to web application via nuget
23. APPLICATION MONITORING Logging with Application Insights ▪ Use ILogger to log to Application Insights ▪ Configure Application Insights logging within your appsettings.json file ▪ Application Insights instrumentation adds a new logging target ▪ ApplicationInsightsLoggerProvider captures ILogger logs and creates TraceTelemetry from them
24. DEMO
25. APPLICATION INSIGHTS Extensibility ▪ Access to TelemetryClient in .NET using Dependency Injection ▪ Custom Metrics ▪ .NET: Garbage Collection, Threads, Memory, CPU ▪ Custom events ▪ Custom spans ▪ Logging of database queries* ▪ Etc. * not recommended
26. CUSTOM QUERIES WITH KQL
27. QUERY DATA How data is stored Application Operating system Azure Resources Azure Subscriptions Azure Tenant Custom sources Data sources Autoscale Alerts Respond Views Dashboards Visualize Application Virtual machines Container Insights Monitor Logs analytics Metric analytics Analyze Access to this information
28. QUERY DATA Query data with Kusto ▪ KQL (Kusto Query Language) ▪ Used to query data from Azure Monitor Logs ▪ Kusto is similar to SQL ▪ Has tons of powerful functions for time series ▪ Filter ▪ Projection ▪ Aggregations ▪ And lots more … ▪ http://aka.ms/kdocs
29. QUERY DATA Sample query // "Dependency failures rate" trendline over the last 12 hours // with 20 minutes granularity. // The last point in the chart represents the detected rate. dependencies | where timestamp >= datetime("2023-05-06T22:01:00") - time(12h) and timestamp <= datetime("2023-05-06T22:21:00") | extend failures = iff(success == "False", itemCount, 0) | summarize failure_rate = (sum(failures) / todouble(sum(itemCount))), failures = sum(failures), total = sum(itemCount) by bin(timestamp, 20m) | project timestamp, failure_rate | render timechart
30. DEMO
31. KUSTO Learn Kusto https://detective.kusto.io/
32. ALERTING IN MICROSOFT AZURE
33. ALERTS What are alerts? ▪ Proactive notifications when monitoring data suggest a problem with the infrastructure or the application ▪ Helps to detect and address issues before users notice ▪ Investigate data in a specific time range coupled to specific conditions
34. ALERTS Alert system in Azure Overview of Azure Monitor alerts - Azure Monitor | Microsoft Learn
35. ALERTS Setup alert ▪ Know the query and data for proactive alerting ▪ Setup an action group ▪ The creation of default action groups for administrators could be part of the landing zone ▪ Use log analytics page to create an alert rule and configure the conditions
36. DEMO
37. MONITORING COSTS
38. MONITORING COSTS Cost calculation ▪ Be sure how much you’re logging ▪ Configure loggers ▪ Configure log level ▪ Optimize costs ▪ Sampling ▪ Set daily caps in Log Analytics Workspace and Application Insights ▪ Check estimated costs by using Azure pricing calculator and Microsoft Learn ▪ Estimating costs to monitor your AKS cluster explains why a default AKS cluster creates 744 mb/day of logs ▪ Learning by doing
39. MONITORING COSTS Pricing calculator https://azure.com/e/7e9fe90e2a014056a44df0940bea8338
40. MONITORING VNEXT
41. MONITORING VNEXT Grafana Azure Managed Grafana | Microsoft Azure
42. MONITORING VNEXT Open Telemetry Logs Tracing Metrics Telemetry
43. MONITORING VNEXT Open Telemetry + Application Insights ▪ Actually available as preview ▪ Install the library (.NET) dotnet add package --prerelease Azure.Monitor.OpenTelemetry.AspNetCore ▪ Add instrumentation (.NET) builder.Services.AddOpenTelemetry().UseAzureMonitor( // Add connection string options => options.ConnectionString = "<Your Connection String>" );
44. QUESTIONS?
45. RESOURCES ▪ Application Insights overview - Azure Monitor | Microsoft Learn ▪ Design a solution to log and monitor Azure resources - Training | Microsoft Learn ▪ Analyze usage in a Log Analytics workspace in Azure Monitor - Azure Monitor | Microsoft Learn
46. Die abtis GmbH verfügt über mehr als 20 Jahre Erfahrung in der Planung und dem Betrieb von Microsoft Infrastrukturen und betreut bereits mehr als 200.000 Anwender:innen der Cloudplattformen Microsoft 365 und Azure. Ausgezeichnet als Microsoft Solutions Partner und MXDR Verified Partner mit 12 Advanced Specializations sind wir einer der wichtigsten Fokuspartner von Microsoft für den Mittelstand in Deutschland. Damit setzen wir ein starkes Zeichen als verlässlicher Partner und Vorreiter in der IT-Branche. Die abtis GmbH ist Teil der abtis Gruppe, die mit vier Tochterunternehmen und über 170 Mitarbeitenden ein fester Bestandteil der IT-Welt ist. Das Portfolio der abtis Gruppe umfasst die Kernthemen einer zukunftsorientierten IT: von Modern Workplace, über Datacenter, Security, Power Platform, Application Development, Industrial IoT, Adoption & Change Management bis hin zu Data & AI. www.abtis.de +49 7231 4431 - 100 vertrieb@abtis.de © 2023 Alle Rechte vorbehalten. Dieses Dokument ist urheberrechtlich geschützt. Sämtliche Inhalte dienen der Dokumentation. Jede andere Nutzung, insbesondere die Weitergabe an Dritte, die Verbreitung oder die Bearbeitung, auch in Teilen, ist ohne schriftliche Einwilligung der abtis GmbH untersagt. Die verwendeten Firmen-, Marken- und Produktnamen und Warenzeichen sind eingetragene Markenzeichen oder Warenzeichen der jeweiligen Inhaber und werden hiermit anerkannt.