PyConIT: Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana

Transcript

1. Monitora le performance della tua applicazione Python Flask con Elasticsearch e Kibana PyCon Nove Florence 20/04/2018
2. About me MATTEO ZUCCON @matteo_zuccon zuccon.matteo@gmail.com https://whiletrue.run Solution Developer | #PyconNove #PyconIT @pyconit
3. Agenda • What is performance? Definition and meaning • APM - Application Performance Management • Open Source APM with and • Monitor Flask Application
4. What is performance? “total effectiveness of an application or computer system, including throughput , individual response time, and availability” Something is successful in producing a desired result; success
5. What is performance? Monitoring • Application Avg Response Time • Application Load • Monitoring CPU of servers • Tracking and log errors • Everything useful for you – depending from your use case and application
6. What is performance? Management and Monitoring Performance matters! NOT ONLY monitoring ALSO management
7. APM - Application Performance Management “In the fields of information technology and systems management, application performance management (APM) is the monitoring and management of performance and availability of software applications.” “the translation of IT metrics into business meaning”
8. Application performance management or application performance monitoring. APM - Management and Monitoring
9. APM - Management and Monitoring being more proactive only being reactive when it comes to the performance Application performance management or application performance monitoring.
10. APM - It seems easy …monitoring and management of performance
11. APM – Components – 1 of 2 • Performance of individual web requests or transactions • Usage and performance of all application dependencies like databases, web services, caching, etc • Detailed transaction traces down to specific lines of code • Code level performance profiling • Basic server metrics like CPU, memory, etc
12. APM – Components – 2 of 2 • Evaluate performance management criteria • Application aware and focused – understand application context • Custom applications metrics created by the dev team or business • Application log data and errors • Real time monitoring
13. APM – Tools • New Relic • Glimpse • Miniprofiler • XRebel • Stackify Prefix • App Dynamics • Scout Devtrace • Rack trace • Zend Z-Ray
14. APM – Tools – Cloud Providers • X-Ray e Pinpoint • Application Insights • APM
15. APM – Literature
16. APM – Practices
17. APM – Not that easy- Where do I start? A lot of components, tools, books, practices….
18. Elasticsearch and Kibana Elasticsearch is a distributed, RESTful search and analytics engine, heart of the Elastic Stack, it centrally stores your data. Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack
19. Open Source APM - Elastic Already housing logs and system metrics in Elasticsearch? Expand to application metrics with Elastic APM. Four lines of code lets you see a bigger picture to quickly fix issues and feel good about the code you push. (basic subscription – Free license) Elasticsearch and doubling down on “open”: https://changelog.com/podcast/292
20. Elastic APM - Components • Elasticsearch • APM Server • APM Agents • Kibana APM UI
21. APM – Components - Elasticsearch •Elasticsearch • APM Server • APM Agents • Kibana APM UI Centrally stores the application metrics
22. APM – Components – APM Server • Elasticsearch •APM Server • APM Agents • Kibana APM UI Open source application written in Go which runs on your servers. It receives data from agents periodically. The API is a simple JSON based HTTP API. APM Server builds Elasticsearch documents from the data received from agents.
23. APM – Components – APM Agents • Elasticsearch • APM Server •APM Agents • Kibana APM UI Hook into your application and start collecting performance metrics and errors automatically. Collected data is buffered for a short period and sent on to APM Server.
24. APM – Components - Kibana • Elasticsearch • APM Server • APM Agents •Kibana APM UI dashboard to visualize the APM metrics
25. APM – Components Schema
26. APM - Agents • Python* • NodeJS • Ruby • Java • Go • JavaScript Version 1.8 – 2.0 Version 0.10+ * no support async frameworks like Twisted or Tornado
27. Elastic APM - Pros • Open Source, free and easy to use • APM Agents • Fully integrated with Elasticsearch • Built in Kibana Dashboard
28. APM – Python Flask - Configuration pip install elastic-apm[flask] For apm-server 6.2+, make sure you use version 2.0 or higher of elastic-apm
29. APM – Python Flask - Routes Automatically track transactions and capture uncaught exceptions within Flask.
30. APM – Python Flask - Dashboard
31. APM – Python Flask - Dashboard List of requests with details
32. APM – Python Flask - Dashboard Requests impact
33. APM – Python Flask - Dashboard Response Times
34. APM – Python Flask - Dashboard Request per minute by HTTP Code
35. APM – Elasticsearch Document
36. APM – Python Flask – Arbitrary Exception Send additional events by raising an exception or logging a generic message
37. APM –Extra Data and ignore specific routes Send additional data Ignore specific route
38. APM – Instrumenting custom code Decorator or context manager Transaction sample
39. APM – Instrumenting custom code Decorator or context manager Transaction sample Transaction Sample Timeline
40. APM – Instrumenting custom code Response Time Distribution
41. APM – Sanitizing data Add processors Custom Processor
42. APM –Tuning and Overhead considerations • CPU time • Memory • Bandwidth use • Elasticsearch storage • Transaction Sample Rate: randomly sample only a subset of transactions • Transaction Queue: reduce queue size and flush time • Spans per transaction: transactions should have well below 100 spans
43. APM – Elastic - Python • Agents ready to go (Flask and Django) • Few lines of code • Custom exceptions and messages • Transactions and requests • Exclude route and sanitize data • APM Server public API
44. APM – Elastic - Python Thank you Questions? @matteo_zuccon zuccon.matteo@gmail.com https://whiletrue.run #PyconNove #PyconIT @pyconit https://bit.ly/2JYGZtZ