Making a Bag of Metrics with Open Source APM and Tracing

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

©2008–18 New Relic, Inc. All rights reserved 5 APM History 1998 Wily the beginning of APM Lew Cirne founds Wily and creates the APM space focusing on Java 2006 CA Acquires Wily CA decides to add to its APM portfolio 2008 New Relic Lew Cirne founds New Relic to be a Ruby on Rails APM 2015 OpenTracing Developers from Lightstep and Jaeger agreed to define a common API called OpenTracing 2016 OpenCensus Google had enough internal adoption of Census metrics and distributed tracing to open source a combined system called OpenCensus . 2019 New Relic ONE New Relic launches the first Entity Based Observability Platform OpenTracing/OpenCensus projects merge

Slide 6

Slide 6 text

©2008–18 New Relic, Inc. All rights reserved 6 Trace Context Abstract This specification defines standard headers and value format to propagate context information that enables distributed tracing scenarios. The specification standardizes how context information is sent and modified between services. Context information uniquely identifies individual requests in a distributed system and also defines a means to add and propagate provider-specific context information. What is Trace Context?

Slide 7

Slide 7 text

©2008–18 New Relic, Inc. All rights reserved 7 Trace Context ● provides an unique identifier for individual traces and requests, allowing trace data of multiple providers to be linked together. ● provides an agreed-upon mechanism to forward vendor-specific trace data and avoid broken traces when multiple trace tools participate in a single transaction. ● provides an industry standard that intermediaries, platforms, and hardware providers can support. WHY? A unified approach for propagating trace data improves visibility into the behavior of distributed applications, facilitating problem and performance analysis. The interoperability provided by trace-context is a prerequisite to manage modern micro-service based applications. What does it do?

Slide 8

Slide 8 text

©2008–18 New Relic, Inc. All rights reserved 8 Open Tracing In 2015, developers from Lightstep and Jaeger agreed to define a common API called OpenTracing. ● The goal was to encourage framework and library developers to provide instrumentation as part of their frameworks. OpenTracing defines the API only. ● It does not dictate the implementation, and it does not define the wire protocol for either passing context (TraceContext) nor exporting trace data. ● OpenTracing became one of the early members of Cloud Native Computing Foundation (CNCF). What is OpenTracing? Distributed tracing, also called distributed request tracing, is a method used to profile and monitor applications, especially those built using a microservices architecture. Distributed tracing helps pinpoint where failures occur and what causes poor performance. Who uses OpenTracing? IT and DevOps teams can use distributed tracing to monitor applications. Distributed tracing is particularly well-suited to debugging and monitoring modern distributed software architectures, such as microservices. Developers can use distributed tracing to help debug and optimize their code. What is OpenTracing?

Slide 9

Slide 9 text

Slide 10

Slide 10 text

©2008–18 New Relic, Inc. All rights reserved 10 OpenCensus ● OpenCensus provides an agent per language with built-in instrumentation for metrics and distributed traces. ● Its APIs are in support of its ecosystem and implementation. ● Google found value in having one agent that correlated metrics, traces, and other monitoring data together and they found that an agent approach was the best way to achieve widespread adoption and consistency. What is OpenCensus? OpenCensus is a set of libraries for various languages that allow you to collect application metrics and distributed traces, then transfer the data to a backend of your choice in real time. This data can be analyzed by developers and admins to understand the health of the application and debug problems. What data does OpenCensus collect? Metrics are any quantifiable piece of data that you would like to track, such as latency in a service or database, request content length, or number of open file descriptors. Viewing graphs of your metrics can help you understand and gauge the performance and overall quality of your application and set of services. Traces show you how a request propagates throughout your application or set of services. Viewing graphs of your traces can help you understand the bottlenecks in your architecture by visualizing how data flows between all of your services. Other types of telemetry will be added to OpenCensus as the project matures. Logs will likely be added next. What is OpenCensus?

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

©2008–19 New Relic, Inc. All rights reserved 18 Deliver more perfect software faster than ever with New Relic One—the industry’s first entity- centric observability platform for teams with complex environments.

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

©2008–18 New Relic, Inc. All rights reserved Why New Relic? 21 ● New Relic has believed and contributed to OpenSource Software since its founding. We believe that it is important to correlate Application metrics with Infrastructure metrics, Logs, Events, i.e. Entities. ● We do not want developers to have to choose a vendor implementation when instrumenting code. ● Many companies start with OSS observibility; Prometheus, Jaeger, etc. and then move to a vendor-based solution that works with all of their open source telemetry because managing systems at scale is difficult. ● 1st entity based observability platform ● New Relic is automated end-to-end implementation.

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Confidential ©2008–17 New Relic, Inc. All rights reserved 11 25 Service 1 Agent Service instrumentation Instrument Propagate Context Service 2 Trace Context Agent Service instrumentation New Relic Trace Data Record Trace Data Ingest Analysis Storage Instrument Visualize Traces New Relic Distributed Tracing Automated end-to-end implementation

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

©2008–18 New Relic, Inc. All rights reserved Links & References 28 https://newrelic.com/ https://en.wikipedia.org/wiki/Application_performance_management https://en.wikipedia.org/wiki/Wily_Technology https://www.w3.org/TR/trace-context/#design-overview https://opentracing.io https://opencensus.io https://opentelemetry.io/ https://github.com/open-telemetry/community https://www.youtube.com/watch?v=IdYZphA5S7s&feature=youtu.be https://www.linkedin.com/in/andrew-chris-satter-7009531a/

Slide 29

Slide 29 text