GDG Ajah - Logging with Stackdriver on GCP

Transcript

1. Bakare Emmanuel, Interswitch Group @tiemmaBakare Lagos Logging with Stackdriver on GCP
2. I’m Bakare Emmanuel • Software Developer Intern @Interswitch • Volunteer Bootcamp Facilitator and LCA @Andela • DevOps and Linux Fan Boy • You can follow me on twitter @TiemmaBakare • General Weird Guy with some humour • People call me Bakman, so there’s also that! @ T i e m m a B a k a r e 2
3. Things to note in this session • This is going to be a very practical one, I’d rather show than tell • All demo’s use the Node SDK, you can use Stackdriver in other languages using their own respective SDKs • If you aren’t used to Javascript, don’t worry. You can use it other languages • For those who don’t code, I’d explain the concepts to the best of my abilities. • Lastly, for those who know all this stuff, Twale!
4. Slides and code for this session Slides bit.ly/stack-slides Github Code bit.ly/stack-github App Engine application bit.ly/stack-demo You can view all the materials for this using the links below
5. What is Stackdriver? 5
6. What is StackDriver? Stackdriver is a logging and metrics tool which aims to help you analyze the logs your application develops to help you find and diagnose possible performance / error issues. If you want to see the full documented services, check here: https://cloud.google.com/stackdriver/docs/
7. What features does it have? Tracing Logging Error Reporting Debug
8. You can set this up to manage the features there likewise Monitoring Alerting Dashboard Monitoring Uptime Checks
9. Let’s start with Error Reporting 9
10. Error Reporting Error reporting involves what it says. We want to send errors that occurs in our application to Stackdriver so that they can be analyzed much better. To put this into perspective, how do you know when your data has nearly finished without you checking? That’s what this does, it helps you with gathering things that broke in your application so that you can see it after it happened and take action.
11. Error Reporting Stackdriver error reporting comes preloaded with an SDK for easy use in your application. It currently supports the following languages: • Go • Java • Node.js • PHP • Python • Ruby • .NET For this demo, I’d be using the Node SDK.
12. //…..Left out other dependencies // Imports the Google Cloud client library const {ErrorReporting} = require('@google-cloud/error-reporting'); // With ES6 style imports via TypeScript or Babel, the following // syntax can be used instead // import {ErrorReporting} from '@google-cloud/error-reporting’; // Instantiates a client const errors = new ErrorReporting(); // Note that express error handling middleware should be attached after all // the other routes and use() calls. See the Express.js docs. app.use(errors.express);
13. app.get('/', function (req, res, next) { const requestRandomNumber = Math.floor(Math.random() * 255); if (requestRandomNumber > 150) { //Let's purposefully break our app and add a weird status alongside const status = Math.floor(Math.random() * (500 - 400)) + 400; res.send({ error: true, random: randomNumber }).status(status); // Forward middleware request back next(new Error(`Error caused by random number >150 with value ${requestRandomNumber} and returned status ${status}`)); } }
14. Let’s see how this works with a DEMO 14
15. After triggering errors
16. From here, we can see what route was called, the status code and even the user agent that called our API.
17. “If debugging is the process of removing software bugs, then programming must be the process of putting them in. ” - Edger Djikstra
18. Let’s continue with Tracing 18 Now that we understand what Error reporting can do for us
19. Tracing In software engineering terms, Tracing involves a specialized use of logging to record information about a program's execution. It tells us things like how long the request took, what other APIs were called during its run, how long it took to render the template and all that. It’s like debugging API calls using a traceroute which is where I’m heading to.
20. //…Left out other dependencies // But all we need to do is just include it and start it, it’ll collect the traces automatically require('@google-cloud/trace-agent').start({ enhancedDatabaseReporting: true}); // This incoming HTTP request should be captured by Trace app.get('/trace', (req, res) => { const DISCOVERY_URL = 'https://www.googleapis.com/discovery/v1/apis’; // This outgoing HTTP request should be captured by Trace // We use the “got” library to query an external API got(DISCOVERY_URL, { json: true }) .then((response) => { //…Send responses }).catch((err) => { //…Handle errors }); });
21. Let’s see how this works with a DEMO 21
22. Trace Example on Stackdriver
23. Here’s a list of all requests that hit my application
24. And from here, I can see every single request that went out making it easy to see where the slow requests happen
25. Here’s an upclose overview, notice that the longest request is the one to googleapis
26. And I can also view where the requests came from, the status code and all that
27. “Once a logger, always a logger. The moment you start, you’ll never stop ” - Anonymous
28. Who knows how to debug on the cloud? 28
29. Stackdriver Debug If you’ve ever used Heroku, this is for you! For those of us who have ever deployed our applications to SaaS (Service As A Service) on platforms like Heroku, you’d know how hellish it is to debug when your application doesn’t start. It’s worse than a bad codebase and it happens to everyone.
30. So how does it work? It uses the same concept as any debugger, you set a breakpoint and it takes a snapshot of the current stack trace at that point (variable values, errors, object definitions) and allows you move onto other points you define. Like all stackdriver features, we can use the SDK to make our lives easier
31. //…Left out other dependencies //We add this and that's it //It's that simple require('@google-cloud/debug-agent').start(); // No need for extra configurations, it’s that easy // Once this dependency is in your application, you can easily head to the // dashboard and begin testing your application
32. We’re off to the dashboard again for this one! 32
33. “Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it? ” - Brian. W. Kernighan
34. Now to where it all started from, LOGGING!!!! 34
35. Stackdriver Logging By default, Stackdriver polls logs from all your server instances (App Engine, Compute Engine, Kubernetes Engine etc) and stores them in its logging ingestion system. Sometimes, we might have applications that don’t necessarily run on Google Cloud, so how we fix that? So we use the SDK as before!
36. Never consider your print lines as logs When it comes down to logging, there’s a format to logging that’s very acceptable and this is always implemented by a library for consistency. Never take your print lines to standard output / console as logs, it’s very wrong and makes it hard to know what is an error versus an info. Always use a logging library, for this one I’m using Bunyan.
37. // Imports the Google Cloud client library for Bunyan const {LoggingBunyan} = require('@google-cloud/logging-bunyan’); const loggingBunyan = new LoggingBunyan(); // Create a Bunyan logger that streams to Stackdriver Logging // Logs will be written to: "projects/YOUR_PROJECT_ID/logs/bunyan_log“ const logger = bunyan.createLogger({ // The JSON payload of the log as it appears in Stackdriver Logging // will contain "name": "my-service" name: 'default-service-stackdriver-node-demo’, streams: [ // Log to the console at 'info' and above {stream: process.stdout, level: 'debug'}, // And log to Stackdriver Logging, logging at 'info' and above loggingBunyan.stream('info'), ], });
38. Final demo of the day, then we’d cover more examples on setting up monitoring and alerting from the stackdriver dashboard 38
39. For a recap of today:
40. References and Documentation 40 Stackdriver Dashboard [Requires a gCloud account] https://app.google.stackdriver.com Stackdriver Logging https://cloud.google.com/logging/docs Stackdriver Error Reporting https://cloud.google.com/error-reporting/ docs Stackdriver Debugger https://cloud.google.com/debugger/docs Stackdriver Tracer https://cloud.google.com/trace/docs Stackdriver Profiler [Couldn’t cover this due to time but it’s worth a check] https://cloud.google.com/profiler/docs
41. Bakare Emmanuel, Interswitch Group @tiemmaBakare Location Thank you!
42. Q&A Ask me anything! For example, what is your favorite food? Or preferably, what is your account number? Location