System.out.println("USE A LOGGER!!!");

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

System.out.println( ); "USE A LOGGER!!"

Slide 3

Slide 3 text

This is a very very very long gag @hendrikEbbers Hendrik • Karakun Co-Founder • Founder of JUG Dortmund • JSR EG member • JavaOne Rockstar, Java Champion • AdoptOpenJDK TSC member

Slide 4

Slide 4 text

This is a very very very long gag @hendrikEbbers Stephan • Karakun Co-Founder • SoCraTes CH community member • OpenWebStart committer • Conference speaker

Slide 5

Slide 5 text

Content

Slide 6

Slide 6 text

This is a very very very long gag @hendrikEbbers Content • Logging 101 • Java Logging & Libraries • Integration & Architecture • Centralized Logging • Outlook - Future of Logging

Slide 7

Slide 7 text

This is a very very very long gag @hendrikEbbers Content Go to www.menti.com and use the code 4490 2536

Slide 8

Slide 8 text

Logging 101

Slide 9

Slide 9 text

This is a very very very long gag @hendrikEbbers Deﬁnition of 'log' • a thick piece of wood cut from a tree • a written record of things that happen, especially an ofﬁcial record of a journey on a ship or in a plane • a logarithm * from the Macmillan Dictionary

Slide 10

Slide 10 text

This is a very very very long gag @hendrikEbbers Why should you log? Log = Events which happened in the past • retrace what has happened • understand why an event occurred • gather information about how the software is used • learn from the past

Slide 11

Slide 11 text

This is a very very very long gag @hendrikEbbers What should you log? • important events • input from outside the software / module • unexpected behavior

Slide 12

Slide 12 text

This is a very very very long gag @hendrikEbbers What should you NOT log? • too many details • events in a loop • sensitive data 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3.'

Slide 13

Slide 13 text

This is a very very very long gag @hendrikEbbers How long should you keep a log? • forever • until you are certain you will never ever need the log any more Since you can never be certain: KEEP IT FOREVER

Slide 14

Slide 14 text

This is a very very very long gag @hendrikEbbers • performance • code cluttering LOG.info("We start the transaction"); manager.beginTransaction(); LOG.info("DB query will be executed"); LOG.debug("DB query: select * from users"); long start = now(); users = manager.query("select * from users"); LOG.info("DB query executed"); LOG.debug("DB query executed in " + (now() - start) + " ms"); LOG.info("Found " + users.size() + " entities"); manager.endTransaction(); LOG.info("Transaction done"); Has logging any downsides?

Slide 15

Slide 15 text

This is a very very very long gag @hendrikEbbers Logging Concept • What's your motivation ? • who is reading / monitoring the log ? • what do you want to extract from the logs ? • what do you not want to extract ? • how is logging different in production/test/dev ?

Slide 16

Slide 16 text

Logging in Java

Slide 17

Slide 17 text

This is a very very very long gag @hendrikEbbers System.out System.out.print("Who needs a logger ??")

Slide 18

Slide 18 text

This is a very very very long gag @hendrikEbbers Logger Architecture

Slide 19

Slide 19 text

This is a very very very long gag @hendrikEbbers Architecture Detail

Slide 20

Slide 20 text

This is a very very very long gag @hendrikEbbers Data from Application • Message • Exception (optional) • Marker (optional)

Slide 21

Slide 21 text

This is a very very very long gag @hendrikEbbers Data from Logger • Logger Name • Log Level (Info, Warning, Debug ...) • Timestamp • Thread • Additional Context

Slide 22

Slide 22 text

This is a very very very long gag @hendrikEbbers Data from Logger • Calling Class • Calling Method • Calling File • Calling Line Number • Calling Stack

Slide 23

Slide 23 text

This is a very very very long gag @hendrikEbbers Additional Context • Context of the Logger • Properties/Attributes of the Logger • Context from the Application (MDC) In general this is based on the current thread

Slide 24

Slide 24 text

This is a very very very long gag @hendrikEbbers Appender - Filter • Filter messages and only append the relevant to the log • By log level • By marker • By any other data provided by the logger

Slide 25

Slide 25 text

This is a very very very long gag @hendrikEbbers Appender - Layout • Layout the message to ﬁt the storage • Plain String / HTML • JSON / XML • GELF (Graylog Extended Log Format) • Optionally group messages to bulks

Slide 26

Slide 26 text

This is a very very very long gag @hendrikEbbers Appender • Send the message to the storage • Console (stdout/stderr) • File (plain text, html, json, xml, etc) • DB • Remote

Slide 27

Slide 27 text

This is a very very very long gag @hendrikEbbers Logging Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 Commons Logging SLF4J Java 8 Java 9 Java 11 TinyLog

Slide 28

Slide 28 text

This is a very very very long gag @hendrikEbbers Logging Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11

Slide 29

Slide 29 text

Let's talk about PERFORMANCE

Slide 30

Slide 30 text

This is a very very very long gag @hendrikEbbers Performance - Create Messages • Guard logging code • Delay string concatenation • Delay arguments calculation LOG.info("Hello " + username + "!"); if (LOG.isInfo()) {LOG.info("Hello " + getUserNameFromServer() + "!");} LOG.info("Hello {}!", userName); LOG.info("Hello {}!", () -> getUserNameFromServer());

Slide 31

Slide 31 text

This is a very very very long gag @hendrikEbbers Performance - Enhance Message • Avoid expensive meta data • Delay calculation of expensive meta data 08:34:23 INFO [com.sample.MyClass] com.sample.MyClass.java:319 Hello World! LOgger name Source Information

Slide 32

Slide 32 text

Slide 33

Slide 33 text

This is a very very very long gag @hendrikEbbers Performance - Persist Message • Asynchronous write to the storage • Bulk write to storage

Slide 34

Slide 34 text

Logging Integration & Architecture

Slide 35

Slide 35 text

This is a very very very long gag @hendrikEbbers Application Architecture • Let's have a look at some samples • As always we start with 'Hello World'

Slide 36

Slide 36 text

This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java Just a simple java class

Slide 37

Slide 37 text

This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java java.util.logging We use the 'Standard' Java logging

Slide 38

Slide 38 text

This is a very very very long gag @hendrikEbbers Application Architecture package com.karakun; import java.util.logging.Logger; public class HelloLogging { private static final Logger LOG = Logger.getLogger("com.karakun.HelloLogging"); public static void main(final String[] args) { LOG.info("Hello World"); } }

Slide 39

Slide 39 text

This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java java.util.logging app-log.txt Level: INFO logging.properties We can configure the logger by using a properties file

Slide 40

Slide 40 text

This is a very very very long gag @hendrikEbbers Application Architecture • Well that worked quite good • Let's have a look at a more complex application

Slide 41

Slide 41 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access security-lib Our application has some dependencies

Slide 42

Slide 42 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access security-lib jul Logging app-log.txt Level: INFO logging.properties We can still configure the logger by using a properties file

Slide 43

Slide 43 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib jul Logging app-log.txt Level: INFO logging.properties The Dependency uses another logger to create log messages

Slide 44

Slide 44 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties ... and here we have another logger

Slide 45

Slide 45 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties

Slide 46

Slide 46 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging ??????? Level: ? app-log.txt Level: INFO ??????? Level: ? logging.properties Where can we find that Logging?

Slide 47

Slide 47 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging db-log.txt Level: INFO app-log.txt Level: INFO sec-log.txt Level: INFO logging.properties log4j.properties tinylog.properties

Slide 48

Slide 48 text

This is a very very very long gag @hendrikEbbers Application Architecture My Application logging.properties log4j.properties tinylog.properties log4j2.properties db-log.txt app-log.txt sec-log.txt

Slide 49

Slide 49 text

This is a very very very long gag @hendrikEbbers Application Architecture • Providing different conﬁgs for all possible logging libs can't be the solution • Let's get back to the table and sketch a good solution

Slide 50

Slide 50 text

This is a very very very long gag @hendrikEbbers A Perfect LOG 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out

Slide 51

Slide 51 text

This is a very very very long gag 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out @hendrikEbbers A Perfect LOG My Application jul Logging app-log.txt Level: ALL

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

This is a very very very long gag @hendrikEbbers A stupid idea • Maybe we can just conﬁgure all logging libs to write in the same ﬁle...

Slide 55

Slide 55 text

This is a very very very long gag @hendrikEbbers A stupid idea • Maybe we can just conﬁgure all logging libs to write in the same ﬁle... Bild: Freepik.com Concurrency issues File Locks Multiple Messages in one line Really a bad idea... Don't do this at home

Slide 56

Slide 56 text

This is a very very very long gag @hendrikEbbers Logging issues • We have several loggers on the classpath • We want to have one central conﬁguration • We want to have one central output • We do not want to miss logging messages from dependencies

Slide 57

Slide 57 text

This is a very very very long gag @hendrikEbbers Let's sketch a solution My Application & Dependencies Logging Framework A Logging Framework B Logging Framework C Central Appender/Config Single Storage collect all messages

Slide 58

Slide 58 text

This is a very very very long gag @hendrikEbbers SLF4J as Logging Facade • The SLF4J projects provide then needed functionalities • Logging facade • Logging bridges / adapters • Logging bindings Common API Forward messages to API/Facade Implementation of API/Facade

Slide 59

Slide 59 text

Slide 60

Slide 60 text

This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Facade / API

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Slide 66

Slide 66 text

This is a very very very long gag @hendrikEbbers Solutions • There is no 'simple' solution that just ﬁts for all scenarios and setups • Projects like SLF4J provides all that is needed to conﬁgure even complex use cases • Knowledge of the libs is needed

Slide 67

Slide 67 text

This is a very very very long gag @hendrikEbbers Logging Facades 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11

Slide 68

Slide 68 text

This is a very very very long gag @hendrikEbbers Java System.Logger • Part of OpenJDK (since 9) • See JEP 264 (https://openjdk.java.net/jeps/264) • Minimal Logging API / Facade • Used internally in the OpenJDK • Default adapter is java.util.logging

Slide 69

Slide 69 text

This is a very very very long gag @hendrikEbbers Java System.Logger public class MainApp { private static System.Logger LOGGER = System.getLogger("MainApp"); public static void main(String[] args) { LOGGER.log(Level.ERROR, "error test"); LOGGER.log(Level.INFO, () -> "info test"); } } Remember our performance discussion?

Slide 70

Slide 70 text

This is a very very very long gag @hendrikEbbers Non-Goals of System.Logger • It is not a goal to define a general-purpose interface for logging. The service interface contains only the minimal set of methods that the JDK needs for its own usage. • It is not a goal to support programmatic logging configuration (setting levels, destination files, etc...) via the API.

Slide 71

Slide 71 text

This is a very very very long gag @hendrikEbbers Best Practice for System.Logger • Do not use it as your application logger • Conﬁgure a bridge within the logger facade you are using (by using SPI) • Use it directly in small libraries to remove dependencies (no slf4j-api is needed anymore)

Slide 72

Slide 72 text

Centralized Logging

Slide 73

Slide 73 text

This is a very very very long gag @hendrikEbbers Application landscapes today • We now have a real good solutions to create and store the logging for one application • In a real world scenario this is not enough! • Let's take this a step further...

Slide 74

Slide 74 text

This is a very very very long gag @hendrikEbbers Modern usecases • Will I have 1 log ﬁle for each node if my application is clustered? • Can I get my application log synchronised with the log of the database or the docker container? • How can I search in my logs and display all logs for 1 speciﬁc request?

Slide 75

Slide 75 text

What happens if we use Microservices? 😱

Slide 76

Slide 76 text

This is a very very very long gag @hendrikEbbers Modern usecases LOG LOG LOG LOG LOG LOG LOG LOG

Slide 77

Slide 77 text

This is a very very very long gag @hendrikEbbers Let's sketch a solution • We need to ﬁnd a way to centralise our logging • All log messages must be pushed / pulled to one central place • Based on the amount of logging this needs a special solution • The solutions needs good search functionality

Slide 78

Slide 78 text

This is a very very very long gag @hendrikEbbers Centralized Log Management Logger Logentries json-file System Journal Syslog fluentd Windows Event Store Splunk GELF Amazon CloudWatch Google Cloud UDP / TCP TCP / HTTPS TCP / HTTPS TCP / HTTPS TCP / (HTTP & HTTPS) TCP UDP / TCP / TLS HTTP / HTTPS

Slide 79

Slide 79 text

This is a very very very long gag @hendrikEbbers Graylog / GELF example Applications GELF Docker Containers GELF UDP / TCP UDP / TCP DB / System / Services GELF UDP / TCP JSON Format

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

No content

Slide 82

Slide 82 text

Remember Markers & MDC?

Slide 83

Slide 83 text

Next steps

Slide 84

Slide 84 text

This is a very very very long gag @hendrikEbbers Tracing • Tracing libraries provide important metadata in a distributed environment • Metadata is added to HTTP header • Follow a request over several services Use MDC & Metadata to see all log messages of this flow

Slide 85

Slide 85 text

This is a very very very long gag @hendrikEbbers Analyze Logs using AI • Use 'log intelligence' to get better feedback • Detect issues automatically • Detect anomalies before they create issues • Understand and react to users and their behavior

Slide 86

Slide 86 text

Stay safe & healthy

Slide 87

Slide 87 text

• Use a Logging Facade • Use Centralized Logging (even of you have only 1 log ﬁle) • Use MDC and Markers