System.out.println("USE A LOGGER!!!");

No matter if you are developing a small library, a complex framework or an application for end users - at some point you will come to the point that you want to log information. Here the Java community offers a whole wealth of solutions: Besides java.util.Logging there are libraries like commons-logging, Log4J or Logback. Where are the differences and unique selling points of the frameworks? In addition, actually everyone says that only the use of slf4J is "best practice". But why should I use an abstraction when developing a concrete application? Or does slf4j perhaps offer completely different advantages? And as if that weren't enough questions, with Java 9 a new logger, System.Logger, was quietly integrated into the OpenJDK.
This talk gives an understandable guide through the logging jungle and shows a few practices that help to get the logging of libraries and applications under control. If you're on the verge of going back to System.out out of frustration, this session is for you.

Transcript

2. System.out.println( ); "USE A LOGGER!!"
3. This is a very very very long gag @hendrikEbbers Hendrik • Karakun Co-Founder • Founder of JUG Dortmund • JSR EG member • JavaOne Rockstar, Java Champion • AdoptOpenJDK TSC member
4. This is a very very very long gag @hendrikEbbers Stephan • Karakun Co-Founder • SoCraTes CH community member • OpenWebStart committer • Conference speaker
5. Content
6. This is a very very very long gag @hendrikEbbers Content • Logging 101 • Java Logging & Libraries • Integration & Architecture • Centralized Logging • Outlook - Future of Logging
7. This is a very very very long gag @hendrikEbbers Content Go to www.menti.com and use the code 4490 2536
8. Logging 101
9. This is a very very very long gag @hendrikEbbers Definition of 'log' • a thick piece of wood cut from a tree • a written record of things that happen, especially an official record of a journey on a ship or in a plane • a logarithm * from the Macmillan Dictionary
10. This is a very very very long gag @hendrikEbbers Why should you log? Log = Events which happened in the past • retrace what has happened • understand why an event occurred • gather information about how the software is used • learn from the past
11. This is a very very very long gag @hendrikEbbers What should you log? • important events • input from outside the software / module • unexpected behavior
12. This is a very very very long gag @hendrikEbbers What should you NOT log? • too many details • events in a loop • sensitive data 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3.'
13. This is a very very very long gag @hendrikEbbers How long should you keep a log? • forever • until you are certain you will never ever need the log any more Since you can never be certain: KEEP IT FOREVER
14. This is a very very very long gag @hendrikEbbers • performance • code cluttering LOG.info("We start the transaction"); manager.beginTransaction(); LOG.info("DB query will be executed"); LOG.debug("DB query: select * from users"); long start = now(); users = manager.query("select * from users"); LOG.info("DB query executed"); LOG.debug("DB query executed in " + (now() - start) + " ms"); LOG.info("Found " + users.size() + " entities"); manager.endTransaction(); LOG.info("Transaction done"); Has logging any downsides?
15. This is a very very very long gag @hendrikEbbers Logging Concept • What's your motivation ? • who is reading / monitoring the log ? • what do you want to extract from the logs ? • what do you not want to extract ? • how is logging different in production/test/dev ?
16. Logging in Java
17. This is a very very very long gag @hendrikEbbers System.out System.out.print("Who needs a logger ??")
18. This is a very very very long gag @hendrikEbbers Logger Architecture
19. This is a very very very long gag @hendrikEbbers Architecture Detail
20. This is a very very very long gag @hendrikEbbers Data from Application • Message • Exception (optional) • Marker (optional)
21. This is a very very very long gag @hendrikEbbers Data from Logger • Logger Name • Log Level (Info, Warning, Debug ...) • Timestamp • Thread • Additional Context
22. This is a very very very long gag @hendrikEbbers Data from Logger • Calling Class • Calling Method • Calling File • Calling Line Number • Calling Stack
23. This is a very very very long gag @hendrikEbbers Additional Context • Context of the Logger • Properties/Attributes of the Logger • Context from the Application (MDC) In general this is based on the current thread
24. This is a very very very long gag @hendrikEbbers Appender - Filter • Filter messages and only append the relevant to the log • By log level • By marker • By any other data provided by the logger
25. This is a very very very long gag @hendrikEbbers Appender - Layout • Layout the message to fit the storage • Plain String / HTML • JSON / XML • GELF (Graylog Extended Log Format) • Optionally group messages to bulks
26. This is a very very very long gag @hendrikEbbers Appender • Send the message to the storage • Console (stdout/stderr) • File (plain text, html, json, xml, etc) • DB • Remote
27. This is a very very very long gag @hendrikEbbers Logging Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 Commons Logging SLF4J Java 8 Java 9 Java 11 TinyLog
28. This is a very very very long gag @hendrikEbbers Logging Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11
29. Let's talk about PERFORMANCE
30. This is a very very very long gag @hendrikEbbers Performance - Create Messages • Guard logging code • Delay string concatenation • Delay arguments calculation LOG.info("Hello " + username + "!"); if (LOG.isInfo()) {LOG.info("Hello " + getUserNameFromServer() + "!");} LOG.info("Hello {}!", userName); LOG.info("Hello {}!", () -> getUserNameFromServer());
31. This is a very very very long gag @hendrikEbbers Performance - Enhance Message • Avoid expensive meta data • Delay calculation of expensive meta data 08:34:23 INFO [com.sample.MyClass] com.sample.MyClass.java:319 Hello World! LOgger name Source Information
32. This is a very very very long gag @hendrikEbbers Performance - Enhance Message • Avoid expensive meta data • Delay calculation of expensive meta data 08:34:23 INFO [com.sample.MyClass] com.sample.MyClass.java:319 Hello World! LOgger name Source Information Do we really need this?
33. This is a very very very long gag @hendrikEbbers Performance - Persist Message • Asynchronous write to the storage • Bulk write to storage
34. Logging Integration & Architecture
35. This is a very very very long gag @hendrikEbbers Application Architecture • Let's have a look at some samples • As always we start with 'Hello World'
36. This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java Just a simple java class
37. This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java java.util.logging We use the 'Standard' Java logging
38. This is a very very very long gag @hendrikEbbers Application Architecture package com.karakun; import java.util.logging.Logger; public class HelloLogging { private static final Logger LOG = Logger.getLogger("com.karakun.HelloLogging"); public static void main(final String[] args) { LOG.info("Hello World"); } }
39. This is a very very very long gag @hendrikEbbers Application Architecture HelloLogger.java java.util.logging app-log.txt Level: INFO logging.properties We can configure the logger by using a properties file
40. This is a very very very long gag @hendrikEbbers Application Architecture • Well that worked quite good • Let's have a look at a more complex application
41. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access security-lib Our application has some dependencies
42. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access security-lib jul Logging app-log.txt Level: INFO logging.properties We can still configure the logger by using a properties file
43. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib jul Logging app-log.txt Level: INFO logging.properties The Dependency uses another logger to create log messages
44. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties ... and here we have another logger
45. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties
46. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging ??????? Level: ? app-log.txt Level: INFO ??????? Level: ? logging.properties Where can we find that Logging?
47. This is a very very very long gag @hendrikEbbers Application Architecture My Application database-access Log4j security-lib TinyLog jul Logging db-log.txt Level: INFO app-log.txt Level: INFO sec-log.txt Level: INFO logging.properties log4j.properties tinylog.properties
48. This is a very very very long gag @hendrikEbbers Application Architecture My Application logging.properties log4j.properties tinylog.properties log4j2.properties db-log.txt app-log.txt sec-log.txt
49. This is a very very very long gag @hendrikEbbers Application Architecture • Providing different configs for all possible logging libs can't be the solution • Let's get back to the table and sketch a good solution
50. This is a very very very long gag @hendrikEbbers A Perfect LOG 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out
51. This is a very very very long gag 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out @hendrikEbbers A Perfect LOG My Application jul Logging app-log.txt Level: ALL
52. This is a very very very long gag 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out @hendrikEbbers A Perfect LOG My Application Log4j Logging db-log.txt Level: ALL DB Lib
53. This is a very very very long gag @hendrikEbbers A Perfect LOG 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out How can we achieve this combination in one file?
54. This is a very very very long gag @hendrikEbbers A stupid idea • Maybe we can just configure all logging libs to write in the same file...
55. This is a very very very long gag @hendrikEbbers A stupid idea • Maybe we can just configure all logging libs to write in the same file... Bild: Freepik.com Concurrency issues File Locks Multiple Messages in one line Really a bad idea... Don't do this at home
56. This is a very very very long gag @hendrikEbbers Logging issues • We have several loggers on the classpath • We want to have one central configuration • We want to have one central output • We do not want to miss logging messages from dependencies
57. This is a very very very long gag @hendrikEbbers Let's sketch a solution My Application & Dependencies Logging Framework A Logging Framework B Logging Framework C Central Appender/Config Single Storage collect all messages
58. This is a very very very long gag @hendrikEbbers SLF4J as Logging Facade • The SLF4J projects provide then needed functionalities • Logging facade • Logging bridges / adapters • Logging bindings Common API Forward messages to API/Facade Implementation of API/Facade
59. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar
60. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Facade / API
61. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Adapters
62. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar log4j-over-slf4j.jar replaces log4j.jar
63. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar log4j-over-slf4j.jar replaces log4j.jar log4j-over-slf4j.jar contains a custom org.apache.log4j.Logger class
64. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Implementation of API & Binding to Logging Framework
65. This is a very very very long gag @hendrikEbbers Logging redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar No configuration! logger.properties Only configure The underlying Logger
66. This is a very very very long gag @hendrikEbbers Solutions • There is no 'simple' solution that just fits for all scenarios and setups • Projects like SLF4J provides all that is needed to configure even complex use cases • Knowledge of the libs is needed
67. This is a very very very long gag @hendrikEbbers Logging Facades 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11
68. This is a very very very long gag @hendrikEbbers Java System.Logger • Part of OpenJDK (since 9) • See JEP 264 (https://openjdk.java.net/jeps/264) • Minimal Logging API / Facade • Used internally in the OpenJDK • Default adapter is java.util.logging
69. This is a very very very long gag @hendrikEbbers Java System.Logger public class MainApp { private static System.Logger LOGGER = System.getLogger("MainApp"); public static void main(String[] args) { LOGGER.log(Level.ERROR, "error test"); LOGGER.log(Level.INFO, () -> "info test"); } } Remember our performance discussion?
70. This is a very very very long gag @hendrikEbbers Non-Goals of System.Logger • It is not a goal to define a general-purpose interface for logging. The service interface contains only the minimal set of methods that the JDK needs for its own usage. • It is not a goal to support programmatic logging configuration (setting levels, destination files, etc...) via the API.
71. This is a very very very long gag @hendrikEbbers Best Practice for System.Logger • Do not use it as your application logger • Configure a bridge within the logger facade you are using (by using SPI) • Use it directly in small libraries to remove dependencies (no slf4j-api is needed anymore)
72. Centralized Logging
73. This is a very very very long gag @hendrikEbbers Application landscapes today • We now have a real good solutions to create and store the logging for one application • In a real world scenario this is not enough! • Let's take this a step further...
74. This is a very very very long gag @hendrikEbbers Modern usecases • Will I have 1 log file for each node if my application is clustered? • Can I get my application log synchronised with the log of the database or the docker container? • How can I search in my logs and display all logs for 1 specific request?
75. What happens if we use Microservices? 😱
76. This is a very very very long gag @hendrikEbbers Modern usecases LOG LOG LOG LOG LOG LOG LOG LOG
77. This is a very very very long gag @hendrikEbbers Let's sketch a solution • We need to find a way to centralise our logging • All log messages must be pushed / pulled to one central place • Based on the amount of logging this needs a special solution • The solutions needs good search functionality
78. This is a very very very long gag @hendrikEbbers Centralized Log Management Logger Logentries json-file System Journal Syslog fluentd Windows Event Store Splunk GELF Amazon CloudWatch Google Cloud UDP / TCP TCP / HTTPS TCP / HTTPS TCP / HTTPS TCP / (HTTP & HTTPS) TCP UDP / TCP / TLS HTTP / HTTPS
79. This is a very very very long gag @hendrikEbbers Graylog / GELF example Applications GELF Docker Containers GELF UDP / TCP UDP / TCP DB / System / Services GELF UDP / TCP JSON Format
82. Remember Markers & MDC?
83. Next steps
84. This is a very very very long gag @hendrikEbbers Tracing • Tracing libraries provide important metadata in a distributed environment • Metadata is added to HTTP header • Follow a request over several services Use MDC & Metadata to see all log messages of this flow
85. This is a very very very long gag @hendrikEbbers Analyze Logs using AI • Use 'log intelligence' to get better feedback • Detect issues automatically • Detect anomalies before they create issues • Understand and react to users and their behavior
86. Stay safe & healthy
87. • Use a Logging Facade • Use Centralized Logging (even of you have only 1 log file) • Use MDC and Markers