Save 37% off PRO during our Black Friday Sale! »

System.out.println("USE A LOGGER!!!");

System.out.println("USE A LOGGER!!!");

No matter if you are developing a small library, a complex framework or an application for end users - at some point you will come to the point that you want to log information. Here the Java community offers a whole wealth of solutions: Besides java.util.Logging there are libraries like commons-logging, Log4J or Logback. Where are the differences and unique selling points of the frameworks? In addition, actually everyone says that only the use of slf4J is "best practice". But why should I use an abstraction when developing a concrete application? Or does slf4j perhaps offer completely different advantages? And as if that weren't enough questions, with Java 9 a new logger, System.Logger, was quietly integrated into the OpenJDK.
This talk gives an understandable guide through the logging jungle and shows a few practices that help to get the logging of libraries and applications under control. If you're on the verge of going back to System.out out of frustration, this session is for you.

C79a3191aa67cae35282fb129587696b?s=128

Hendrik Ebbers

March 18, 2021
Tweet

Transcript

  1. None
  2. System.out.println( ); "USE A LOGGER!!"

  3. This is a very very very long gag @hendrikEbbers Hendrik

    • Karakun Co-Founder • Founder of JUG Dortmund • JSR EG member • JavaOne Rockstar, Java Champion • AdoptOpenJDK TSC member
  4. This is a very very very long gag @hendrikEbbers Stephan

    • Karakun Co-Founder • SoCraTes CH community member • OpenWebStart committer • Conference speaker
  5. Content

  6. This is a very very very long gag @hendrikEbbers Content

    • Logging 101 • Java Logging & Libraries • Integration & Architecture • Centralized Logging • Outlook - Future of Logging
  7. This is a very very very long gag @hendrikEbbers Content

    Go to www.menti.com and use the code 4490 2536
  8. Logging 101

  9. This is a very very very long gag @hendrikEbbers Definition

    of 'log' • a thick piece of wood cut from a tree • a written record of things that happen, especially an official record of a journey on a ship or in a plane • a logarithm * from the Macmillan Dictionary
  10. This is a very very very long gag @hendrikEbbers Why

    should you log? Log = Events which happened in the past • retrace what has happened • understand why an event occurred • gather information about how the software is used • learn from the past
  11. This is a very very very long gag @hendrikEbbers What

    should you log? • important events • input from outside the software / module • unexpected behavior
  12. This is a very very very long gag @hendrikEbbers What

    should you NOT log? • too many details • events in a loop • sensitive data 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3' 08:34:23 INFO User 'Max' with birthday '01/01/1970' mutates password field with new value '1A3.'
  13. This is a very very very long gag @hendrikEbbers How

    long should you keep a log? • forever • until you are certain you will never ever need the log any more Since you can never be certain: KEEP IT FOREVER
  14. This is a very very very long gag @hendrikEbbers •

    performance • code cluttering LOG.info("We start the transaction"); manager.beginTransaction(); LOG.info("DB query will be executed"); LOG.debug("DB query: select * from users"); long start = now(); users = manager.query("select * from users"); LOG.info("DB query executed"); LOG.debug("DB query executed in " + (now() - start) + " ms"); LOG.info("Found " + users.size() + " entities"); manager.endTransaction(); LOG.info("Transaction done"); Has logging any downsides?
  15. This is a very very very long gag @hendrikEbbers Logging

    Concept • What's your motivation ? • who is reading / monitoring the log ? • what do you want to extract from the logs ? • what do you not want to extract ? • how is logging different in production/test/dev ?
  16. Logging in Java

  17. This is a very very very long gag @hendrikEbbers System.out

    System.out.print("Who needs a logger ??")
  18. This is a very very very long gag @hendrikEbbers Logger

    Architecture
  19. This is a very very very long gag @hendrikEbbers Architecture

    Detail
  20. This is a very very very long gag @hendrikEbbers Data

    from Application • Message • Exception (optional) • Marker (optional)
  21. This is a very very very long gag @hendrikEbbers Data

    from Logger • Logger Name • Log Level (Info, Warning, Debug ...) • Timestamp • Thread • Additional Context
  22. This is a very very very long gag @hendrikEbbers Data

    from Logger • Calling Class • Calling Method • Calling File • Calling Line Number • Calling Stack
  23. This is a very very very long gag @hendrikEbbers Additional

    Context • Context of the Logger • Properties/Attributes of the Logger • Context from the Application (MDC) In general this is based on the current thread
  24. This is a very very very long gag @hendrikEbbers Appender

    - Filter • Filter messages and only append the relevant to the log • By log level • By marker • By any other data provided by the logger
  25. This is a very very very long gag @hendrikEbbers Appender

    - Layout • Layout the message to fit the storage • Plain String / HTML • JSON / XML • GELF (Graylog Extended Log Format) • Optionally group messages to bulks
  26. This is a very very very long gag @hendrikEbbers Appender

    • Send the message to the storage • Console (stdout/stderr) • File (plain text, html, json, xml, etc) • DB • Remote
  27. This is a very very very long gag @hendrikEbbers Logging

    Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 Commons Logging SLF4J Java 8 Java 9 Java 11 TinyLog
  28. This is a very very very long gag @hendrikEbbers Logging

    Frameworks 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11
  29. Let's talk about PERFORMANCE

  30. This is a very very very long gag @hendrikEbbers Performance

    - Create Messages • Guard logging code • Delay string concatenation • Delay arguments calculation LOG.info("Hello " + username + "!"); if (LOG.isInfo()) {LOG.info("Hello " + getUserNameFromServer() + "!");} LOG.info("Hello {}!", userName); LOG.info("Hello {}!", () -> getUserNameFromServer());
  31. This is a very very very long gag @hendrikEbbers Performance

    - Enhance Message • Avoid expensive meta data • Delay calculation of expensive meta data 08:34:23 INFO [com.sample.MyClass] com.sample.MyClass.java:319 Hello World! LOgger name Source Information
  32. This is a very very very long gag @hendrikEbbers Performance

    - Enhance Message • Avoid expensive meta data • Delay calculation of expensive meta data 08:34:23 INFO [com.sample.MyClass] com.sample.MyClass.java:319 Hello World! LOgger name Source Information Do we really need this?
  33. This is a very very very long gag @hendrikEbbers Performance

    - Persist Message • Asynchronous write to the storage • Bulk write to storage
  34. Logging Integration & Architecture

  35. This is a very very very long gag @hendrikEbbers Application

    Architecture • Let's have a look at some samples • As always we start with 'Hello World'
  36. This is a very very very long gag @hendrikEbbers Application

    Architecture HelloLogger.java Just a simple java class
  37. This is a very very very long gag @hendrikEbbers Application

    Architecture HelloLogger.java java.util.logging We use the 'Standard' Java logging
  38. This is a very very very long gag @hendrikEbbers Application

    Architecture package com.karakun; import java.util.logging.Logger; public class HelloLogging { private static final Logger LOG = Logger.getLogger("com.karakun.HelloLogging"); public static void main(final String[] args) { LOG.info("Hello World"); } }
  39. This is a very very very long gag @hendrikEbbers Application

    Architecture HelloLogger.java java.util.logging app-log.txt Level: INFO logging.properties We can configure the logger by using a properties file
  40. This is a very very very long gag @hendrikEbbers Application

    Architecture • Well that worked quite good • Let's have a look at a more complex application
  41. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access security-lib Our application has some dependencies
  42. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access security-lib jul Logging app-log.txt Level: INFO logging.properties We can still configure the logger by using a properties file
  43. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access Log4j security-lib jul Logging app-log.txt Level: INFO logging.properties The Dependency uses another logger to create log messages
  44. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties ... and here we have another logger
  45. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access Log4j security-lib TinyLog jul Logging app-log.txt Level: INFO logging.properties
  46. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access Log4j security-lib TinyLog jul Logging ??????? Level: ? app-log.txt Level: INFO ??????? Level: ? logging.properties Where can we find that Logging?
  47. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application database-access Log4j security-lib TinyLog jul Logging db-log.txt Level: INFO app-log.txt Level: INFO sec-log.txt Level: INFO logging.properties log4j.properties tinylog.properties
  48. This is a very very very long gag @hendrikEbbers Application

    Architecture My Application logging.properties log4j.properties tinylog.properties log4j2.properties db-log.txt app-log.txt sec-log.txt
  49. This is a very very very long gag @hendrikEbbers Application

    Architecture • Providing different configs for all possible logging libs can't be the solution • Let's get back to the table and sketch a good solution
  50. This is a very very very long gag @hendrikEbbers A

    Perfect LOG 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out
  51. This is a very very very long gag 08:34:23 INFO

    Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out @hendrikEbbers A Perfect LOG My Application jul Logging app-log.txt Level: ALL
  52. This is a very very very long gag 08:34:23 INFO

    Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out @hendrikEbbers A Perfect LOG My Application Log4j Logging db-log.txt Level: ALL DB Lib
  53. This is a very very very long gag @hendrikEbbers A

    Perfect LOG 08:34:23 INFO Application starts 08:34:24 DEBUG Connection to DB 'my-db' 08:34:26 DEBUG Database connection created 08:34:28 INFO Application started 08:35:12 INFO User 'Hendrik' tries to log in to the system 08:35:13 WARN Password for user 'Hendrik' is incorrect 08:35:46 INFO User 'Hendrik' tries to log in to the system 08:35:47 INFO User 'Hendrik' logged in 08:36:22 DEBUG User 'Hendrik' opens search dialog 08:36:34 INFO User 'Hendrik' starts search 08:36:34 DEBUG User 'Hendrik' starts search with name='*java*' 08:36:34 DEBUG Executing query on db 08:36:47 DEBUG ResulSet contains 7 entries 08:36:47 DEBUG User 'Hendrik' - Search result count: 7 08:36:48 INFO User 'Hendrik' successfully executed search 08:36:52 INFO User 'Hendrik' logged out How can we achieve this combination in one file?
  54. This is a very very very long gag @hendrikEbbers A

    stupid idea • Maybe we can just configure all logging libs to write in the same file...
  55. This is a very very very long gag @hendrikEbbers A

    stupid idea • Maybe we can just configure all logging libs to write in the same file... Bild: Freepik.com Concurrency issues File Locks Multiple Messages in one line Really a bad idea... Don't do this at home
  56. This is a very very very long gag @hendrikEbbers Logging

    issues • We have several loggers on the classpath • We want to have one central configuration • We want to have one central output • We do not want to miss logging messages from dependencies
  57. This is a very very very long gag @hendrikEbbers Let's

    sketch a solution My Application & Dependencies Logging Framework A Logging Framework B Logging Framework C Central Appender/Config Single Storage collect all messages
  58. This is a very very very long gag @hendrikEbbers SLF4J

    as Logging Facade • The SLF4J projects provide then needed functionalities • Logging facade • Logging bridges / adapters • Logging bindings Common API Forward messages to API/Facade Implementation of API/Facade
  59. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar
  60. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Facade / API
  61. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Adapters
  62. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar log4j-over-slf4j.jar replaces log4j.jar
  63. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar log4j-over-slf4j.jar replaces log4j.jar log4j-over-slf4j.jar contains a custom org.apache.log4j.Logger class
  64. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar Implementation of API & Binding to Logging Framework
  65. This is a very very very long gag @hendrikEbbers Logging

    redirection My Application & Dependencies Commons logging API SLF4J API slf4j-api.jar Log4j API jcl-over-slf4j.jar log4j-over-slf4j.jar Underlying logging framework logback-classic.jar No configuration! logger.properties Only configure The underlying Logger
  66. This is a very very very long gag @hendrikEbbers Solutions

    • There is no 'simple' solution that just fits for all scenarios and setups • Projects like SLF4J provides all that is needed to configure even complex use cases • Knowledge of the libs is needed
  67. This is a very very very long gag @hendrikEbbers Logging

    Facades 2012 2013 2014 2016 2017 2019 2020 2015 2018 Log4j Logback Log4j2 TinyLog Commons Logging SLF4J Java 8 Java 9 Java 11
  68. This is a very very very long gag @hendrikEbbers Java

    System.Logger • Part of OpenJDK (since 9) • See JEP 264 (https://openjdk.java.net/jeps/264) • Minimal Logging API / Facade • Used internally in the OpenJDK • Default adapter is java.util.logging
  69. This is a very very very long gag @hendrikEbbers Java

    System.Logger public class MainApp { private static System.Logger LOGGER = System.getLogger("MainApp"); public static void main(String[] args) { LOGGER.log(Level.ERROR, "error test"); LOGGER.log(Level.INFO, () -> "info test"); } } Remember our performance discussion?
  70. This is a very very very long gag @hendrikEbbers Non-Goals

    of System.Logger • It is not a goal to define a general-purpose interface for logging. The service interface contains only the minimal set of methods that the JDK needs for its own usage. • It is not a goal to support programmatic logging configuration (setting levels, destination files, etc...) via the API.
  71. This is a very very very long gag @hendrikEbbers Best

    Practice for System.Logger • Do not use it as your application logger • Configure a bridge within the logger facade you are using (by using SPI) • Use it directly in small libraries to remove dependencies (no slf4j-api is needed anymore)
  72. Centralized Logging

  73. This is a very very very long gag @hendrikEbbers Application

    landscapes today • We now have a real good solutions to create and store the logging for one application • In a real world scenario this is not enough! • Let's take this a step further...
  74. This is a very very very long gag @hendrikEbbers Modern

    usecases • Will I have 1 log file for each node if my application is clustered? • Can I get my application log synchronised with the log of the database or the docker container? • How can I search in my logs and display all logs for 1 specific request?
  75. What happens if we use Microservices? 😱

  76. This is a very very very long gag @hendrikEbbers Modern

    usecases LOG LOG LOG LOG LOG LOG LOG LOG
  77. This is a very very very long gag @hendrikEbbers Let's

    sketch a solution • We need to find a way to centralise our logging • All log messages must be pushed / pulled to one central place • Based on the amount of logging this needs a special solution • The solutions needs good search functionality
  78. This is a very very very long gag @hendrikEbbers Centralized

    Log Management Logger Logentries json-file System Journal Syslog fluentd Windows Event Store Splunk GELF Amazon CloudWatch Google Cloud UDP / TCP TCP / HTTPS TCP / HTTPS TCP / HTTPS TCP / (HTTP & HTTPS) TCP UDP / TCP / TLS HTTP / HTTPS
  79. This is a very very very long gag @hendrikEbbers Graylog

    / GELF example Applications GELF Docker Containers GELF UDP / TCP UDP / TCP DB / System / Services GELF UDP / TCP JSON Format
  80. None
  81. None
  82. Remember Markers & MDC?

  83. Next steps

  84. This is a very very very long gag @hendrikEbbers Tracing

    • Tracing libraries provide important metadata in a distributed environment • Metadata is added to HTTP header • Follow a request over several services Use MDC & Metadata to see all log messages of this flow
  85. This is a very very very long gag @hendrikEbbers Analyze

    Logs using AI • Use 'log intelligence' to get better feedback • Detect issues automatically • Detect anomalies before they create issues • Understand and react to users and their behavior
  86. Stay safe & healthy

  87. • Use a Logging Facade • Use Centralized Logging (even

    of you have only 1 log file) • Use MDC and Markers