Rolling Log Files with Logback

 Imagine you are developing a long running service in Java (most likely you already have one) and found a bug in production at some point, how can you identify the root cause of the bug? The answer is simple: analyze your logs. If you already had your logs written to a file you are in a good position. If not, you may have to rely on the mercy of Docker container logs (if your service is containerized) or else you are doomed. 

Rolling Log Files with Logback

Even if you have logs appended to a log file, you must consider rolling log files to keep them small and concise. If not, be prepared to open a megabyte/gigabyte size file with millions of lines and find a needle in a haystack. Enabling rolling file log in your application is not a big deal if you know the configuration. However finding the right configuration every time you need it requires a lot of web surfing.

In this article, I'm sharing the logback configuration file I'm using in my recent Spring Boot applications as a go to template. This configuration file has evolved over couple of Spring Boot projects and shaped along the way to support the following features:

  1. Log file path from environment variables. This is handy if you containerize your application. You can control where to write the log from Docker environment variables.
  2. Rolling log file every day. If your application logs more often, consider rolling files every six hour or so.
  3. Write the warning and error logs to the stdout. If you are using a log analyzer to check the health of your application, you can limit the log analyzer to to only look at the stdout.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>

<configuration>

    <property name="logs_dir" value="${MY_APP_LOGS_DIR:-logs}"/>
    <property name="logs_file" value="${logs_dir}/my-app.log"/>
    <property name="logs_file_pattern" value="${logs_dir}/my-app-%d{yyyy-MM-dd_HH}.log"/>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <target>System.out</target>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>DEBUG</level>
            <onMatch>ACCEPT</onMatch>
        </filter>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>INFO</level>
            <onMatch>ACCEPT</onMatch>
        </filter>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>TRACE</level>
            <onMatch>ACCEPT</onMatch>
        </filter>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>WARN</level>
            <onMatch>DENY</onMatch>
        </filter>
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>DENY</onMatch>
        </filter>

        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
        <target>System.err</target>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>WARN</level>
        </filter>
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${logs_file}</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${logs_file_pattern}</fileNamePattern>
            <maxHistory>60</maxHistory>
        </rollingPolicy>

        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <root>
        <level value="INFO"/>
        <appender-ref ref="FILE"/>
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="STDERR"/>
    </root>
</configuration>

How to Use:

Step 1:

Make sure that you are using Apache Logback to log events. The above configuration is for Logback. If you are using Log4j, please find the equivalent configuration for Log4j.

Step 2:

Copy the above configuration and add it to your project under src/main/resources/logback.xml file.

Step 3:

Replace the environment variable name (MY_APP_LOGS_DIR) and the log file names (my-app) by something relevant to your application.


Now you are good to go. Simply run the application and the Logback will take care of your logs.

Latest
Previous
Next Post »

Contact Form

Name

Email *

Message *