Long-running Docker containers can rapidly accumulate a large number of log lines. These consume storage capacity and reduce performance when accessing and filtering the data. While Docker includes integrated tools for viewing the logs, there’s no built-in mechanism for cleaning them up.
In this article you’ll learn how to clear up the logs of running Docker containers, without restarting or replacing them. You’ll also see some techniques for more efficiently handling large logs while retaining old data.
Understanding the Problem
Docker collects logs from the standard output and error streams of container foreground processes. The docker logs command is used to retrieve these logs but it doesn’t let you delete old lines.
Docker supports many different logging drivers so it’s not possible to offer a standardized clean-up mechanism. This article focuses on the standard json-file logging driver, where log lines are stored in a JSON file on your container host’s filesystem. Refer to your storage driver’s documentation if you need to clean up logs streamed to a remote host.
Log files created by the json-file driver are stored under the /var/lib/docker/containers directory. Each container gets its own log file which is used throughout its lifetime. There’s no log rotation configured by default.
Simply deleting the log file isn’t an effective solution. Docker expects the file to be available continually and won’t recreate it automatically if it’s removed. It’s safer to clear the existing log’s contents to avoid affecting any current writes.
Discovering the Log File’s Path
First find the path to your target container’s log file. You can retrieve the log file path for a container called my-app by running the following command:
$ docker inspect –format=”{{.LogPath}}” my-app
/var/lib/containers/1380d936…-json.log
Clearing the Log File
You can clear out the contents of the log without deleting it by echoing an empty string into its content. The file will be owned by root so you’ll need to perform this operation in a root shell. The following command will empty the log file for you:
$ sudo sh -c ‘echo “” > $(docker inspect –format=”{{.LogPath}}” my-app)’
Shell interpolation is being used to dynamically retrieve the log file path for the my-app container. You could manually substitute in the path retrieved earlier instead.
When you run docker logs my-app, you’ll now see empty output unless the container has resumed writing lines in the interim.
Limiting Log Output
The docker logs command accepts a few arguments that can be used to limit the output from noisy logs. These help mitigate the need to clean the log while preventing your terminal from being inundated with old lines each time you access the file.
- –tail – This flag instructs Docker to emit only a specific number of old log lines, starting with the most recent.
- –until and –since – These flags restrict the output to log lines emitted during a specific time period. They’re useful when you know the approximate timeframe in which an event occurred.
Here are a few examples:
Display the 100 most recent log lines
docker logs my-app –tail 100
Display log lines written in the past hour
docker logs my-app –since 1h
Display up to 100 log lines written in the past hour
docker logs my-app –tail 100 –since 1h
Setting Up Log Rotation
Regularly having to clean up log files in this manner usually signals that your application’s logs are either excessively verbose or there’s simply too much activity to store in one file.
Many Docker logging drivers, including json-file, have optional log rotation support that you can enable globally for the Docker daemon or on a per-container basis.
Daemon settings are configured in /etc/docker/daemon.json. Here’s an example that rotates container logs once they reach 8MB. Up to five files will be retained at any time, with old ones automatically deleted as new rotations occur.
{
“log-opts”: {
“max-size”: “8m”,
“max-file”: “5”
}
}
Restart the Docker daemon to apply the change:
$ sudo service docker restart
Daemon-level rotation applies to all newly created containers. Changes won’t affect any containers that already exist on your host.
Rotation can be configured for individual containers using –log-opts flags. These will override your default Docker daemon settings.
docker run –name app
–log-driver json-file
–log-opts max-size=8M
–log-opts max-file=5
app-image:latest
Summary
Docker container logs can get noisy when busy applications write extensive output. Access to verbose logs is helpful for debugging but they consume storage space, make it harder to sift through data, and could reduce performance.
You can clean up log files on-demand by using shell commands to empty their content. However it’s better practice to set up log rotation so this occurs automatically once files reach a specific size. The docker logs command also supports flags that can help tame unwieldy logs at the point of access.
Source by www.howtogeek.com
