Configuring Logging
Hazelcast has a flexible logging configuration and does not depend on any logging framework except JDK logging. It has built-in adapters for a number of logging frameworks and it also supports custom loggers by providing logging interfaces.
Locating the Logs
If you start a member using the hz-start
script from the Hazelcast distribution package, the logs are written
to <HAZELCAST HOME>/logs
directory by default. The logs are configured to roll over daily.
Note that the logs are also shown in the terminal where you start the member while it is up and running.
In case you use Docker to start a member, the logs are written to stdout.
You can use the docker logs <Hazelcast container ID>
command to see the member logs at any point in time. The container ID can be seen using docker ps
.
You can also use docker logs -f <Hazelcast container ID>
to follow the logs in real-time.
Setting Logging Type
To use the built-in adapters, set the hazelcast.logging.type
property
to one of the predefined types below:
-
jdk: JDK logging (default)
-
log4j: Log4j
-
log4j2: Log4j2
-
slf4j: Slf4j
-
none: disable logging
The following shows how you can set this property:
If you choose to use log4j , log4j2 , or slf4j , you should include
the proper dependencies in the classpath.
|
hazelcast:
properties:
hazelcast.logging.type: log4j
<hazelcast>
...
<properties>
<property name="hazelcast.logging.type">log4j</property>
</properties>
...
</hazelcast>
Config config = new Config() ; config.setProperty( "hazelcast.logging.type", "log4j" );
java -Dhazelcast.logging.type=log4j
System.setProperty( "hazelcast.logging.type", "log4j" );
Logging Pattern
The default log output resembles the following example:
2022-02-16 11:40:52,463 [ INFO] [main] [c.h.system]: [172.18.0.2]:5701 [hello-world] [5.2.3] Copyright (c) 2008-2022, Hazelcast, Inc. All Rights Reserved.
2022-02-16 11:40:52,463 [ INFO] [main] [c.h.system]: [172.18.0.2]:5701 [hello-world] [5.2.3] Hazelcast Platform 5.2.3 (20220210 - 1d718cf) starting at [172.18.0.2]:5701
2022-02-16 11:40:52,463 [ INFO] [main] [c.h.system]: [172.18.0.2]:5701 [hello-world] [5.2.3] Cluster name: hello-world
The logging pattern is shown below.
[Date and time] [Log level] [Thread name] [Logger Class Name] [Member IP and port] [Cluster name] [Platform version] [Log message]
You can customize the default logging pattern using the LOGGING_PATTERN
environment variable while starting a member.
Here is an example usage.
LOGGING_PATTERN='{"time":"%date{ISO8601}", "logger": "%logger{36}", "level": "%level", "msg": "%enc{%m %xEx}{JSON}"}%n' bin/hz start
docker run \
-e LOGGING_PATTERN='{"time":"%date{ISO8601}", "logger": "%logger{36}", "level": "%level", "msg": "%enc{%m %xEx}{JSON}"}%n' \
hazelcast/hazelcast:5.2.3
Specifying Logging Configuration File
You can specify the logging configuration file using the LOGGING_CONFIG
environment variable. If it is not set, the default <HAZELCAST HOME>/config/log4j2.properties
is used;
you can change the logging configuration by modifying this file.
Here is an example usage.
Using JSON Template
You can use JSON templates for the member log files.
The JSON logging configuration file built on the default JSON template layout is <HAZELCAST HOME>/config/log4j2-json.properties
.
You can use a different JSON template via the LOGGING_JSON_TEMPLATE
environment variable. Here is an example usage.
LOGGING_CONFIG=log4j2-json.properties LOGGING_JSON_TEMPLATE="classpath:EcsLayout.json" bin/hz start
docker run \
-e LOGGING_CONFIG=config/log4j2-json.properties \
-e LOGGING_JSON_TEMPLATE="classpath:EcsLayout.json" \
hazelcast/hazelcast:5.2.3
See Event Templates for available templates.
Changing Log Levels for JDK Logging
As mentioned in the introduction of this section above, the default logging type of Hazelcast is JDK logging.
The default logging level is INFO
.
You can change the JDK logging level as follows while starting a member.
Dynamically Changing Log Levels
You can change log levels without the need of restarting the cluster members. This may be useful while monitoring or diagnosing the events in your cluster.
This feature is supported for the default (JDK/JUL), Log4j, and Log4j2 frameworks. The Slf4j framework is not supported since it does not provide any log level changing APIs.
You can use either of the following ways to dynamically change the level of your cluster’s logs:
-
Using JMX API: The logging service exposes its JMX MBean as
LoggingServiceMBean
. You can retrieve, set, and reset the level. See the Monitoring with JMX section. -
Using REST API: You can use the
/hazelcast/rest/log-level
REST endpoint to retrieve (GET
), set (POST
), and reset (DELETE
) the level. See the REST Endpoint Groups section.
Logging for Client and Embedded Mode
When using Hazelcast through the client or in embedded mode, Hazelcast doesn’t automatically add any dependencies to any logging framework and allows configuration of which facade the logging should be done through.
To configure the logging facade to use, you need to set a property in the configuration file:
hazelcast-client:
properties:
hazelcast.logging.type: log4j2
Alternatively, you can use the system property
-Dhazelcast.logging.type
to configure the logging framework to use.
Using a Custom Logger
If the provided logging mechanisms are not satisfactory, you can implement
your own using the custom logging feature. To use it, implement the
com.hazelcast.logging.LoggerFactory
and com.hazelcast.logging.ILogger
interfaces and set the system property hazelcast.logging.class
as your
custom LoggerFactory
class name.
-Dhazelcast.logging.class=foo.bar.MyLoggingFactory
Listening to Logging Events
You can also listen to logging events generated by Hazelcast runtime
by registering LogListener
s to LoggingService
.
LogListener listener = new LogListener() {
public void log( LogEvent logEvent ) {
// do something
}
};
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
LoggingService loggingService = instance.getLoggingService();
loggingService.addLogListener( Level.INFO, listener );
Through the LoggingService
, you can get the currently used
ILogger implementation and log your own messages too.
If you are not using command line for configuring logging, you should be careful
about Hazelcast classes. They may be defaulted to jdk logging before newly configured
logging is read. When logging mechanism is selected, it will not change.
|
Example Configuration Files
Below are example configurations for Log4j2 and Log4j. Note that Hazelcast does not recommend any specific logging library, these examples are provided only to demonstrate how to configure the logging. You can use your custom logging as explained above.
First, specify the logging type as Log4j2 or Log4j and a separate logging configuration file as shown below.
Using JVM arguments:
-Dhazelcast.logging.type=log4j2 (or log4j)
-Dlog4j.configurationFile=/path/to/properties/log4j2.properties (or log4j)
Using declarative configuration (hazelcast.xml/yaml
):
<hazelcast>
...
<properties>
<property name="hazelcast.logging.type">log4j2</property>
</properties>
...
</hazelcast>
hazelcast:
properties:
hazelcast.logging.type: log4j2
Specifying a separated configuration file is only possible using the JVM argument approach as shown above. |
Following is an example log4j2.properties
file:
rootLogger.level=info
property.filepath=log
property.filename=hazelcast
appenders = console, file
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c\{1}:%L - %m%n
appender.file.type=RollingFile
appender.file.name=RollingFile
appender.file.fileName=${filepath}/${filename}.log
appender.file.filePattern=${filepath}/${filename}-%d{yyyy-MM-dd}-%i.log.gz
appender.file.layout.type=PatternLayout
appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c\{1}:%L - %m%n
appender.file.policies.type=Policies
appender.file.policies.time.type=TimeBasedTriggeringPolicy
appender.file.policies.time.interval=1
appender.file.policies.time.modulate=true
appender.file.policies.size.type=SizeBasedTriggeringPolicy
appender.file.policies.size.size=50MB
appender.file.strategy.type=DefaultRolloverStrategy
appender.file.strategy.max=100
rootLogger.appenderRefs= STDOUT
rootLogger.appenderRef.stdout.ref = STDOUT
rootLogger.appenderRef.file.ref=RollingFile
#Hazelcast specific logs.
#log4j.logger.com.hazelcast=debug
#log4j.logger.com.hazelcast.cluster=debug
#log4j.logger.com.hazelcast.partition=debug
#log4j.logger.com.hazelcast.partition.InternalPartitionService=debug
#log4j.logger.com.hazelcast.nio=debug
#log4j.logger.com.hazelcast.hibernate=debug
To enable the debug logs for all Hazelcast operations uncomment the below line in the above configuration file:
log4j.logger.com.hazelcast=debug
If you do not need detailed logs, the default settings are enough. Using the Hazelcast specific lines in the above configuration file, you can select to see specific logs (cluster, partition, hibernate, etc.) in desired levels.
You can also use the hazelcast.logging.details.enabled
property to
specify whether the name, IP address and version of the cluster are included
in the logs. When there are lots of log lines, it may be hard to follow.
When set to false
, those information will not appear.
And, the following is an example log4j.properties
file:
log4j.rootLogger=INFO,file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=/path/to/log/files/hazelcast.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %p [%c\{1}] - %m%n
log4j.appender.file.maxFileSize=50MB
log4j.appender.file.maxBackupIndex=100
log4j.appender.file.threshold=DEBUG
#log4j.logger.com.hazelcast=debug
#log4j.logger.com.hazelcast.cluster=debug
#log4j.logger.com.hazelcast.partition=debug
#log4j.logger.com.hazelcast.partition.InternalPartitionService=debug
#log4j.logger.com.hazelcast.nio=debug
#log4j.logger.com.hazelcast.hibernate=debug