Cluster Utilities

6.0-SNAPSHOT 5.5 5.4 5.3 5.2 5.1 5.0

This section provides information about the Hazelcast command line and other programmatic utilities you can use to listen to the cluster events, to change the state of your cluster, to check whether the cluster and/or members are safe before shutting down a member and to define the minimum number of cluster members required for the cluster to remain up and running. It also gives information about the Hazelcast Lite Member.

Hazelcast Command Line Tool

This is a tool using which you can install and run Hazelcast and Management Center on your Unix-like local environments. You need JRE 8 or newer as a prerequisite.

You can install this tool using its Homebrew, RPM, or Debian packages:

  • Homebrew

  • Debian

  • RPM

brew tap hazelcast/hz
brew install hazelcast@5.5.2
wget -qO - https://repository.hazelcast.com/api/gpg/key/public | gpg --dearmor | sudo tee /usr/share/keyrings/hazelcast-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/hazelcast-archive-keyring.gpg] https://repository.hazelcast.com/debian stable main" | sudo tee -a /etc/apt/sources.list
sudo apt update && sudo apt install hazelcast=5.5.2
wget https://repository.hazelcast.com/rpm/stable/hazelcast-rpm-stable.repo -O hazelcast-rpm-stable.repo
sudo mv hazelcast-rpm-stable.repo /etc/yum.repos.d/
sudo yum install hazelcast-5.5.2

Alternatively, you can manually download and install this tool using the TAR package on its GitHub repo. After you download and extract the package, you see the hazelcast-<version> directory. Simply run the following commands to install it:

cd hazelcast-<version>/bin
./hz

You are now ready to use the tool.

  • Starting a standalone Hazelcast member with the default configuration:

    ./hz start *

  • Starting Hazelcast Management Center:

    ./hz mc start

Please see the tool’s documentation for all the other usages.

Using the hz-cluster-admin Script

You can use the script hz-cluster-admin, which comes with the Hazelcast package, to get/change the state of your cluster, to shutdown your cluster and to force your cluster to clean its persisted data and make a fresh start. The latter is the force-start operation of Persistence. See the Force-start section.

The script hz-cluster-admin uses curl command and curl must be installed to be able to use the script.

The script hz-cluster-admin takes the following parameters to operate according to your needs. If these parameters are not provided, the default values are used.

Parameter Default Value Description

-o or --operation

get-state

Executes a cluster-wide operation. Operations can be the following:

  • Community Edition operations: get-state, change-state, shutdown and get-cluster-version.

  • Enterprise Edition operations: force-start, partial-start and change-cluster-version.

-s or --state

None

Updates the state of the cluster to a new state. New state can be active, no_migration, frozen, passive. This is used with the operation change-state. This parameter has no default value; when you use this, you should provide a valid state.

-a or --address

127.0.0.1

Defines the IP address of a cluster member. If you want to manage your cluster remotely, you should use this parameter to provide the IP address of a member to this script.

-p or --port

5701

Defines on which port Hazelcast is running on the local or remote machine.

-c or --clustername

dev

Defines the name of a cluster which is used for a simple authentication. See the Creating Clusters section.

-P or --password

dev-pass

Defines the password of a cluster (valid only for Hazelcast releases older than 3.8.2). See the Creating Clusters section.

-v or --version

no argument expected

Defines the cluster version to change to. It is used in conjunction with the change-cluster-version operation.

-d or --debug

no argument expected

Prints error output.

--https

no argument expected

Uses HTTPS protocol for REST calls.

--cacert

set of well-known CA certificates

Defines trusted PEM-encoded certificate file path. It’s used to verify member certificates.

--cert

None

Defines PEM-encoded client certificate file path. Only needed when client certificate authentication is used.

--key

None

Defines PEM-encoded client private key file path. Only needed when client certificate authentication is used.

--insecure

no argument expected

Disables member certificate verification.

The script hz-cluster-admin is self-documented; you can see the parameter descriptions using the command ./hz-cluster-admin -h or ./hz-cluster-admin --help.

You can perform the above operations using the Persistence tab of Hazelcast Management Center or using the REST API. See Persistence in the Hazelcast Management Center documentation, and Managing Cluster’s State.
Deprecation Notice for the Community Edition REST API

The Community Edition REST API has been deprecated and will be removed as of Hazelcast version 7.0. An improved Enterprise version of this feature is available and actively being developed. For more info, see Enterprise REST API

Example Usages for hz-cluster-admin

Let’s say you have a cluster running on remote machines and one Hazelcast member is running on the IP 172.16.254.1 and on the port 5702. The cluster name and password of the cluster are test and test.

Getting the cluster state:

To get the state of the cluster, use the following command:

./hz-cluster-admin -o get-state -a 172.16.254.1 -p 5702 -g test -P test

The following also gets the cluster state, using the alternative parameter names, e.g., --port instead of -p:

./hz-cluster-admin --operation get-state --address 172.16.254.1 --port 5702 --clustername test --password test

Changing the cluster state:

To change the state of the cluster to frozen, use the following command:

./hz-cluster-admin -o change-state -s frozen -a 172.16.254.1 -p 5702 -g test -P test

Similarly, you can use the following command for the same purpose:

./hz-cluster-admin --operation change-state --state frozen --address 172.16.254.1 --port 5702 --clustername test --password test

Shutting down the cluster:

To shutdown the cluster, use the following command:

./hz-cluster-admin -o shutdown -a 172.16.254.1 -p 5702 -g test -P test

Similarly, you can use the following command for the same purpose:

./hz-cluster-admin --operation shutdown --address 172.16.254.1 --port 5702 --clustername test --password test

Triggering a partial-start on the cluster:

To trigger a partial-start when Persistence is enabled, use the following command:

./hz-cluster-admin -o partial-start -a 172.16.254.1 -p 5702 -g test -P test

Similarly, you can use the following command for the same purpose:

./hz-cluster-admin --operation partial-start --address 172.16.254.1 --port 5702 --clustername test --password test

Triggering a force-start the cluster:

To trigger a force-start when Persistence is enabled, use the following command:

./hz-cluster-admin -o force-start -a 172.16.254.1 -p 5702 -g test -P test

Similarly, you can use the following command for the same purpose:

./hz-cluster-admin --operation force-start --address 172.16.254.1 --port 5702 --clustername test --password test

Getting the current cluster version:

To get the cluster version, use the following command:

./hz-cluster-admin -o get-cluster-version -a 172.16.254.1 -p 5702 -g test -P test

The following also gets the cluster state, using the alternative parameter names, e.g., --port instead of -p:

./hz-cluster-admin --operation get-cluster-version --address 172.16.254.1 --port 5702 --clustername test --password test

Changing the cluster version:

See the Rolling Member Upgrades chapter to learn more about the cases when you should change the cluster version.

To change the cluster version to X.Y, use the following command:

./hz-cluster-admin -o change-cluster-version -v X.Y -a 172.16.254.1 -p 5702 -g test -P test

The cluster version is always in the major.minor format, e.g., 3.12. Using other formats results in a failure.

Calls against the TLS protected members (using HTTPS protocol):

When the member has TLS configured, use the --https argument to instruct hz-cluster-admin to use the proper URL scheme:

./hz-cluster-admin --https \
  --operation get-state --address member1.example.com --port 5701

If the default set of trusted certificate authorities is not sufficient, e.g, you use a self-signed certificate, you can provide a custom file with the root certificates:

./hz-cluster-admin --https \
  --cacert /path/to/ca-certs.pem \
  --operation get-state --address member1.example.com --port 5701

When the TLS mutual authentication is enabled, you have to provide the client certificate and related private key:

./hz-cluster-admin --https \
  --key privkey.pem \
  --cert cert.pem \
  --operation get-state --address member1.example.com --port 5701
Currently, this script is not supported on the Windows platforms.

Integrity Checker

The integrity checker is a utility developed for Java developers who build Hazelcast applications. It checks the classpath of your applications built with Hazelcast JAR dependencies (e.g., hazelcast or hazelcast-sql, using the assembly or other uber-JAR plugins). This check involves the verification of all serializer hooks are loaded during a Hazelcast member startup, that is, it verifies that the executable of your application contains all the required META-INF/services files.

For example, there may be cases where:

  • you add a new DataSerializer hook for your application, but do not update the corresponding /META-INF/services/com.hazelcast.DataSerializerHook file

  • you build an application with a plugin that merges dependency JARs into the application’s executable JAR; the plugin might be misconfigured and DataSerializerHook files are not merged correctly, resulting in a corrupted executable.

The integrity checker is disabled by default and can be used for your development clusters on your local machines (not recommended for production clusters where the integrity of applications are already verified, and since it is a compute-intensive check).

You can enable it in the configuration as follows.

  • YAML

  • XML

  • Java member API

  • System Prop

  • Environment Variable

hazelcast:
  integrity-checker:
    enabled: true
<hazelcast>
    <integrity-checker enabled="true"/>
</hazelcast>
Config config = new Config();
config.setIntegrityCheckerEnabled(true);

HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance(config);
-Dhz.integritychecker.enabled=true
HZ_INTEGRITYCHECKER_ENABLED=true

If the verification fails, the member does not start and the checker prints which hook is missing its corresponding /META-INF/service/com.hazelcast.DataSerializerHook. An example log output in the terminal is as follows.

SEVERE: [192.168.56.1]:5701 [dev] [5.5.2] Node creation failed
com.hazelcast.core.HazelcastException: Failed to verify distribution integrity, unable to load DataSerializerHook: com.hazelcast.aggregation.impl.AggregatorDataSerializerHook
lifebuoy

Help and support