Hazelcast WAN Replication in Kubernetes

Synchronize data in two Hazelcast clusters that are hosted in different Kubernetes environments, using WAN Replication.

Before you Begin

This tutorial focuses on the WAN Replication feature and assumes that you have some general knowledge about Hazelcast in Kubernetes. Here are some resources:

The tutorial also assumes that you have two running Kubernetes clusters, and the kubectl tool installed. For all the commands the term receiver means that kubectl uses the context of the receiver cluster, and the term publisher means that kubectl uses the context of the publisher cluster.

Note

The code samples present WAN Replication via LoadBalancer, which may result in the communication over only one of the target members. If you need higher performance, see External Smart Client and use such a configuration in the WAN Replication part.

1. Create Receiver Cluster

Hazelcast uses Kubernetes API for the member discovery and it therefore requires granting view permission to certain resources.

(Receiver) $ kubectl apply -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
shell

Then, create a secret with the Hazelcast Enterprise license key.

(Receiver) $ kubectl create secret generic hz-license-key --from-literal key=<hz-license-key>
shell

Now, you can create the cluster with the following command.

(Receiver) $ kubectl apply -f receiver/statefulset.yaml
shell
Note

If you use Minikube, you need to execute minikube tunnel now in order to get LoadBalancer External IPs assigned.

Check that the cluster works correctly and note its External Load Balancer IP.

(Receiver) $ kubectl get all
NAME              READY   STATUS    RESTARTS   AGE
pod/hazelcast-0   1/1     Running   0          2m21s
pod/hazelcast-1   1/1     Running   0          2m11s
pod/hazelcast-2   1/1     Running   0          2m1s

NAME                 TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
service/hazelcast    LoadBalancer   10.171.245.205   35.240.109.13   5701:31092/TCP   2m22s
service/kubernetes   ClusterIP      10.171.240.1     <none>          443/TCP          38m

NAME                         READY   AGE
statefulset.apps/hazelcast   3/3     2m22s
shell

The external IP of the Hazelcast cluster is: 35.240.109.13.

2. Create Publisher Cluster

Again, we need to grant Kubernetes API resource permissions.

(Publisher) $ kubectl apply -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
shell

Then, update the WAN Replication configuration in publisher/hazelcast.yaml with the external IP of the Receiver cluster.

wan-replication:
  my-wan-replication:
    batch-publisher:
      my-publisher:
        cluster-name: dev
        target-endpoints: 35.240.109.13
yaml

Create ConfigMap with the Hazelcast configuration.

(Publisher) $ kubectl create configmap hazelcast-configuration --from-file=publisher/hazelcast.yaml
shell

Again, we need to create a secret with the Hazelcast Enterprise license key.

(Publisher) $ kubectl create secret generic hz-license-key --from-literal key=<hz-license-key>
shell

Finally, we can start the publisher Hazelcast cluster.

(Publisher) $ kubectl apply -f publisher/statefulset.yaml
shell

Your two Hazelcast clusters are set up with the WAN Replication. Now, we can check if everything works correctly.

3. Verify WAN Replication

Insert data into the publisher Hazelcast cluster

Note Before using CLC, it should be installed in your system. Check the installation instructions for CLC: Installing the Hazelcast CLC.

Run the following command for adding the publisher cluster config to the CLC.

clc config add hz-publisher cluster.name=dev cluster.address=<PUBLISHER-CLUSTER-EXTERNAL-IP>
bash

Run the following command for inserting data into the map called rep

for i in {1..10};
do
   clc -c hz-publisher map set --name rep key-$i value-$i;
done
bash

Check that the data was replicated to the receiver Hazelcast cluster.

Run the following command for adding the receiver cluster config to the CLC.

clc config add hz-receiver cluster.name=dev cluster.address=<RECEIVER-CLUSTER-EXTERNAL-IP>
bash

Run the following command for checking the corresponding map’s size.

clc -c hz-receiver map size --name rep
bash

4. Clean up

To delete all created resources, execute the following commands.

(Receiver)

kubectl delete -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
kubectl delete secret hz-license-key
kubectl delete -f receiver/statefulset.yaml
shell

(Publisher)

kubectl delete -f https://raw.githubusercontent.com/hazelcast/hazelcast/master/kubernetes-rbac.yaml
kubectl delete secret hz-license-key
kubectl delete configmap hazelcast-configuration
kubectl delete -f publisher/statefulset.yaml
shell

Summary

In this tutorial, you have seen how to use Hazelcast WAN replication features that can be used to keep multiple Hazelcast clusters in sync. WAN replication can be very useful for Multi-Cloud Deployments.