Connect to Hazelcast from Outside Kubernetes

What You’ll Learn

In this tutorial, you’ll connect to a Hazelcast cluster running in Kubernetes from outside of the Kubernetes environment.

Before you Begin

Introduction

There are two available options for Expose Externally feature of Hazelcast Platform Operator:

  • Unisocket - client requests are load balanced between Hazelcast members.

  • Smart - client connects to all members and sends requests directly to the members owning the data.

Let’s see both approaches.

Unisocket

The first option is to use the Unisocket type. This option will use the standard Kubernetes mechanism that automatically load balances the traffic to Hazelcast members.

Hazelcast Unisocket Client
Figure 1. Hazelcast Unisocket Client

Start the Hazelcast Cluster

Run the following command to create the Hazelcast cluster with Expose Externally feature enabled using Unisocket type.

cat <<EOF | kubectl apply -f -
apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: my-hazelcast
spec:
  exposeExternally:
    type: Unisocket
    discoveryServiceType: LoadBalancer
EOF

For discoveryServiceType you can use:

  • LoadBalancer - will create an external LoadBalancer for discovery service;

  • NodePort - will expose the discovery service via NodePort.

Verify the Hazelcast Cluster

Check the cluster status by running the following command.

$ kubectl get hazelcast my-hazelcast
NAME           STATUS    MEMBERS   EXTERNAL-ADDRESSES   WAN-ADDRESSES
my-hazelcast   Running   3/3       34.136.18.51:5701    34.136.18.51:5710

After verifying that the cluster is Running and all the members are ready (3/3), run the following command to find the discovery service address.

$ kubectl get service my-hazelcast
NAME           TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                                                       AGE
my-hazelcast   LoadBalancer   10.64.11.240   34.136.18.51   5701:30627/TCP,5702:30163/TCP,8081:31266/TCP,5710:32275/TCP   3m47s

The field EXTERNAL-IP is the address of your Hazelcast cluster.

Connect Hazelcast Clients to the Cluster

To access all examples (excluding CLC), clone the following repository.

git clone https://github.com/hazelcast-guides/hazelcast-platform-operator-expose-externally.git
cd hazelcast-platform-operator-expose-externally

Configure the Hazelcast client with the external address and disable smart routing to use the unisocket connection.

  • CLC

  • Java

  • NodeJS

  • Go

  • Python

  • .NET

Before using CLC, it should be installed in your system. Check the installation instructions for CLC: Installing the Hazelcast CLC.

Run the following command for adding the cluster config to the CLC.

clc config add hz cluster.name=dev cluster.address=<EXTERNAL-IP>
ClientConfig config = new ClientConfig();
config.getNetworkConfig().addAddress("<EXTERNAL-IP>")
                         .setSmartRouting(false);
const { Client } = require('hazelcast-client');

const clientConfig = {
    network: {
        clusterMembers: [
            '<EXTERNAL-IP>'
        ],
        smartRouting: false
    }
};
const client = await Client.newHazelcastClient(clientConfig);
import (
	"log"

	"github.com/hazelcast/hazelcast-go-client"
)

func main() {
	config := hazelcast.Config{}
	cc := &config.Cluster
	cc.Network.SetAddresses("<EXTERNAL-IP>")
	cc.Unisocket = true
	ctx := context.TODO()
	client, err := hazelcast.StartNewClientWithConfig(ctx, config)
	if err != nil {
		panic(err)
	}
}
import logging
import hazelcast

logging.basicConfig(level=logging.INFO)

client = hazelcast.HazelcastClient(
    cluster_members=["<EXTERNAL-IP>"],
    use_public_ip=True,
    smart_routing=False,
)
var options = new HazelcastOptionsBuilder()
    .With(args)
    .With((configuration, options) =>
    {
        options.LoggerFactory.Creator = () => LoggerFactory.Create(loggingBuilder => loggingBuilder.AddConfiguration(configuration.GetSection("logging")).AddConsole());
        options.Networking.Addresses.Add("<EXTERNAL IP>");
        options.Networking.SmartRouting = false;
    })
    .Build();
var client = await HazelcastClientFactory.StartNewClientAsync(options);

Now you can start the application.

  • CLC

  • Java

  • NodeJS

  • Go

  • Python

  • .NET

Run the following command to fill a map.

for i in {1..10};
do
   clc -c hz map set --name map1 key-$i value-$i;
done

Run the following command to check the map size.

clc -c hz map size --name map1
cd java-unisocket
mvn package
java -jar target/*jar-with-dependencies*.jar

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd nodejs-unisocket
npm install
npm start

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd go-unisocket
go run main.go

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd python-unisocket
pip install -r requirements.txt
python main.py

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd dotnet-unisocket
dotnet build
dotnet run

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Smart Client

The second option is to use the Smart type. With this option, each Hazelcast member will be exposed with its own service (it can be either LoadBalancer or NodePort). Hazelcast smart client is capable of mapping the given key with its owner member, which means that it sends the data directly to the member which contains the right data partition.

Hazelcast Smart Client
Figure 2. Hazelcast Smart Client

Start the Hazelcast Cluster

Run the following command to create the Hazelcast cluster with Expose Externally feature enabled using Smart type.

cat <<EOF | kubectl apply -f -
apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: my-hazelcast
spec:
  exposeExternally:
    type: Smart
    discoveryServiceType: LoadBalancer
    memberAccess: LoadBalancer
EOF

This will create the Hazelcast cluster and one LoadBalancer service for discovery and one LoadBalancer service for each pod.

For discoveryServiceType you can use:

  • LoadBalancer - will create an external LoadBalancer for discovery service;

  • NodePort - will expose the discovery service via NodePort.

For memberAccess you can use the following options:

  • LoadBalancer - lets the client access Hazelcast member with the LoadBalancer service;

  • NodePortNodeName - lets the client access Hazelcast member with the NodePort service and the node name;

  • NodePortExternalIP - lets the client access Hazelcast member with the NodePort service and the node external IP/hostname.

Verify the Hazelcast Cluster

Check the cluster status by running the following command.

$ kubectl get hazelcast my-hazelcast
NAME           STATUS    MEMBERS   EXTERNAL-ADDRESSES                                                           WAN-ADDRESSES
my-hazelcast   Running   3/3       34.136.18.51:5701,34.134.251.35:5701,34.28.159.250:5701,35.192.20.253:5701   34.136.18.51:5710

After verifying that the cluster is Running and all the members are ready (3/3), run the following command to find the discovery service address.

$ kubectl get service my-hazelcast
NAME           TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                                                       AGE
my-hazelcast   LoadBalancer   10.64.5.171   34.136.18.51   5701:31128/TCP,5702:32396/TCP,8081:32538/TCP,5710:31871/TCP   2m49s

The field EXTERNAL-IP is the address of your Hazelcast cluster.

Connect Hazelcast Clients to the Cluster

Configure the Hazelcast client to connect to the cluster external address.

  • CLC

  • Java

  • NodeJS

  • Go

  • Python

  • .Net

Before using CLC, it should be installed in your system. Check the installation instructions for CLC: Installing the Hazelcast CLC.

Run the following command for adding the cluster config to the CLC.

clc config add hz cluster.name=dev cluster.address=<EXTERNAL-IP>
ClientConfig config = new ClientConfig();
config.getNetworkConfig().addAddress("<EXTERNAL-IP>");
const { Client } = require('hazelcast-client');

const clientConfig = {
    network: {
        clusterMembers: [
            '<EXTERNAL-IP>'
        ]
    }
};
const client = await Client.newHazelcastClient(clientConfig);
import (
	"log"

	"github.com/hazelcast/hazelcast-go-client"
)

func main() {
	config := hazelcast.Config{}
	cc := &config.Cluster
	cc.Network.SetAddresses("<EXTERNAL-IP>")
    cc.Discovery.UsePublicIP = true
	ctx := context.TODO()
	client, err := hazelcast.StartNewClientWithConfig(ctx, config)
	if err != nil {
		panic(err)
	}
}
import logging
import hazelcast

logging.basicConfig(level=logging.INFO)

client = hazelcast.HazelcastClient(
    cluster_members=["<EXTERNAL-IP>"],
    use_public_ip=True,
)
var options = new HazelcastOptionsBuilder()
    .With(args)
    .With((configuration, options) =>
    {
        options.LoggerFactory.Creator = () => LoggerFactory.Create(loggingBuilder => loggingBuilder.AddConfiguration(configuration.GetSection("logging")).AddConsole());
        options.Networking.Addresses.Add("<EXTERNAL IP>");
        options.Networking.UsePublicAddresses = true;
    })
    .Build();
var client = await HazelcastClientFactory.StartNewClientAsync(options);

Now you can start the application.

  • CLC

  • Java

  • NodeJS

  • Go

  • Python

  • .NET

Run the following command to fill a map.

for i in {1..10};
do
   clc -c hz map set --name map1 key-$i value-$i;
done

Run the following command to check the map size.

clc -c hz map size --name map1
cd java
mvn package
java -jar target/*jar-with-dependencies*.jar

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd nodejs
npm install
npm start

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd go
go run main.go

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd python
pip install -r requirements.txt
python main.py

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....
cd dotnet
dotnet build
dotnet run

You should see the following output.

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Clean Up

To clean up the created resources remove the Hazelcast Custom Resource.

kubectl delete hazelcast my-hazelcast

See Also