Tutorial: Connect to Hazelcast from outside Kubernetes

Learn how to connect to a Hazelcast cluster running in Kubernetes from outside of the Kubernetes environment.

Overview

In this tutorial, you'll configure two available options for exposing your Hazelcast cluster to clients outside Kubernetes:

• Unisocket - client requests are load balanced between Hazelcast members.
• Smart - the client connects to all members and sends requests directly to the members owning the data.

The tutorial should take approximately 5 to 10 minutes to complete.

Prerequisites

Before you begin, make sure that you have:

• An up and running Kubernetes cluster.

• The Kubernetes command line tool, kubectl.

• Deployed Hazelcast Platform Operator.

• The Hazelcast Code Samples repository https://github.com/hazelcast/hazelcast-code-samples cloned to your local machine.

• Created a license secret.

  To create a secret with your Hazelcast Enterprise License:

  shell

  kubectl create secret generic hazelcast-license-key --from-literal=license-key=<hz-license-key>

• If you want to use the Hazelcast Command Line Client (CLC) to manage cluster configuration, it must be installed on your system. See the CLC installation instructions.

This tutorial uses load balancer services to connect to Hazelcast from outside of the Kubernetes cluster. Therefore, it is essential to ensure that your Kubernetes cluster can assign public IP addresses to load balancer services. This is particularly important if you are using a local Kubernetes cluster such as Minikube or Kind.

Unisocket

The first option is to use the Unisocket type. This option uses the standard Kubernetes mechanism that automatically load balances the traffic to Hazelcast members.

Start the Hazelcast cluster

Create the Hazelcast cluster and enable exposing externally using the unisocket type:

shell

kubectl apply -f - <<EOF
apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: my-hazelcast-unisocket
spec:
  licenseKeySecretName: hazelcast-license-key
  exposeExternally:
    type: Unisocket
    discoveryServiceType: LoadBalancer
EOF

For discoveryServiceType you can use:

• LoadBalancer - create an external LoadBalancer for discovery service;
• NodePort - expose the discovery service via NodePort.

Verify the Hazelcast cluster

Check the cluster status:

shell

kubectl get hazelcast my-hazelcast-unisocket
NAME           STATUS    MEMBERS
my-hazelcast   Running   3/3

After verifying that the cluster is Running and all the members are ready (3/3), find the discovery address:

shell

kubectl get hazelcastendpoint my-hazelcast-unisocket
NAME                     TYPE        ADDRESS
my-hazelcast-unisocket   Discovery   192.0.2.0:5701

The ADDRESS column shows the external address of your Hazelcast cluster.

Connect Hazelcast Clients to the cluster

1. Open the examples folder to access all sample clients:

   shell

   cd hazelcast-platform-operator-expose-externally

   The sample code for this tutorial is in the docs/modules/ROOT/examples/operator-expose-externally directory.

2. Configure the Hazelcast client with the external address and disable smart routing to use the unisocket connection.

CLC

bash

clc config add hz cluster.name=dev cluster.address=<EXTERNAL-IP>

Java

java

package com.hazelcast;

import com.hazelcast.client.HazelcastClient;
import com.hazelcast.client.config.ClientConfig;
import com.hazelcast.client.properties.ClientProperty;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.map.IMap;

import java.util.Random;

public class Main {
    public static void main(String[] args) throws Exception {
        ClientConfig config = new ClientConfig();
        config.getNetworkConfig().addAddress("<EXTERNAL-IP>");
        config.getProperties().setProperty(ClientProperty.DISCOVERY_SPI_PUBLIC_IP_ENABLED.toString(), "true");
        HazelcastInstance client = HazelcastClient.newHazelcastClient(config);

        System.out.println("Successful connection!");
        System.out.println("Starting to fill the map with random entries.");

        IMap<String, String> map = client.getMap("map");
        Random random = new Random();
        while (true) {
            int randomKey = random.nextInt(100_000);
            map.put("key-" + randomKey, "value-" + randomKey);
            System.out.println("Current map size: " + map.size());
        }
    }
}

NodeJS

javascript

'use strict';

const { Client } = require('hazelcast-client');

const clientConfig = {
    network: {
        clusterMembers: [
            '<EXTERNAL-IP>'
        ]
    },
    properties: {
        ['hazelcast.discovery.public.ip.enabled']: true
    }
};

(async () => {
    try {
        const client = await Client.newHazelcastClient(clientConfig);
        const map = await client.getMap('map');
        await map.put('key', 'value');
        const res = await map.get('key');
        if (res !== 'value') {
            throw new Error('Connection failed, check your configuration.');
        }
        console.log('Successful connection!');
        console.log('Starting to fill the map with random entries.');
        while (true) {
            const randomKey = Math.floor(Math.random() * 100000);
            await map.put('key' + randomKey, 'value' + randomKey);
            const size = await map.size();
            console.log(`Current map size: ${size}`);
        }
    } catch (err) {
        console.error('Error occurred:', err);
    }
})();

Go

go

package main

import (
	"context"
	"fmt"
	"math/rand"

	"github.com/hazelcast/hazelcast-go-client"
)

func main() {
	config := hazelcast.Config{}
	cc := &config.Cluster
	cc.Network.SetAddresses("<EXTERNAL-IP>")
	cc.Discovery.UsePublicIP = true
	ctx := context.TODO()
	client, err := hazelcast.StartNewClientWithConfig(ctx, config)
	if err != nil {
		panic(err)
	}
	fmt.Println("Successful connection!")
	fmt.Println("Starting to fill the map with random entries.")
	m, err := client.GetMap(ctx, "map")
	if err != nil {
		panic(err)
	}
	for {
		num := rand.Intn(100_000)
		key := fmt.Sprintf("key-%d", num)
		value := fmt.Sprintf("value-%d", num)
		if _, err = m.Put(ctx, key, value); err != nil {
			fmt.Println("ERR:", err.Error())
		} else {
			if mapSize, err := m.Size(ctx); err != nil {
				fmt.Println("ERR:", err.Error())
			} else {
				fmt.Println("Current map size:", mapSize)
			}
		}
	}
}

Python

python

import logging
import random

import hazelcast

logging.basicConfig(level=logging.INFO)

client = hazelcast.HazelcastClient(
    cluster_members=["<EXTERNAL-IP>"],
    use_public_ip=True,
)

print("Successful connection!", flush=True)
print("Starting to fill the map with random entries.", flush=True)

m = client.get_map("map").blocking()

while True:
    random_number = str(random.randrange(0, 100000))
    m.put("key-" + random_number, "value-" + random_number)
    print("Current map size:", m.size())

.NET

cs

using Hazelcast;
using Microsoft.Extensions.Logging;


class CsharpExample
{
    static async Task Main(string[] args)
    {

        var options = new HazelcastOptionsBuilder()
            .With(args)
            .With((configuration, options) =>
            {
                options.LoggerFactory.Creator = () => LoggerFactory.Create(loggingBuilder => loggingBuilder.AddConfiguration(configuration.GetSection("logging")).AddConsole());
                options.Networking.Addresses.Add("<EXTERNAL-IP>");
                options.Networking.SmartRouting = false;
            })
            .Build();
        var client = await HazelcastClientFactory.StartNewClientAsync(options);


        Console.WriteLine("Successful connection!");
        Console.WriteLine("Starting to fill the map with random entries.");

        var map = await client.GetMapAsync<string, string>("map");
        var random = new Random();
        while (true)
        {
            var randomKey = random.Next(100_000);
            await map.PutAsync("key-" + randomKey, "value-" + randomKey);
            Console.WriteLine("Current map size: " + await map.GetSizeAsync());
        }
    }
}

Now you can start the application.

CLC

Fill a map:

bash

for i in {1..10};
do
   clc -c hz map set --name map1 key-$i value-$i;
done

Check the map size:

bash

clc -c hz map size --name map1

Java

bash

cd java-unisocket
mvn package
java -jar target/*jar-with-dependencies*.jar

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

NodeJS

bash

cd nodejs-unisocket
npm install
npm start

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Go

bash

cd go-unisocket
go run main.go

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Python

bash

cd python-unisocket
pip install -r requirements.txt
python main.py

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

.NET

bash

cd dotnet-unisocket
dotnet build
dotnet run

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Smart client

The second option is to use the Smart type. With this option, each Hazelcast member is exposed with its own service (it can be either LoadBalancer or NodePort). Hazelcast maps keys to members and sends data directly to the member that contains the right data partition.

Start the Hazelcast cluster

Create the Hazelcast cluster and enable exposing externally using the smart type:

shell

kubectl apply -f - <<EOF
apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: my-hazelcast-smart
spec:
  licenseKeySecretName: hazelcast-license-key
  exposeExternally:
    type: Smart
    discoveryServiceType: LoadBalancer
    memberAccess: LoadBalancer
EOF

This creates the Hazelcast cluster along with one LoadBalancer service for discovery and one LoadBalancer service for each pod.

For discoveryServiceType you can use:

• LoadBalancer - creates an external LoadBalancer for discovery service.
• NodePort - exposes the discovery service via NodePort.

For memberAccess you can use:

• LoadBalancer - lets the client access Hazelcast member with the LoadBalancer service.
• NodePortNodeName - lets the client access Hazelcast member with the NodePort service and the node name.
• NodePortExternalIP - lets the client access Hazelcast member with the NodePort service and the node external IP/hostname.

Verify the Hazelcast cluster

Check the cluster status:

shell

kubectl get hazelcast my-hazelcast-smart
NAME           STATUS    MEMBERS
my-hazelcast   Running   3/3

After verifying that the cluster is Running and all the members are ready (3/3), find the discovery and member addresses:

shell

kubectl get hazelcastendpoint --selector="app.kubernetes.io/instance=my-hazelcast-smart"
NAME               TYPE        ADDRESS
my-hazelcast       Discovery   198.51.100.0:5701
my-hazelcast-0     Member      203.0.113.134:5701
my-hazelcast-1     Member      203.0.113.134:5701
my-hazelcast-2     Member      203.0.113.128:5701
my-hazelcast-wan   WAN         198.51.100.0:5701

The ADDRESS column displays the external addresses of your Hazelcast cluster and members.

Connect Hazelcast clients to the cluster

Configure the Hazelcast client to connect to the cluster external address.

CLC

bash

clc config add hz cluster.name=dev cluster.address=<EXTERNAL-IP>

Java

java

package com.hazelcast;

import com.hazelcast.client.HazelcastClient;
import com.hazelcast.client.config.ClientConfig;
import com.hazelcast.client.properties.ClientProperty;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.map.IMap;

import java.util.Random;

public class Main {
    public static void main(String[] args) throws Exception {
        ClientConfig config = new ClientConfig();
        config.getNetworkConfig().addAddress("<EXTERNAL-IP>");
        config.getProperties().setProperty(ClientProperty.DISCOVERY_SPI_PUBLIC_IP_ENABLED.toString(), "true");
        HazelcastInstance client = HazelcastClient.newHazelcastClient(config);

        System.out.println("Successful connection!");
        System.out.println("Starting to fill the map with random entries.");

        IMap<String, String> map = client.getMap("map");
        Random random = new Random();
        while (true) {
            int randomKey = random.nextInt(100_000);
            map.put("key-" + randomKey, "value-" + randomKey);
            System.out.println("Current map size: " + map.size());
        }
    }
}

NodeJS

javascript

'use strict';

const { Client } = require('hazelcast-client');

const clientConfig = {
    network: {
        clusterMembers: [
            '<EXTERNAL-IP>'
        ]
    },
    properties: {
        ['hazelcast.discovery.public.ip.enabled']: true
    }
};

(async () => {
    try {
        const client = await Client.newHazelcastClient(clientConfig);
        const map = await client.getMap('map');
        await map.put('key', 'value');
        const res = await map.get('key');
        if (res !== 'value') {
            throw new Error('Connection failed, check your configuration.');
        }
        console.log('Successful connection!');
        console.log('Starting to fill the map with random entries.');
        while (true) {
            const randomKey = Math.floor(Math.random() * 100000);
            await map.put('key' + randomKey, 'value' + randomKey);
            const size = await map.size();
            console.log(`Current map size: ${size}`);
        }
    } catch (err) {
        console.error('Error occurred:', err);
    }
})();

Go

go

package main

import (
	"context"
	"fmt"
	"math/rand"

	"github.com/hazelcast/hazelcast-go-client"
)

func main() {
	config := hazelcast.Config{}
	cc := &config.Cluster
	cc.Network.SetAddresses("<EXTERNAL-IP>")
	cc.Discovery.UsePublicIP = true
	ctx := context.TODO()
	client, err := hazelcast.StartNewClientWithConfig(ctx, config)
	if err != nil {
		panic(err)
	}
	fmt.Println("Successful connection!")
	fmt.Println("Starting to fill the map with random entries.")
	m, err := client.GetMap(ctx, "map")
	if err != nil {
		panic(err)
	}
	for {
		num := rand.Intn(100_000)
		key := fmt.Sprintf("key-%d", num)
		value := fmt.Sprintf("value-%d", num)
		if _, err = m.Put(ctx, key, value); err != nil {
			fmt.Println("ERR:", err.Error())
		} else {
			if mapSize, err := m.Size(ctx); err != nil {
				fmt.Println("ERR:", err.Error())
			} else {
				fmt.Println("Current map size:", mapSize)
			}
		}
	}
}

Python

python

import logging
import random

import hazelcast

logging.basicConfig(level=logging.INFO)

client = hazelcast.HazelcastClient(
    cluster_members=["<EXTERNAL-IP>"],
    use_public_ip=True,
)

print("Successful connection!", flush=True)
print("Starting to fill the map with random entries.", flush=True)

m = client.get_map("map").blocking()

while True:
    random_number = str(random.randrange(0, 100000))
    m.put("key-" + random_number, "value-" + random_number)
    print("Current map size:", m.size())

.Net

cs

using Hazelcast;
using Microsoft.Extensions.Logging;


class CsharpExample
{
    static async Task Main(string[] args)
    {

        var options = new HazelcastOptionsBuilder()
            .With(args)
            .With((configuration, options) =>
            {
                options.LoggerFactory.Creator = () => LoggerFactory.Create(loggingBuilder => loggingBuilder.AddConfiguration(configuration.GetSection("logging")).AddConsole());
                options.Networking.Addresses.Add("<EXTERNAL-IP>");
                options.Networking.SmartRouting = false;
            })
            .Build();
        var client = await HazelcastClientFactory.StartNewClientAsync(options);


        Console.WriteLine("Successful connection!");
        Console.WriteLine("Starting to fill the map with random entries.");

        var map = await client.GetMapAsync<string, string>("map");
        var random = new Random();
        while (true)
        {
            var randomKey = random.Next(100_000);
            await map.PutAsync("key-" + randomKey, "value-" + randomKey);
            Console.WriteLine("Current map size: " + await map.GetSizeAsync());
        }
    }
}

Now you can start the application.

CLC

Fill a map:

bash

for i in {1..10};
do
   clc -c hz map set --name map1 key-$i value-$i;
done

Check the map size:

bash

clc -c hz map size --name map1

Java

bash

cd java
mvn package
java -jar target/*jar-with-dependencies*.jar

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

NodeJS

bash

cd nodejs
npm install
npm start

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Go

bash

cd go
go run main.go

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Python

bash

cd python
pip install -r requirements.txt
python main.py

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

.NET

bash

cd dotnet
dotnet build
dotnet run

You should see the following output:

shell

Successful connection!
Starting to fill the map with random entries.
Current map size: 2
Current map size: 3
Current map size: 4
....
....

Clean up

When you've finished, clean up the created resources remove the Hazelcast Custom Resources:

shell

kubectl delete hazelcast my-hazelcast-unisocket
kubectl delete hazelcast my-hazelcast-smart

Summary

In this tutorial, you learned how to connect to a Hazelcast cluster running in Kubernetes from outside of the Kubernetes environment, using both the Unisocket and Smart options.

Next steps

See also:

• Connect to Hazelcast from outside Kubernetes