Configure MongoDB Atlas as an External Data Store for the Cluster with Hazelcast Platform Operator


In this tutorial, you’ll do the following:

  • Build and deploy a custom MapStore/MapLoader implementation to cloud storage.

  • Start Hazelcast cluster with classpath that includes MapStore/MapLoader JAR

  • Create a Hazelcast Map with MapStore/MapLoader configuration

  • Run Hazelcast Java client to manipulate data via populating all entries from MongoDB Atlas.

Before you Begin

Before starting this tutorial, make sure that you meet the following prerequisites:

  • Up and running public Kubernetes cluster. Make sure the Kubernetes cluster is public.

  • Kubernetes command-line tool, kubectl

  • Maven command-line tool, mvn

  • MongoDB Atlas database, Get Started with Atlas (free-tier is suitable for completing this guide).

  • Created blob storage and access credentials in one of the cloud providers: AWS - GCP - Azure

Step 1. Add Node IPs in MongoDB Atlas whitelist

Hazelcast pods need access to the database. To allow Hazelcast pods to connect to the database you need to add the public IP addresses of the Kubernetes nodes to the whitelist in the Atlas dashboard.

  1. Get external IPs of node(s) at the Kubernetes cluster:

    kubectl get nodes -o wide
    NAME       STATUS        EXTERNAL-IP
    gke-....   Ready   ...   34...           ...
    gke-....   Ready   ...   35...           ...
    gke-....   Ready   ...   34...           ...
  2. Add IPs into whitelist at Atlas dashboard:

mongo atlas network access

Step 2. Deploy MapStore/MapLoader JAR to external storage

  1. Build MapStore/MapLoader JAR

    cd mapstore
    mvn package
  2. Upload the JAR to the cloud provider storage blob/bucket that you have already created:

    • S3

    • GCS

    • ABS

    aws s3 cp mapstore/target/mapstore-mongodb-1.0-SNAPSHOT.jar s3://<BUCKET_NAME>
    gsutil cp mapstore/target/mapstore-mongodb-1.0-SNAPSHOT.jar gs://<BUCKET_NAME>
    az storage blob upload --account-name <ACCOUNT_NAME> --container-name <CONTAINER_NAME> --file mapstore/target/mapstore-mongodb-1.0-SNAPSHOT.jar

Step 3. Start Hazelcast Cluster

  1. Create Secret for Bucket credentials

    Run one of the following command to create the secret according to the cloud provider you want to backup.

    • S3

    • GCS

    • Azure

    kubectl create secret generic <secret-name> --from-literal=region=<region> \
    	--from-literal=access-key-id=<access-key-id> \
    kubectl create secret generic <secret-name> --from-file=google-credentials-path=<service_account_json_file>
    kubectl create secret generic <secret-name> \
    	--from-literal=storage-account=<storage-account> \
  2. Create the Hazelcast Cluster with MapStore/MapLoader JAR

    Run the following command to create the Hazelcast cluster with User Code Deployment configuration:

    cat <<EOF | kubectl apply -f -
    kind: Hazelcast
      name: hazelcast
        type: Unisocket
        discoveryServiceType: LoadBalancer
          secret: <SECRET_NAME>
          bucketURI: "<BUCKET_URI>"
Sample bucket URIs:
Exposing the cluster for unisocket clients is required for local client that you will run in next steps.

Step 4. Create Hazelcast Map with MapStore/MapLoader configuration

  1. Create secret for database properties

    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: Secret
      name: mongodb-atlas-properties
      mongo.url: "MONGODB_URL"
      mongo.db: "DB_NAME"
      mongo.collection: "supplements"
    Sample MongoDB Atlas URL → mongodb+srv://<USERNAME>:<PASSWORD>@<DB_NAME>
  2. Create a Map with MapStore configuration:

    cat <<EOF | kubectl apply -f -
    kind: Map
      name: supplements
      hazelcastResourceName: hazelcast
        className: com.operator.tutorial.mongodb.MongoMapStore
        propertiesSecretName: mongodb-atlas-properties

Step 5. Run local Hazelcast client to manipulate data

  1. Check external IP of the LoadBalancer:

    kubectl get services
    NAME         TYPE           EXTERNAL-IP   PORT(S)
    hazelcast    LoadBalancer   34....   5701:30164/TCP
  2. Replace <EXTERNAL-IP> at mapstore/src/main/java/com/hazelcast/tutorial/ with the LoadBalancer’s external IP:

    ClientConfig config = new ClientConfig();
  3. Run Hazelcast Java client that manipulates data

    cd mapstore
    mvn clean package -Pclient
    java -cp target/mapstore-mongodb-1.0-SNAPSHOT.jar com.hazelcast.tutorial.Client
    Members [3] {
            Member []:5701 - 8616cb8d-b0e0-48bd-996c-9e81c6ee98e0
            Member []:5701 - 9c1179ae-a445-4051-83bc-5777a8feb15d
            Member []:5701 - f40f5f5e-61dc-49e1-b511-eab616076f77
    Initial map size:
    Map size after eviction:
    Loading entries form the Database...
    Map size:
  4. Check supplements collection which is populated by the application at MongoDB Atlas dashboard.

mongo atlas supplements

Clean Up

To clean up the created resources remove the all Custom Resources and secrets.

kubectl delete secret <secret-name> mongodb-atlas-properties hazelcast-license-key
kubectl delete $(kubectl get hazelcast,map -o name)