Build a MapStore for Hazelcast Viridian Cloud

The Hazelcast map module classes in the Hazelcast Root API provide in-memory data stores. The MapStore module classes allow you to integrate data from external data sources, such as databases. This means that, if the requested value is not available in the in-memory data store, Hazelcast can attempt to load it from your external data sources. You can also use MapStore to store or delete values in your external data source. You can implement your own logic in the MapStore methods.

MapStore builds a cache on top of Hazelcast with defined triggers for operations on a map. You can build a MapStore using either of the following module classes, depending on your requirements:

  • MapLoader allows you to load data to your cluster from external systems. It supports read-through caching only.

  • MapStore allows you to extend the functionality offered by MapLoader to store and delete data in your external systems. This ensures that all your data stores are synchronized. It supports read-through, write-through, and write-behind caching.

The caching patterns are as follows:

Caching pattern Supported by Description

Read-through

MapLoader and MapStore

If the requested entry is not present in memory, MapStore requests it from the data store.

If present in the data store, the entry is loaded to the distributed map and remains in memory until removed.

Write-through

MapStore only

Adds entries to the data store synchronously when added to a map.

Write-behind

MapStore only

Adds entries to the data store asynchronously when added to a map.

This approach can prevent the thread that is updating the map from becoming blocked, or be used to collect updates into a single batch.

A configured delay limits the time that can be taken between writes to the map and writes to the database.

To develop a MapStore, you must do the following:

To update a MapStore, you must delete the existing MapStore and deploy a new one.

Implement a MapStore

When implementing a MapStore for Viridian Cloud, you must manage the lifecycle to ensure that connections to the external data store are not left open. Closing the connections prevents them from consuming unneccessary resources. You can use the MapLoaderLifeCycleSupport interface to clean up the resources on graceful shutdown, including closing the connections.

With this in mind, you can follow the Hazelcast Platform documentation to implement a MapStore.

When you have finished, return to this page to continue with the testing of your implementation.

Test a MapStore

Before you deploy the MapStore to Viridian Cloud, ensure that:

  • The MapStore connects to the data store without errors. Your data source accepts incoming connections from your Viridian Cloud cluster.

  • Operations on the map trigger your MapStore implementations. For a list of operations that trigger the MapStore, see the Hazelcast Platform documentation. You can use one of the available clients to invoke map operations.

For general guidance on testing cluster-side modules, see the Working with Cluster-Side Modules in Viridian Cloud topic.

Deploy a MapStore

Before you can configure the MapStore, you must deploy it. To do this, package your classes into a file that includes any dependencies and upload that file to the cluster.

For further information on deploying your MapStore, see the Deploying Cluster-Side Modules to Viridian Cloud Clusters topic.

Configure a MapStore

To use a MapStore, it must be configured for a specific map in Viridian Cloud. When a map is configured with a MapStore, Hazelcast plugs the MapStore implementation into the lifecycle of the map to trigger the MapStore when certain map operations are invoked.

To configure a MapStore, complete the following steps:

  1. Sign into the Viridian Cloud console and select your cluster.

  2. Select Manage > Data Structures from the navigation bar options, located on the left.

  3. Select the Add New Configuration button, located at the top right.

  4. Complete the fields to configure the map.

    For further information on the available fields, see the Map Configuration topic.

  5. Select the Enable Map Store checkbox, located at the bottom left.

    Further fields display to configure the Map Store.

    If you have not deployed the MapStore to the cluster, the classname dropdown menu is not displayed. Deploy your MapStore before continuing.
  6. Select a classname from the dropdown menu.

  7. Configure the fields according to your requirements.

    The fields are described in the Configuration Fields section below.

    For a write-through cache, leave the Write Delay Seconds field set to 0.

    For a write-behind cache, enter the time in seconds to wait before adding entries to the data store.

  8. Select the Save Configuration button.

Configuration Fields

The MapStore configuration elements and attributes are as follows:

Class Name

Select the MapStore you require from the dropdown menu.

The menu options are populated with uploaded files containing MapStores.

Write Delay Seconds

Enter the number of seconds to wait before calling the MapStore.store(key, value) method in the text box.

Default is 0.

A value of zero defines a write-through cache. The MapStore.store(key, value) method is called as soon as the entry is updated.

Any other value defines a write-behind cache. Updates are stored for the specified period using the Hazelcast.storeAll(map) method.

Write Batch Size

Enter the number of map entries to include in each batch chunk when writing a map store.

By default, each batch chunk contains a single entry.

The minimum meaningful value is 2. If you enter a value of less than 2, the default is used.

If you do not have sufficient entries to form a complete batch chunk, the final entries are included in a smaller batch chunk after a timeout.
Write Coalescing

Select or de-select the checkbox to determine how to store updates for a key.

Applies only when using write-behind caching.

By default, Hazelcast combines updates on a key.

De-select the checkbox to store all updates on a key.

Select the checkbox to store only the final update on a key that has been changed multiple times.

Initial Mode

Select the initial load mode from the dropdown menu.

By default, lazy loading is used.

The menu options are as follows:

  • Lazy. Select to use asynchronous loading.

  • Eager. Select to pre-load all partitions before loading.

Properties

Enter the name and value of a property, which can be read inside the MapStore, in the text boxes.

For example, the connection credentials for a database.

To add the specified property, select the Add button.

You can continue to add properties in the same way.

When you save the configuration, the properties are encrypted.

Update a MapStore

To update a MapStore after you’ve deployed it, you must delete it and redeploy another one as follows:

  1. Select Upload Custom Classes and remove the file that contains your MapStore.

  2. Optionally, Test your new MapStore.

  3. Deploy the new version of your MapStore.

  4. If the name of your MapStore class is different to the one that you are updating, Reconfigure the MapStore.

Next Steps

Build and run sample projects that use a MapStore in Viridian Cloud. You can find the sample projects in the Hazelcast Cloud Code Samples GitHub repository.

Work through the tutorial to build an application that writes changes to a map back to MongoDB Atlas.