Building a MapStore for Hazelcast Viridian

MapStore is an API for building a cache on top of Hazelcast. The MapStore interface includes methods that are triggered when operations are invoked on a map. You can implement your own logic in these methods to connect to an external data store, load data from it, and write data back to it. For example, you can use a MapStore to load data into your cluster from a MongoDB, MySQL, or PostgreSQL database.

MapStore supports the following caching patterns:

  • Read-through: If an entry does not exist in memory when an application asks for it, Hazelcast asks the MapStore implementation to load that entry from the data store. If the entry exists, the MapStore implementation requests it from the data store and Hazelcast puts it into memory.

  • Write-through: When new entries are added to a map, those entries are added to the data store synchronously.

  • Write-behind: When new entries are added to a map, those entries are added to the data store asynchronously, after a configured delay.

MapStore extends the MapLoader interface with the methods to store and delete data in the external data source, whereas MapLoader contains the methods to load data into Hazelcast. You can use either of these interfaces to meet your requirements, but this guide refers to both interfaces as MapStore.

The following steps describe the workflow for developing a MapStore:

  1. Implement

  2. Test

  3. Deploy

  4. Configure

To use this feature, you must have either an admin or developer role. See Teams and User Roles.

Implementing a MapStore

To implement a MapStore for Hazelcast Viridian, you can mostly follow the reference in the Hazelcast Platform documentation. However, you must make sure to manage the lifecycle of the MapStore. When the cluster loads the MapStore, it makes a connection to the external data store. If open connections are left open, these connections will consume unnecessary resources. To manage this connection, make sure to implement the MapLoaderLifeCycleSupport interface to allow the cluster to close connections to the data store on a graceful shutdown.

Testing a MapStore

Before you deploy the MapStore to Hazelcast Viridian, you should test the MapStore for the following:

  • The MapStore connects to the data store without errors. Your data source should accept incoming connections from your Viridian cluster.

  • Operations on the map trigger your MapStore implementations. For a list of operations that trigger the MapStore, see the Hazelcast Platform documentation. You can use one of the available clients to invoke map operations.

For general guidance on testing cluster-side modules, see Working with Cluster-Side Modules in Hazelcast Viridian.

Deploying a MapStore

Before you can configure the MapStore, you need to deploy it by packaging your classes into a file along with any dependencies and uploading that file to the cluster. See Deploying Cluster-Side Modules to Viridian Clusters.

Configuring a MapStore

To use a MapStore, it must be configured for a specific map in Hazelcast Viridian. When a map is configured with a MapStore, Hazelcast plugs the MapStore implementation into the lifecycle of the map so that the MapStore is triggered when certain map operations are invoked.

  1. Sign into the Hazelcast Viridian console and select your cluster.

  2. Go to Data Structure Config > Add New Configuration.

  3. Configure the map.

  4. In the bottom left corner, select Enable MapStore.

  5. Select a classname from the dropdown. If you don’t see a classname, you need to deploy the MapStore to the cluster.

  6. Configure the fields, depending on your requirements.

    1. To configure a write-through cache, leave the Write Delay Seconds field set to 0.

    2. To configure a write-behind cache, set the Write Delay Seconds field to the number of seconds that you want to wait before adding entries to the data store.

  7. Click Save Configuration.

Configuration Fields

The following are the descriptions of MapStore configuration elements and attributes:

  • Class Name: Name of the uploaded file that contains the MapStore.

  • Write Delay Seconds: Number of seconds to delay to call the MapStore.store(key, value). If the value is zero then it is write-through, so the MapStore.store(key, value) method is called as soon as the entry is updated. Otherwise, it is write-behind; so the updates will be stored after the Write Delay Seconds value by calling the Hazelcast.storeAll(map) method. Its default value is 0.

  • Write Batch Size: Used to create batch chunks when writing map store. In default mode, all map entries are tried to be written in one go. To create batch chunks, the minimum meaningful value for write-batch-size is 2. For values smaller than 2, it works as in default mode.

  • Write Coalescing: In write-behind mode, Hazelcast coalesces updates on a specific key by default; it applies only the last update on it. You can set this element to false to store all updates performed on a key to the data store.

  • Initial Mode: Sets the initial load mode. Lazy is the default load mode, where the load is asynchronous. Eager means the load is blocked till all partitions are loaded.

  • Properties: The name/value properties that can be read inside the MapStore. For example connection credentials for a database. These properties are encrypted after you save the configuration.

Updating a MapStore

To update a MapStore after you’ve deployed it, you must delete it and redeploy another one.

If the name of your MapStore class is the same as the one that you want to update, do the following:

  1. Click Upload Custom Classes and remove the file that contains your MapStore.

  2. Optional: Test your new MapStore.

  3. Deploy the new version of your MapStore.

If the name of your MapStore class is the different to the one that you want to update, do the following:

  1. Click Upload Custom Classes and remove the file that contains your MapStore.

  2. Optional: Test your new MapStore.

  3. Deploy the new version of your MapStore.

  4. Reconfigure the MapStore.

Next Steps

For sample projects that use a MapStore in Hazelcast Viridian, see this GitHub repository.

Or, follow a tutorial.