Configuring Tiered Storage
Before you Begin
Tiered Storage works only with maps whose in-memory format is
NATIVE. Before you configure Tiered Storage for a map, make sure that it uses this in-memory format. See High-Density Memory to learn how to enable and configure the
NATIVE in-memory format for maps.
You cannot yet create indexes on maps that have Tiered Storage enabled.
Configuration for Tiered Storage includes setting the device (local disk) parameters for the Hazelcast member and providing Tiered Storage details for the map. The following shows the minimal configuration to start using Tiered Storage. See Fine Tuning for additional device-specific configuration elements.
<hazelcast> <native-memory enabled="true"> (1) ... (2) </native-memory> <local-device name="my-disk"> (3) <base-dir>tiered-store</base-dir> (4) </local-device> <map name="my-map"> <in-memory-format>NATIVE</in-memory-format> (5) <tiered-store enabled="true"> (6) <memory-tier> <capacity unit="MEGABYTES" value="256"/> (7) </memory-tier> <disk-tier enabled="true" device-name="my-disk"/> (8) (9) </tiered-store> </map> </hazelcast>
hazelcast: native-memory: enabled: true (1) ... (2) local-device: my-disk: (3) base-dir: "tiered-store" (4) map: my-map: in-memory-format: NATIVE (5) tiered-store: enabled: true (6) memory-tier: capacity: (7) unit: MEGABYTES value: 256 disk-tier: enabled: true (8) device-name: "my-disk" (9)
|1||The native memory usage for the member must be enabled.|
|2||See here for available native memory configuration options.|
|3||Name of the disk-based (local) device;
|4||The directory to be used for storing device data; it can be an absolute or relative path to the member startup directory.|
|5||The in-memory format for the map having Tiered Storage enabled MUST be
|6||Tiered Storage is disabled by default; set this parameter
|7||Capacity of the memory in which the frequently accessed data will be stored; cannot be set to
|8||Whether to use disk as an additional (overflow) tier for storage.|
|9||Name of the device (disk) that you set under the
|For now, Tiered Storage supports the local machine’s disk as the storage device. It will also support additional systems such as Amazon S3 and Google Blobstore in the future releases, so that when the local disk is filled up, you will be able to flush data to, for example, Amazon S3.|
Tiered Storage works fine for most use cases with the default settings provided in Configuring Tiered Storage. However, there may be specific use cases where you might want to change the behavior related to the device to be used.
hazelcast: local-device: my-disk: base-dir: "tiered-store" capacity: (1) unit: GIGABYTES value: 512 block-size: 4096 (2) (3) read-io-thread-count: 16 write-io-thread-count: 4 map: my-map: ...
<hazelcast> <local-device name="my-disk"> <base-dir>tiered-store</base-dir> <capacity unit="GIGABYTES" value="512"/> (1) <block-size>4096</block-size> (2) (3) <read-io-thread-count>16</read-io-thread-count> <write-io-thread-count>4</write-io-thread-count> </local-device> <map name="my-map"> ... </hazelcast>
|1||Soft upper limit capacity of the device. You can give the size using its
|2||Allocated size of the device’s IO (read/write), in bytes; refers to the minimum size to request IO. If it is 4096 Bytes, then even if you read or write 100 Bytes, it will still read 4096 Bytes from disk.|
|3||Number of threads for read/write operations from/to disk. For hard disk drives, having write thread IO count of 1 or 2 is optimal; whereas for solid state drives (SSDs) this would lead to under-utilization, so higher writer thread count would be better.|