Configuring Tiered Storage
Before you Begin
Tiered Storage works only with maps whose in-memory format is NATIVE. Before you configure Tiered Storage for a map, make sure that it uses this in-memory format. See High-Density Memory to learn how to enable and configure the NATIVE in-memory format for maps.
You cannot yet create indexes on maps that have Tiered Storage enabled.
Configuration Options
Configuration for Tiered Storage includes setting the device (local disk) parameters for the Hazelcast member and providing Tiered Storage details for the map. The following shows the minimal configuration to start using Tiered Storage. See Fine Tuning for additional device-specific configuration elements.
<hazelcast>
<native-memory enabled="true"> (1)
... (2)
</native-memory>
<local-device name="my-disk"> (3)
<base-dir>tiered-store</base-dir> (4)
</local-device>
<map name="my-map">
<in-memory-format>NATIVE</in-memory-format> (5)
<tiered-store enabled="true"> (6)
<memory-tier>
<capacity unit="MEGABYTES" value="256"/> (7)
</memory-tier>
<disk-tier enabled="true" device-name="my-disk"/> (8) (9)
</tiered-store>
</map>
</hazelcast>hazelcast:
native-memory:
enabled: true (1)
... (2)
local-device:
my-disk: (3)
base-dir: "tiered-store" (4)
map:
my-map:
in-memory-format: NATIVE (5)
tiered-store:
enabled: true (6)
memory-tier:
capacity: (7)
unit: MEGABYTES
value: 256
disk-tier:
enabled: true (8)
device-name: "my-disk" (9)| 1 | The native memory usage for the member must be enabled. |
| 2 | See here for available native memory configuration options. |
| 3 | Name of the disk-based (local) device; my-disk in this example. |
| 4 | The directory to be used for storing device data; it can be an absolute or relative path to the member startup directory. |
| 5 | The in-memory format for the map having Tiered Storage enabled MUST be NATIVE. You also need to enable the native memory usage |
| 6 | Tiered Storage is disabled by default; set this parameter true to enable it. |
| 7 | Capacity of the memory in which the frequently accessed data will be stored; cannot be set to 0. The default capacity is 256 MB. Available unit options are BYTES, KILOBYTES, MEGABYTES, and GIGABYTES. |
| 8 | Whether to use disk as an additional (overflow) tier for storage. |
| 9 | Name of the device (disk) that you set under the local-device configuration. |
| For now, Tiered Storage supports the local machine’s disk as the storage device. It will also support additional systems such as Amazon S3 and Google Blobstore in the future releases, so that when the local disk is filled up, you will be able to flush data to, for example, Amazon S3. |
Fine-Tuning
Tiered Storage works fine for most use cases with the default settings provided in Configuring Tiered Storage. However, there may be specific use cases where you might want to change the behavior related to the device to be used.
hazelcast:
local-device:
my-disk:
base-dir: "tiered-store"
capacity: (1)
unit: GIGABYTES
value: 512
block-size: 4096 (2)
(3)
read-io-thread-count: 16
write-io-thread-count: 4
map:
my-map:
...<hazelcast>
<local-device name="my-disk">
<base-dir>tiered-store</base-dir>
<capacity unit="GIGABYTES" value="512"/> (1)
<block-size>4096</block-size> (2)
(3)
<read-io-thread-count>16</read-io-thread-count>
<write-io-thread-count>4</write-io-thread-count>
</local-device>
<map name="my-map">
...
</hazelcast>| 1 | Soft upper limit capacity of the device. You can give the size using its value attribute and the unit using its unit attribute. Unit can be bytes, kilobytes, megabytes and gigabytes. The default is 256 gigabytes. |
| 2 | Allocated size of the device’s IO (read/write), in bytes; refers to the minimum size to request IO. If it is 4096 Bytes, then even if you read or write 100 Bytes, it will still read 4096 Bytes from disk. |
| 3 | Number of threads for read/write operations from/to disk. For hard disk drives, having write thread IO count of 1 or 2 is optimal; whereas for solid state drives (SSDs) this would lead to under-utilization, so higher writer thread count would be better. |
