Near Cache
Map or Cache entries in Hazelcast are partitioned across the cluster members.
Hazelcast clients do not have local data at all. Suppose you read the key k
a number of times from
a Hazelcast client or k
is owned by another member in your cluster.
Then each map.get(k)
or cache.get(k)
will be a remote operation, which creates a lot of network trips.
If you have a data structure that is mostly read, then you should consider creating a local Near Cache,
so that reads are sped up and less network traffic is created.
These benefits do not come for free. See the following trade-offs:
-
Members with a Near Cache has to hold the extra cached data, which increases memory consumption.
-
If invalidation is enabled and entries are updated frequently, then invalidations will be costly.
-
Near Cache breaks the strong consistency guarantees; you might be reading stale data.
Near Cache is highly recommended for data structures that are mostly read.
In a client/server system you must enable the Near Cache separately on the client, without the need to configure it on the server. Please note that Near Cache configuration is specific to the server or client itself: a data structure on a server may not have Near Cache configured while the same data structure on a client may have Near Cache configured. They also can have different Near Cache configurations.
If you are using Near Cache, you should take into account that your hits to the keys in the Near Cache are not reflected as hits to the original keys on the primary members. This has for example an impact on IMap’s maximum idle seconds or time-to-live seconds expiration. Therefore, even though there is a hit on a key in Near Cache, your original key on the primary member may expire.
Near Cache works only when you access data via map.get(k) or cache.get(k) methods.
Data returned using a predicate is not stored in the Near Cache.
|
Hazelcast Data Structures with Near Cache Support
The following matrix shows the Hazelcast data structures with Near Cache support.
Please have a look at the next section for a detailed explanation of cache-local-entries
, local-update-policy
, preloader
and serialize-keys
.
Data structure | Near Cache Support | cache-local-entries |
local-update-policy |
preloader |
serialize-keys |
---|---|---|---|---|---|
IMap member |
yes |
yes |
no |
no |
yes |
IMap client |
yes |
no |
no |
yes |
yes |
JCache member |
no |
no |
no |
no |
no |
JCache client |
yes |
no |
yes |
yes |
yes |
ReplicatedMap member |
no |
no |
no |
no |
no |
ReplicatedMap client |
yes |
no |
no |
no |
no |
TransactionalMap member |
limited |
no |
no |
no |
no |
TransactionalMap client |
no |
no |
no |
no |
no |
Even though lite members do not store any data for Hazelcast data structures, you can enable Near Cache on lite members for faster reads. |
Configuring Near Cache
The following shows the configuration for the Hazelcast Near Cache.
Please keep in mind that, if you want to use near cache on a Hazelcast member, configure it on the member; if you want to use it on a Hazelcast client, configure it on the client. |
Declarative Configuration:
<hazelcast>
...
<near-cache name="myDataStructure">
<in-memory-format>BINARY</in-memory-format>
<invalidate-on-change>true</invalidate-on-change>
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>60</max-idle-seconds>
<eviction eviction-policy="LFU"
max-size-policy= "ENTRY_COUNT"
size="1000"/>
<cache-local-entries>false</cache-local-entries>
<local-update-policy>INVALIDATE</local-update-policy>
<preloader enabled="true"
directory="nearcache-example"
store-initial-delay-seconds="0"
store-interval-seconds="0"/>
</near-cache>
...
</hazelcast>
hazelcast:
near-cache:
myDataStructure:
in-memory-format: BINARY
invalidate-on-change: true
time-to-live-seconds: 0
max-idle-seconds: 60
eviction:
size: 1000
max-size-policy: ENTRY_COUNT
eviction-policy: LFU
cache-local-entries: false
local-update-policy: INVALIDATE
preloader:
enabled: true
directory: nearcache-example
store-initial-delay-seconds: 0
store-interval-seconds: 0
The element <near-cache>
has an optional attribute name
whose default value is default
.
Programmatic Configuration:
EvictionConfig evictionConfig = new EvictionConfig()
.setMaxSizePolicy(MaxSizePolicy.ENTRY_COUNT)
.setEvictionPolicy(EvictionPolicy.LRU)
.setSize( 1 );
NearCachePreloaderConfig preloaderConfig = new NearCachePreloaderConfig()
.setEnabled(true)
.setDirectory("nearcache-example")
.setStoreInitialDelaySeconds( 1 )
.setStoreIntervalSeconds( 2 );
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setName("myDataStructure")
.setInMemoryFormat(InMemoryFormat.BINARY)
.setSerializeKeys(true)
.setInvalidateOnChange(false)
.setTimeToLiveSeconds( 1 )
.setMaxIdleSeconds( 5 )
.setEvictionConfig(evictionConfig)
.setCacheLocalEntries(true)
.setLocalUpdatePolicy(NearCacheConfig.LocalUpdatePolicy.CACHE_ON_UPDATE)
.setPreloaderConfig(preloaderConfig);
The class NearCacheConfig is used for all supported Hazelcast data structures on members and clients.
The following are the descriptions of all configuration elements and attributes:
-
in-memory-format
: Specifies in which format data is stored in your Near Cache. Note that a map’s in-memory format can be different from that of its Near Cache. Available values are as follows:-
BINARY
: Data is stored in serialized binary format (default value). -
OBJECT
: Data is stored in deserialized form. -
NATIVE
: Data is stored in the Near Cache that uses Hazelcast’s High-Density Memory Store feature. This option is available only in Hazelcast IMDG Enterprise HD. Note that a map and its Near Cache can independently use High-Density Memory Store. For example, while your map does not use High-Density Memory Store, its Near Cache can use it.
-
-
invalidate-on-change
: Specifies whether the cached entries are evicted when the entries are updated or removed. Its default value is true. -
time-to-live-seconds
: Maximum number of seconds for each entry to stay in the Near Cache. Entries that are older than this period are automatically evicted from the Near Cache. Regardless of the eviction policy used,time-to-live-seconds
still applies. Any integer between 0 andInteger.MAX_VALUE
. 0 means infinite. Its default value is 0. -
max-idle-seconds
: Maximum number of seconds each entry can stay in the Near Cache as untouched (not read). Entries that are not read more than this period are removed from the Near Cache. Any integer between 0 andInteger.MAX_VALUE
. 0 meansInteger.MAX_VALUE
. Its default value is 0. -
eviction
: Specifies the eviction behavior when you use High-Density Memory Store for your Near Cache. It has the following attributes:-
eviction-policy
: Eviction policy configuration. Available values are as follows:-
LRU
: Least Recently Used (default value). -
LFU
: Least Frequently Used. -
NONE
: No items are evicted and the propertymax-size
is ignored. You still can combine it withtime-to-live-seconds
andmax-idle-seconds
to evict items from the Near Cache. -
RANDOM
: A random item is evicted.
-
-
max-size-policy
: Maximum size policy for eviction of the Near Cache. Available values are as follows:-
ENTRY_COUNT
: Maximum size based on the entry count in the Near Cache (default value). -
USED_NATIVE_MEMORY_SIZE
: Maximum used native memory size of the specified Near Cache in MB to trigger the eviction. If the used native memory size exceeds this threshold, the eviction is triggered. Available only forNATIVE
in-memory format. This is supported only by Hazelcast IMDG Enterprise. -
USED_NATIVE_MEMORY_PERCENTAGE
: Maximum used native memory percentage of the specified Near Cache to trigger the eviction. If the native memory usage percentage (relative to maximum native memory size) exceeds this threshold, the eviction is triggered. Available only forNATIVE
in-memory format. This is supported only by Hazelcast IMDG Enterprise. -
FREE_NATIVE_MEMORY_SIZE
: Minimum free native memory size of the specified Near Cache in MB to trigger the eviction. If free native memory size goes below this threshold, eviction is triggered. Available only forNATIVE
in-memory format. This is supported only by Hazelcast IMDG Enterprise. -
FREE_NATIVE_MEMORY_PERCENTAGE
: Minimum free native memory percentage of the specified Near Cache to trigger eviction. If free native memory percentage (relative to maximum native memory size) goes below this threshold, eviction is triggered. Available only forNATIVE
in-memory format. This is supported only by Hazelcast IMDG Enterprise.
-
-
size
: Maximum size of the Near Cache used formax-size-policy
. When this is reached the Near Cache is evicted based on the policy defined. Any integer between1
andInteger.MAX_VALUE
. This value has different defaults, depending on the data structure.-
IMap
: Its default value isInteger.MAX_VALUE
for on-heap maps and10000
for theNATIVE
in-memory format. -
JCache
: Its default value is10000
.
-
-
-
cache-local-entries
: Specifies whether the local entries are cached. It can be useful when in-memory format for Near Cache is different from that of the map. By default, it is disabled. Is just available on Hazelcast members, not on Hazelcast clients (which have no local entries). -
local-update-policy
: Specifies the update policy of the local Near Cache. It is available on JCache clients. Available values are as follows:-
INVALIDATE
: Removes the Near Cache entry on mutation. After the mutative call to the member completes but before the operation returns to the caller, the Near Cache entry is removed. Until the mutative operation completes, the readers still continue to read the old value. But as soon as the update completes the Near Cache entry is removed. Any threads reading the key after this point will have a Near Cache miss and call through to the member, obtaining the new entry. This setting provides read-your-writes consistency. This is the default setting. -
CACHE_ON_UPDATE
: Updates the Near Cache entry on mutation. After the mutative call to the member completes but before the put returns to the caller, the Near Cache entry is updated. So a remove will remove it and one of the put methods will update it to the new value. Until the update/remove operation completes, the entry’s old value can still be read from the Near Cache. But before the call completes the Near Cache entry is updated. Any threads reading the key after this point read the new entry. If the mutative operation was a remove, the key will no longer exist in the cache, both the Near Cache and the original copy in the member. The member initiates an invalidate event to any other Near Caches, however the caller Near Cache is not invalidated as it already has the new value. This setting also provides read-your-writes consistency.
-
-
preloader
: Specifies if the Near Cache should store and pre-load its keys for a faster re-population after a Hazelcast client restart. Is just available on IMap and JCache clients. It has the following attributes:-
enabled
: Specifies whether the preloader for this Near Cache is enabled or not,true
orfalse
. -
directory
: Specifies the parent directory for the preloader of this Near Cache. The filenames for the preloader storage are generated from the Near Cache name. You can additionally specify the parent directory to have multiple clients on the same machine with the same Near Cache names. -
store-initial-delay-seconds
: Specifies the delay in seconds until the keys of this Near Cache are stored for the first time. Its default value is600
seconds. -
store-interval-seconds
: Specifies the interval in seconds in which the keys of this Near Cache are stored. Its default value is600
seconds.
-
Near Cache Configuration Examples
This section shows some configuration examples for different Hazelcast data structures.
Near Cache Example for IMap
The following are configuration examples for IMap Near Caches for Hazelcast members and clients.
<hazelcast>
...
<map name="mostlyReadMap">
<in-memory-format>BINARY</in-memory-format>
<near-cache>
<in-memory-format>OBJECT</in-memory-format>
<invalidate-on-change>false</invalidate-on-change>
<time-to-live-seconds>600</time-to-live-seconds>
<eviction eviction-policy="NONE" max-size-policy="ENTRY_COUNT" size="5000"/>
<cache-local-entries>true</cache-local-entries>
</near-cache>
</map>
...
</hazelcast>
hazelcast:
map:
mostlyReadMap:
in-memory-format: BINARY
near-cache:
in-memory-format: OBJECT
invalidate-on-change: false
time-to-live-seconds: 600
eviction:
eviction-policy: NONE
max-size-policy: ENTRY_COUNT
size: 5000
cache-local-entries: true
EvictionConfig evictionConfig = new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.NONE)
.setMaximumSizePolicy(MaxSizePolicy.ENTRY_COUNT)
.setSize(5000);
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setInvalidateOnChange(false)
.setTimeToLiveSeconds(600)
.setEvictionConfig(evictionConfig);
Config config = new Config();
config.getMapConfig("mostlyReadMap")
.setInMemoryFormat(InMemoryFormat.BINARY)
.setNearCacheConfig(nearCacheConfig);
The Near Cache configuration for maps on members is a child of the map configuration, so you do not have to define the map name in the Near Cache configuration.
<hazelcast-client>
...
<near-cache name="mostlyReadMap">
<in-memory-format>OBJECT</in-memory-format>
<invalidate-on-change>true</invalidate-on-change>
<eviction eviction-policy="LRU" max-size-policy="ENTRY_COUNT" size="50000"/>
</near-cache>
...
</hazelcast-client>
hazelcast-client:
near-cache:
mostlyReadMap:
in-memory-format: OBJECT
invalidate-on-change: true
eviction:
eviction-policy: LRU
max-size-policy: ENTRY_COUNT
size: 50000
EvictionConfig evictionConfig = new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LRU)
.setMaximumSizePolicy(MaxSizePolicy.ENTRY_COUNT)
.setSize(50000);
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setName("mostlyReadMap")
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setInvalidateOnChange(true)
.setEvictionConfig(evictionConfig);
ClientConfig clientConfig = new ClientConfig()
.addNearCacheConfig(nearCacheConfig);
The Near Cache on the client side must have the same name as the data structure on the member for which
this Near Cache is being created. You can use wildcards, so in this example mostlyRead*
would also match the map mostlyReadMap
.
A Near Cache can have its own in-memory-format
which is independent of the in-memory-format
of the data structure.
Near Cache Example for JCache Clients
The following is a configuration example for a JCache Near Cache for a Hazelcast client.
<hazelcast-client>
...
<near-cache name="mostlyReadCache">
<in-memory-format>OBJECT</in-memory-format>
<invalidate-on-change>true</invalidate-on-change>
<eviction eviction-policy="LRU" max-size-policy="ENTRY_COUNT" size="30000"/>
<local-update-policy>CACHE_ON_UPDATE</local-update-policy>
</near-cache>
...
</hazelcast-client>
hazelcast-client:
near-cache:
mostlyReadCache:
in-memory-format: OBJECT
invalidate-on-change: true
eviction:
eviction-policy: LRU
max-size-policy: ENTRY_COUNT
size: 30000
local-update-policy: CACHE_ON_UPDATE
EvictionConfig evictionConfig = new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LRU)
.setMaximumSizePolicy(MaxSizePolicy.ENTRY_COUNT)
.setSize(30000);
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setName("mostlyReadCache")
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setInvalidateOnChange(true)
.setEvictionConfig(evictionConfig)
.setLocalUpdatePolicy(LocalUpdatePolicy.CACHE_ON_UPDATE);
ClientConfig clientConfig = new ClientConfig()
.addNearCacheConfig(nearCacheConfig);
Example for Near Cache with High-Density Memory Store
Hazelcast IMDG Enterprise HD Feature
The following is a configuration example for an IMap High-Density Near Cache for a Hazelcast member.
<hazelcast>
...
<map name="mostlyReadMapWithHighDensityNearCache">
<in-memory-format>OBJECT</in-memory-format>
<near-cache>
<in-memory-format>NATIVE</in-memory-format>
<eviction eviction-policy="LFU" max-size-policy="USED_NATIVE_MEMORY_PERCENTAGE" size="90"/>
</near-cache>
</map>
...
</hazelcast>
hazelcast:
map:
mostlyReadMapWithHighDensityNearCache
in-memory-format: OBJECT
near-cache:
in-memory-format: NATIVE
eviction:
eviction-policy: LFU
max-size-policy: USED_NATIVE_MEMORY_PERCENTAGE
size: 90
EvictionConfig evictionConfig = new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LFU)
.setMaximumSizePolicy(MaxSizePolicy.USED_NATIVE_MEMORY_PERCENTAGE)
.setSize(90);
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setInMemoryFormat(InMemoryFormat.NATIVE)
.setEvictionConfig(evictionConfig);
Config config = new Config();
config.getMapConfig("mostlyReadMapWithHighDensityNearCache")
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setNearCacheConfig(nearCacheConfig);
Keep in mind that you should have already enabled the High-Density Memory Store usage for your cluster. See the Configuring High-Density Memory Store section.
Note that a map and its Near Cache can independently use High-Density Memory Store. For example, if your map does not use High-Density Memory Store, its Near Cache can still use it.
Near Cache Eviction
In the scope of Near Cache, eviction means evicting (clearing) the entries selected according to
the given eviction-policy
when the specified max-size-policy
has been reached.
The max-size-policy
defines the state when the Near Cache is full and determines whether
the eviction should be triggered. The size
is either interpreted as entry count, memory size or percentage, depending on the chosen policy.
Once the eviction is triggered the configured eviction-policy
determines which, if any, entries must be evicted.
Note that the policies mentioned are configured under the near-cache
configuration block, as seen in the above
configuration examples.
Near Cache Expiration
Expiration means the eviction of expired records. A record is expired:
-
if it is not touched (accessed/read) for
max-idle-seconds
-
time-to-live-seconds
passed since it is put to Near Cache.
The actual expiration is performed in the following cases:
-
When a record is accessed: it is checked if the record is expired or not. If it is expired, it is evicted and
null
is returned as the value to the caller. -
In the background: there is an expiration task that periodically (currently 5 seconds) scans records and evicts the expired records.
Note that max-idle-seconds
and time-to-live-seconds
are configured under the near-cache
configuration block, as seen in the above
configuration examples.
Near Cache Invalidation
Invalidation is the process of removing an entry from the Near Cache when its value is updated or it is removed from the original data structure (to prevent stale reads). Near Cache invalidation happens asynchronously at the cluster level, but synchronously at the current member. This means that the Near Cache is invalidated within the whole cluster after the modifying operation is finished, but updated from the current member before the modifying operation is done. A modifying operation can be an EntryProcessor, an explicit update or remove as well as an expiration or eviction. Generally, whenever the state of an entry changes in the record store by updating its value or removing it, the invalidation event is sent for that entry.
Invalidations can be sent from members to client Near Caches or to member Near Caches, either individually or in batches. Default behavior is sending in batches. If there are lots of mutating operations such as put/remove on data structures, it is advised that you configure batch invalidations. This reduces the network traffic and keeps the eventing system less busy, but may increase the delay of individual invalidations.
You can use the following system properties to configure the Near Cache invalidation:
-
hazelcast.map.invalidation.batch.enabled
: Enable or disable batching. Its default value istrue
. When it is set tofalse
, all invalidations are sent immediately. -
hazelcast.map.invalidation.batch.size
: Maximum number of invalidations in a batch. Its default value is100
. -
hazelcast.map.invalidation.batchfrequency.seconds
: If the collected invalidations do not reach the configured batch size, a background process sends them periodically. Its default value is10
seconds.
If there are a lot of clients or many mutating operations, batching should remain enabled and
the batch size should be configured with the hazelcast.map.invalidation.batch.size
system property to a suitable value.
Near Cache Consistency
Eventual Consistency
Near Caches are invalidated by invalidation events. Invalidation events can be lost due to the fire-and-forget fashion of eventing system. If an event is lost, reads from Near Cache can indefinitely be stale.
To solve this problem, Hazelcast provides
eventually consistent behavior for IMap/JCache Near Caches by detecting invalidation losses.
After detection of an invalidation loss, stale data is made unreachable and Near Cache’s get
calls to
that data are directed to the underlying IMap/JCache to fetch the fresh data.
You can configure eventual consistency with the system properties below (same properties are valid for both member and client side Near Caches):
-
hazelcast.invalidation.max.tolerated.miss.count
: Default value is 10. If missed invalidation count is bigger than this value, relevant cached data is made unreachable. -
hazelcast.invalidation.reconciliation.interval.seconds
: Default value is 60 seconds. This is a periodic task that scans cluster members periodically to compare generated invalidation events with the received ones from Near Cache.
Locally Initiated Changes
For local invalidations, when a record is updated/removed, future reads will see this update/remove to provide read-your-writes consistency. To achieve this consistency, Near Cache configuration provides the following update policies:
-
INVALIDATE
-
CACHE_ON_UPDATE
If you choose INVALIDATE
, the entry is removed from the Near Cache after the update/remove occurs in
the underlying data structure and before the operation (get) returns to the caller.
Until the update/remove operation completes, the entry’s old value can still be read from the Near Cache.
If you choose CACHE_ON_UPDATE
, the entry is updated after the
update/remove occurs in the underlying data structure and before the operation (put/get) returns to the caller.
If it is an update operation, it removes the entry and the new value is placed.
Until the update/remove operation completes, the entry’s old value can still be read from the Near Cache.
Any threads reading the key after this point read the new entry. If the mutative operation was a remove,
the key will no longer exist in the Near Cache and the original copy in the member.
Near Cache Preloader
The Near Cache preloader is a functionality to store the keys from a Near Cache to provide a fast re-population of the previous hot data set after a Hazelcast Client has been restarted. It is available on IMap and JCache clients.
The Near Cache preloader stores the keys (not the values) of Near Cache entries in regular intervals.
You can define the initial delay via store-initial-delay-seconds
, e.g., if you know that your hot data set needs
some time to build up. You can configure the interval via store-interval-seconds
which determines how often
the key-set is stored. The persistence does not run continuously. Whenever the storage is scheduled, it is performed on the actual keys in the Near Cache.
The Near Cache preloader is triggered on the first initialization of the data structure on the client, e.g., client.getMap("myNearCacheMap")
.
This schedules the preloader, which works in the background, so your application is not blocked.
The storage is enabled after the loading is completed.
The configuration parameter directory
is optional.
If you omit it, the base folder becomes the user working directory (normally where the JVM was started or
configured with the system property user.dir
).
The storage filenames are always created from the Near Cache name.
So even if you use a wildcard name in the Near Cache Configuration, the preloader filenames are unique.
If you run multiple Hazelcast clients with enabled Near Cache preloader on the same machine, you have to configure a unique storage filename for each client or run them from different user directories. If two clients would write into the same file, only the first client succeeds. The following clients throw an exception as soon as the Near Cache preloader is triggered. |