Skip to main content
Version: 6.0-SNAPSHOT

List of Hazelcast Metrics

The tables below list Hazelcast metrics and descriptions, grouped by subject. This page includes both member-side and client-side metrics.

Unless noted otherwise, metrics are member-side and local to the instance that reports them. For members, distributed data structure metrics reflect the local statistics for the partition data held on that member.

Some metrics represent cluster-wide agreed values obtained by communicating with other members in the cluster. In those cases, each value still reflects the local member's current view of the cluster (for example, in split-brain scenarios). The clusterStartTime metric is one example; on a local member, its value is obtained from the master member.

Metric names on this page are canonical names. Some metrics are exposed with different prefixes depending on where they are collected (member, client, or subsystem). Sections that have prefix variants call them out explicitly.

note

If you use Management Center to export cluster-wide metrics to Prometheus, Management Center reformats the metrics to align with Prometheus best practice recommendations. See Prometheus Monitoring.

Streaming Engine Cluster-Wide Metrics
NameDescriptionTags
blockingWorkerCountNumber of non-cooperative workers employed.

none

Each Hazelcast member will have one instance of each of these metrics.

jobs.submittedNumber of computational jobs submitted.
jobs.completedSuccessfullyNumber of computational jobs successfully completed.
jobs.completedWithFailureNumber of computational jobs that have failed.
jobs.executionStartedNumber of computational job executions started. Each job can execute multiple times, for example when it's restarted or suspended and then resumed.
jobs.executionTerminatedNumber of computational job executions finished. Each job can execute multiple times, for example when it's restarted or suspended and then resumed.
jobs.executionCompletedNumber of computational job executions completed (successfully or otherwise).
iterationCountThe total number of iterations the driver of tasklets in cooperative thread N made. It should increase by at least 250 iterations/s. Lower value means some of the cooperative processors blocks for too long. Somewhat lower value is normal if there are many tasklets assigned to the processor. Lower value affects the latency.

cooperativeWorker

Each Hazelcast member will have one instance of each of these metrics for each of its cooperative worker threads.

taskletCountThe number of assigned tasklets to cooperative thread N.
Streaming Engine Job-specific Metrics

All job specific metrics have their job (name and ID of the job) and exec(job execution ID) tags set, and most also have the vertex (vertex name) tag set (with very few exceptions). This means that most of these metrics will have at least one instance for each vertex of each current job execution.

Additionally, if the vertex sourcing them is a data source or data sink, then the source or sink tags will also be set to true.

NameDescriptionTags
executionStartTimeStart time of the current execution of the job (epoch time in milliseconds).

job, exec

There will be a single instance of these metrics for each job execution.

executionCompletionTimeCompletion time of the current execution of the job (epoch time in milliseconds).
status

Information regarding the job's status. Potential values:

  • 0 - submitted
  • 1 - initialization phase
  • 2 - running
  • 3 - suspended
  • 4 - exporting snapshot
  • 5 - completing phase
  • 6 - failed
  • 7 - completed successfully
userCancelled

Details on whether the user canceled the job. Potential values:

  • 1 - canceled by the user
  • 0 - means the job is either running, completed successfully, or failed due to an error other than user cancellation

Exposed via Job API only.

snapshotBytesTotal number of bytes written out in the last snapshot.

job, exec, vertex

There will be a single instance of these metrics for each vertex.

snapshotKeysTotal number of keys written out in the last snapshot.
distributedBytesInTotal number of bytes received from remote members.

job, exec, vertex, ordinal

Each Hazelcast member will have an instance of these metrics for each ordinal of each vertex of each job execution.

Note: These metrics are only present for distributed edges, i.e., edges producing network traffic.

distributedBytesOutTotal number of bytes sent to remote members.
distributedItemsInTotal number of items received from remote members.
distributedItemsOutTotal number of items sent to remote members.
topObservedWmThis value is equal to the highest coalescedWm on any input edge of this processor.

job, exec, vertex, proc

Each Hazelcast member will have one instances of these metrics for each processor instance N, the N denotes the global processor index. Processor is the parallel worker doing the work of the vertex.

coalescedWmThe highest watermark received from all inputs that was sent to the processor to handle.
lastForwardedWmLast watermark emitted by the processor to output.
lastForwardedWmLatencyThe difference between <i>lastForwardedWn</i> and the system time at the moment when metrics were collected.
queuesCapacityThe total capacity of input queues.
queuesSizeThe total number of items waiting in input queues.
topObservedWmThe highest received watermark from any input on edge N.

job, exec, vertex, proc, ordinal

Each Hazelcast member will have one instance of these metrics for each edge M (input or output) of each processor N. N is the global processor index and M is either the ordinal of the edge or has the value snapshot for output items written to state snapshot.

coalescedWmThe highest watermark received from all upstream processors on edge N.
emittedCountThe number of emitted items. This number includes watermarks, snapshot barriers etc. Unlike distributedItemsOut, it includes items emitted items to local processors.
receivedCountThe number of received items. This number does not include watermarks, snapshot barriers etc. It's the number of items the Processor.process method will receive.
receivedBatchesThe number of received batches. Processor.process receives a batch of items at a time, this is the number of such batches. By dividing receivedCount by receivedBatches, you get the average batch size. It will be 1 under low load.
numInFlightOpsThe number of pending (in flight) operations when using asynchronous mapping processors. See Processors.mapUsingServiceAsyncP.

job, exec, vertex, proc, procType

Processor specific metrics, only certain types of processors have them. The procType tag can be used to identify the exact type of processor sourcing them. Like all processor metrics, each Hazelcast member will have one instances of these metrics for each processor instance N, the N denotes the global processor index.

totalKeysThe number of active keys being tracked by a session window processor.
totalWindowsThe number of active windows being tracked by a session window processor. See Processors.aggregateToSessionWindowP.
totalFramesThe number of active frames being tracked by a sliding window processor.
totalKeysInFramesThe number of grouping keys associated with the current active frames of a sliding window processor. See Processors.aggregateToSlidingWindowP.
totalStatesNumber of active state objects held by stateful processors in a running job. For keyed stages, the metric shows the number of per-key state objects currently allocated. For non-keyed stages, it reports 1 if the processor has processed at least one item, or 0 otherwise. See Processors.mapStatefulP Processor.flatMapStatefulP
lateEventsDroppedThe number of late events dropped by various processor, due to the watermark already having passed their windows.
Map

The Reset column shows the reset behavior of the metrics. There are two types of reset: one occurs on a node restart (indicated by 'N'), and the other on a cluster restart (indicated by 'C'). For example, metrics relating to map operations such as get, put, and remove track operations executed by a member during its runtime, and these are reset upon a node restart. Metrics such as hits, evictions, and expirations are tracked at the cluster level, and these are retained when a member is restarted.

NameUnitDescriptionReset
map.backupCountcountNumber of backups per entryC
map.backupEntryCountcountNumber of backup entries held by the memberC
map.backupEntryMemoryCostbytesMemory cost of backup entries in this memberC
map.creationTimemsCreation time of the map on the memberN
map.dirtyEntryCountcountNumber of updated but not persisted yet entries, dirty entries, that the member ownsN
map.entrySetCountcountNumber of entry set operations on this memberN
map.evictionCountcountNumber of evictions happened on locally owned entries, backups are not includedC
map.expirationCountcountNumber of expirations happened on locally owned entries, backups are not includedC
map.getCountcountNumber of local get operations on the map; it is incremented for every get operation even if the entries do not existN
map.heapCostcountTotal heap cost for the map on this memberC
map.hitscountNumber of reads of the locally owned entries; it is incremented for every read by any type of operation (get, set, put). So the entries should existC
map.indexedQueryCountcountTotal number of indexed local queries performed on the mapN
map.indexesSkippedQueryCountcountTotal number of local queries performed on the map which cannot use indexesN
map.noMatchingIndexQueryCountcountTotal number of local queries performed on the map which had no matching indexN
map.lastAccessTimemsLast access (read) time of the locally owned entriesC
map.lastUpdateTimemsLast update time of the locally owned entriesC
map.lockedEntryCountcountNumber of locked entries that the member ownsC
map.merkleTreesCostcountTotal heap cost of the Merkle trees usedN
map.numberOfEventscountNumber of local events received on the mapN
map.numberOfOtherOperationscountTotal number of other operations performed on this memberN
map.ownedEntryCountcountNumber of map entries owned by the memberC
map.ownedEntryMemoryCostbytesMemory cost of owned map entries on this memberC
map.putCountcountNumber of local put operations on the mapN
map.queryCountcountNumber of queries executed on the map (it may be imprecise for queries involving partition predicates (PartitionPredicate) on the off-heap storage)N
map.queryLimiterHitCountcountNumber of times the query result size limiter was hit on this memberN
map.removeCountcountNumber of local remove operations on the mapN
map.setCountcountNumber of local set operations on the mapN
map.totalGetLatencymsTotal latency of local get operations on the mapN
map.totalMaxGetLatencymsMaximum latency of local get operations on the mapN
map.totalMaxPutLatencymsMaximum latency of local put operations on the mapN
map.totalMaxRemoveLatencymsMaximum latency of local remove operations on the mapN
map.totalMaxSetLatencymsMaximum latency of local set operations on the mapN
map.totalPutLatencymsTotal latency of local put operations on the mapN
map.totalRemoveLatencymsTotal latency of local remove operations on the mapN
map.totalSetLatencymsTotal latency of local set operations on the mapN
map.valuesCountcountNumber of values operations on this memberN
map.store.offloaded.operations.waitingToBeProcessedCountcountNumber of offloaded map store operations waiting to be processedN

The above *latency metrics are only measured for the members and they do not represent the overall performance of the cluster. Hazelcast recommends monitoring the average latency for each operation, for example, map.totalGetLatency / map.getCount and map.totalSetLatency / map.setCount. Increased average latency is a sign that the cluster would experience performance problems, or there is a spike in the load. The following may be the reasons:

  • Increase in the load on the cluster: If the cluster is under heavy load, this can lead to increased latency for all operations, slowing down the overall performance.
  • Increasing member count in the cluster: As the number of cluster members increases, the total latency for operations can also increase. This is because the cluster has to communicate with more members, which can add to the overall latency. This might be a data architecture problem.
  • Increasing the data set size: This causes the cluster to search through more data to find the requested data, which can slow down the overall performance. Creating indexes may solve these kind of problems.
  • Increasing the number of concurrent operations: This causes the cluster to process more requests at the same time, which can slow down the overall performance. This is a potential bottleneck on resources (CPU, memory, network).
map.index.averageHitLatencynsAverage hit latency for the index on this memberN
map.index.averageHitSelectivitypercentAverage selectivity of the hits served by the index on this member (the returned value is in the range from 0.0 to 1.0 - values close to 1.0 indicate a high selectivity meaning the index is efficient; values close to 0.0 indicate a low selectivity meaning the index efficiency is approaching an efficiency of a simple full scan)N
map.index.creationTimemsCreation time of the index on this memberN
map.index.hitCountcountTotal number of index hits (the value of this metric may be greater than the map.index.queryCount since a single query may hit the same index more than once)N
map.index.insertCountcountNumber of insert operations performed on the indexN
map.index.memoryCostbytesLocal memory cost of the index (for on-heap indexes in OBJECT or BINARY formats, the returned value is just a best-effort approximation and doesn't indicate a precise on-heap memory usage of the index)N
map.index.queryCountcountTotal number of queries served by the indexN
map.index.removeCountcountNumber of remove operations performed on the indexN
map.index.totalInsertLatencynsTotal latency of insert operations performed on the indexN
map.index.totalRemoveLatencynsTotal latency of remove operations performed on the indexN
map.index.totalUpdateLatencynsTotal latency of update operations performed on the indexN
map.index.updateCountcountNumber of update operations performed on the indexN
map.index.partitionsIndexedcountNumber of partitions the index is built for on the local memberN
map.index.partitionUpdatesStartedcountNumber of times a partition set change started for the indexN
map.index.partitionUpdatesFinishedcountNumber of times a partition set change completed for the indexN
map.index.notReadyQueryCountcountNumber of queries which matched the index during a partition set changeN
Vector Collection (Beta)
NameUnitDescription
vectorCollection.backupCountcountNumber of backups per entry
vectorCollection.backupEntryCountcountNumber of backup entries held by the member
vectorCollection.backupEntryHeapMemoryCostbytesHeap memory cost of backup entries in this member
vectorCollection.clearCountcountNumber of local clear operations on the vector collection
vectorCollection.creationTimemsCreation time of the vector collection on the member
vectorCollection.deleteCountcountNumber of local delete operations on the vector collection
vectorCollection.getCountcountNumber of local get operations on the vector collection; it is incremented for every get operation even if the entries do not exist.
vectorCollection.heapCostcountTotal heap cost of the vector collection on this member
vectorCollection.lastAccessTimemsLast access time of the locally owned entries
vectorCollection.lastUpdateTimemsLast update time of the locally owned entries
vectorCollection.maxClearLatencymsMaximum latency of local clear operations on the vector collection
vectorCollection.maxOptimizeLatencymsMaximum latency of local optimize operations on the vector collection
vectorCollection.maxSearchLatencymsMaximum latency of search operations coordinated by this member on the vector collection
vectorCollection.maxSizeLatencymsMaximum latency of local size operations on the vector collection
vectorCollection.optimizeCountcountNumber of local optimize operations on the vector collection
vectorCollection.ownedEntryCountcountNumber of vector collection entries owned by the member
vectorCollection.ownedEntryHeapMemoryCostbytesHeap memory cost of owned vector collection entries on this member
vectorCollection.putAllCountcount

Number of local putAll operations on the vector collection.

Note that for each putAll invocation on the client there can be multiple invocations of putAll on the members and some invocations may be translated to set if there is only 1 entry.

vectorCollection.putAllEntryCountcountNumber of entries passed to local putAll operations on the vector collection
vectorCollection.putCountcountNumber of local put operations on the vector collection
vectorCollection.removeCountcountNumber of local remove operations on the vector collection
vectorCollection.searchCountcountNumber of search operations coordinated by this member on the vector collection
vectorCollection.searchIndexQueryCountcount

Number of actual searches on vector index instance (partition) in this vector collection, aggregated for all indexes on the collection.

Includes search retries caused by concurrent modifications and other reasons. If retries were not needed it should be equal to searchCount * non-empty partitions count. In case of retries it will be larger.

vectorCollection.searchIndexVisitedNodescountNumber of graph nodes visited during searches. It is equal to the number of distance/score calculations during searches.
vectorCollection.searchResultsCountcountNumber of results returned by search operations coordinated by this member on the vector collection
vectorCollection.setCountcountNumber of local set operations on the vector collection
vectorCollection.sizeCountcountNumber of local size operations on the vector collection
vectorCollection.totalClearLatencymsTotal latency of local clear operations on the vector collection
vectorCollection.totalDeleteLatencymsTotal latency of local delete operations on the vector collection
vectorCollection.totalGetLatencymsTotal latency of local get operations on the vector collection
vectorCollection.totalMaxDeleteLatencymsMaximum latency of local delete operations on the vector collection
vectorCollection.totalMaxGetLatencymsMaximum latency of local get operations on the vector collection
vectorCollection.totalMaxPutAllLatencymsMaximum latency of local putAll operations on the vector collection
vectorCollection.totalMaxPutLatencymsMaximum latency of local put operations on the vector collection
vectorCollection.totalMaxRemoveLatencymsMaximum latency of local remove operations on the vector collection
vectorCollection.totalMaxSetLatencymsMaximum latency of local set operations on the vector collection
vectorCollection.totalOptimizeLatencymsTotal latency of local optimize operations on the vector collection
vectorCollection.totalPutAllLatencymsTotal latency of local putAll operations on the vector collection
vectorCollection.totalPutLatencymsTotal latency of local put operations on the vector collection
vectorCollection.totalRemoveLatencymsTotal latency of local remove operations on the vector collection
vectorCollection.totalSearchLatencymsTotal latency of search operations coordinated by this member on the vector collection
vectorCollection.totalSetLatencymsTotal latency of local set operations on the vector collection
vectorCollection.totalSizeLatencymsTotal latency of local size operations on the vector collection

The above latency metrics are only measured for the members and do not represent the overall performance of the cluster. We recommend monitoring the average latency for each operation, for example, vectorCollection.totalGetLatency / vectorCollection.getCount and vectorCollection.totalSetLatency / vectorCollection.setCount. Increased average latency is a sign that the cluster is experiencing performance problems, or there is a spike in the load. The following list details potential reasons:

  • Increase in the load on the cluster: if the cluster is under heavy load, this can lead to increased latency for all operations, slowing down the overall performance.
  • Increasing member count in the cluster: as the number of cluster members increases, the total latency for operations can also increase. This is because the cluster has to communicate with more members, which can add to the overall latency. This might be a data architecture problem.
  • Increasing the data set size: this causes the cluster to search through more data to find the requested data, which can slow down the overall performance.
  • Increasing the number of concurrent operations: this causes the cluster to process more requests at the same time, which can slow down the overall performance. This is a potential bottleneck on resources (CPU, memory, network).
MultiMap
NameUnitDescription
multiMap.backupCountcountNumber of backups per entry
multiMap.backupEntryCountcountNumber of backup entries held by the member
multiMap.creationTimemsCreation time of the multimap in the member
multiMap.dirtyEntryCountcountNumber of dirty (updated but not persisted yet) entries that the member owns
multiMap.entrySetCountcountNumber of entry set operations on this member
multiMap.evictionCountcountNumber of evictions completed on locally owned entries, backups are not included
multiMap.expirationCountcountNumber of expirations completed on locally owned entries, backups are not included
multiMap.getCountcountNumber of local get operations on the multimap
multiMap.hitscountNumber of hits (reads) of the locally owned entries
multiMap.indexedQueryCountcountTotal number of indexed local queries performed on the multimap
multiMap.indexesSkippedQueryCountcountTotal number of local queries performed on the multimap which cannot use indexes
multiMap.noMatchingIndexQueryCountcountTotal number of local queries performed on the multimap which had no matching index
multiMap.lastAccessTimemsLast access (read) time of the locally owned entries
multiMap.lastUpdateTimemsLast update time of the locally owned entries
multiMap.lockedEntryCountcountNumber of locked entries that the member owns
multiMap.merkleTreesCostcountHeap cost of the Merkle trees
multiMap.numberOfEventscountNumber of local events received
multiMap.numberOfOtherOperationscountTotal number of other operations
multiMap.ownedEntryCountcountNumber of multimap entries owned by the member
multiMap.putCountcountNumber of local put operations on the multimap
multiMap.queryCountcountNumber of local queries executed on the multimap (it may be imprecise for queries involving partition predicates (PartitionPredicate) on the off-heap storage)
multiMap.queryLimiterHitCountcountNumber of times the query result size limiter was hit on this member
multiMap.removeCountcountNumber of local remove operations on the multimap
multiMap.setCountcountNumber of local set operations on the multimap
multiMap.totalGetLatencymsTotal latency of local get operations
multiMap.totalMaxGetLatencymsMaximum latency of local get operations
multiMap.totalMaxPutLatencymsMaximum latency of local put operations
multiMap.totalMaxRemoveLatencymsMaximum latency of local remove operations
multiMap.totalMaxSetLatencymsMaximum latency of local set operations
multiMap.totalPutLatencymsTotal latency of local put operations
multiMap.totalRemoveLatencymsTotal latency of local remove operations
multiMap.totalSetLatencymsTotal latency of local set operations
multiMap.valuesCountcountNumber of values operations on this member
Replicated Map
NameUnitDescription
replicatedMap.creationTimemsCreation time of this replicated map on this member
replicatedMap.entrySetCountcountNumber of entry set operations on this member
replicatedMap.getCountcountNumber of get operations on this member
replicatedMap.hitscountNumber of hits (reads) of the locally owned entries
replicatedMap.lastAccessTimemsLast access (read) time of the locally owned entries
replicatedMap.lastUpdateTimemsLast update time of the locally owned entries
replicatedMap.maxGetLatencymsMaximum latency of get operations
replicatedMap.maxPutLatencymsMaximum latency of put operations
replicatedMap.maxRemoveLatencymsMaximum latency of remove operations
replicatedMap.numberOfEventscountNumber of events received on this member
replicatedMap.numberOfOtherOperationscountTotal number of other operations on this member
replicatedMap.ownedEntryCountcountNumber of entries owned on this member
replicatedMap.ownedEntryMemoryCostbytesMemory cost of owned entries on this member
replicatedMap.putCountcountNumber of put operations on this member
replicatedMap.removeCountcountNumber of remove operations on this member
replicatedMap.totalGetLatenciesmsTotal latency of get operations
replicatedMap.totalPutLatenciesmsTotal latency of put operations
replicatedMap.totalRemoveLatenciesmsTotal latency of remove operations
replicatedMap.totalcountTotal number of operations on this member
replicatedMap.valuesCountcountNumber of values operations on this member
Cache
NameUnitDescription
cache.averageGetTimeµsMean time to execute gets on the cache
cache.averagePutTimeµsMean time to execute puts on the cache
cache.averageRemovalTimeµsMean time to execute removes on the cache
cache.cacheEvictionscountNumber of evictions on the cache
cache.cacheGetscountNumber of gets on the cache
cache.cacheHitscountNumber of successful get operations, hits, on the cache
cache.cacheHitPercentagepercentPercentage of successful get operations, hits, out of all get operations on the cache
cache.cachePutscountNumber of puts to the cache
cache.cacheRemovalscountNumber of removals from the cache
cache.cacheMissescountNumber of missed cache accesses on the cache
cache.cacheMissPercentagepercentPercentage of missed cache accesses out of all the cache accesses/access attempts
cache.creationTimemsCreation time of the cache on the member
cache.lastAccessTimemsCache's last access time
cache.lastUpdateTimemsCache's last update time
cache.ownedEntryCountcountLocally owned entry count in the cache
Queue
NameUnitDescription
queue.averageAgemsAverage age of the items in this member
queue.backupItemCountcountNumber of backup items held by the member
queue.creationTimemsCreation time of the topic on the member
queue.eventOperationCountcountNumber of event operations
queue.maxAgemsMaximum age of the items in this member
queue.minAgemsMinimum age of the items in this member
queue.numberOfEmptyPollscountNumber of null returning poll operations
queue.numberOfEventscountNumber of event operations (duplicate of eventOperationCount)
queue.numberOfOfferscountNumber of offer/put/add operations
queue.numberOfOtherOperationscountNumber of other operations
queue.numberOfPollscountNumber of poll/take/remove operations.
queue.numberOfRejectedOfferscountNumber of rejected offers
queue.ownedItemCountcountNumber of owned items in this member
queue.totalcountTotal number of operations (numberOfOffers + numberOfPolls + numberOfOtherOperations)
Set
NameUnitDescription
set.creationTimemsCreation time of the set on the member
set.lastAccessTimemsLast access (read) time of the locally owned items
set.lastUpdateTimemsLast update time of the locally owned items
List
NameUnitDescription
list.creationTimemsCreation time of this list on the member
list.lastAccessTimemsLast access (read) time of the locally owned items
list.lastUpdateTimemsLast update time of the locally owned items
Topic
NameUnitDescription
topic.creationTimemsCreation time of the topic on the member
topic.totalPublishescountTotal number of published messages of this topic on this member
topic.totalReceivedMessagescountTotal number of received messages of this topic on this member
Reliable Topic
NameUnitDescription
reliableTopic.creationTimemsCreation time of this reliable topic on the member
reliableTopic.totalPublishescountTotal number of published messages of this reliable topic on this member
reliableTopic.totalReceivedMessagescountTotal number of received messages of this reliable topic on this member
Flake ID Generator
NameUnitDescription
flakeIdGenerator.batchCountcountTotal number of times the Flake ID generator has been used to generate a new ID batch
flakeIdGenerator.creationTimemsCreation time of this Flake ID Generator on the member
flakeIdGenerator.idCountcountTotal number of IDs generated (the sum of IDs for all batches)
PN Counter
NameUnitDescription
pnCounter.creationTimemsCreation time of the PN counter on the member
pnCounter.totalDecrementOperationCountcountNumber of subtract (including decrement) operations on this PN counter
pnCounter.totalIncrementOperationCountcountNumber of add (including increment) operations on this PN counter
pnCounter.valuecountCurrent value of the PN counter
Executor Service
NameUnitDescription
executor.cancelledcountNumber of cancelled operations on the executor service
executor.completedcountNumber of completed operations on the executor service
executor.creationTimemsCreation time of this executor on the member
executor.pendingcountNumber of pending operations on the executor service
executor.startedcountNumber of started operations on the executor service
executor.totalExecutionTimemsTotal execution time of the finished operations
executor.totalStartLatencymsTotal start latency of operations started
executor.internal.completedTaskscountNumber of completed tasks by this executor
executor.internal.maximumPoolSizecountMaximum number of threads in the executor's thread pool
executor.internal.poolSizecountNumber of threads in the executor's thread pool
executor.internal.queueSizecountNumber of pending tasks in this executor's task queue
executor.internal.remainingQueueCapacitycountRemaining capacity on the executor's task queue
User Code Deployment
NameUnitDescription
classloading.loadedClassesCountcountNumber of classes that are currently loaded
classloading.totalLoadedClassesCountcountTotal number of classes that have been loaded since the instance has started execution.
classloading.unloadedClassesCountcountTotal number of unloaded classes.
Cluster
NameUnitDescription
cluster.clock.clusterStartTimemsStart time of the cluster (when the first member in cluster becomes master, its localClockTime value is saved as clusterStartTime)
cluster.clock.clusterTimemsElapsed time since the master member was created (cluster.clock.clusterStartTime)
cluster.clock.clusterTimeDiffmsDifference between the local time (cluster.clock.localClockTime) of the local member and the master member
cluster.clock.clusterUpTimemsUptime of the cluster (current time - cluster.clock.clusterStartTime)
cluster.clock.localClockTimemsMember's local clock timestamp
cluster.clock.maxClusterTimeDiffmsMaximum observed cluster time difference
cluster.connection.closedTimecountConnection close time for this connection
cluster.connection.connectionIdcountConnection ID for this client connection
cluster.connection.eventHandlerCountcountNumber of event handlers for this client connection
cluster.heartbeat.lastHeartbeatmsLast time that this member sent a heartbeat to other known cluster members
cluster.sizecountNumber of members in the cluster
Clients

Scope: member-side only. These metrics describe clients connected to the local member.

NameUnitDescription
client.endpoint.countcountNumber of active client endpoints for this member
client.endpoint.totalRegistrationscountTotal number of client endpoint registrations
Client Invocations

Scope: client-side only.

NameUnitDescription
invocations.maxCurrentInvocationscountMaximum number of concurrent client invocations
invocations.pendingCallscountNumber of pending client invocations on this client
invocations.startedInvocationscountNumber of started client invocations on this client
CP Subsystem
NameUnitDescription
cp.atomiclong.valuecountValue of this IAtomicLong
cp.countdownlatch.countcountInitial count of ICountDownLatch
cp.countdownlatch.remainingcountRemaining number of expected countdowns
cp.countdownlatch.roundcountRound number of the ICountDownLatch; each time ICountDownLatch is initialized with a new count after it downs to zero, a new round begins
cp.lock.acquireLimitcountMaximum number of reentrant acquires of this FencedLock
cp.lock.lockCountcountTotal number of times this FencedLock has been acquired since its creation
cp.lock.ownercountAddress of the FencedLock owner
cp.lock.ownerSessionIdcountSession Id of the FencedLock owner
cp.semaphore.availablecountNumber of the remaining available permits
cp.semaphore.initializedcountState value which shows whether semaphore is initialized with a value or not; in exposed metrics, it shows 0 when semaphore is not initialized, a positive value otherwise
cp.map.sizecountNumber of keys in this CPMap
cp.map.sizeBytescountTotal number of bytes used by key-value pairs for this CPMap
cp.session.creationTimemsCreation time of this session
cp.session.endpointAddress of the endpoint which the CP session belongs to
cp.session.endpointTypeType of the endpoint; either SERVER or CLIENT
cp.session.expirationTimemsExpiration time of the CP session
cp.session.versioncountVersion number of the CP session, basically it shows how many times the session heartbeat is received
cp.session.summarySummary information across all CP sessions grouped by CP group. The summary entails sessionCount (count) which is the count of CP sessions per CP group that are not closed.

We also have a summary section per object type which provides live and destroyed object counts grouped by CP Group. These can be found under cp.{atomiclong,atomicref,countdownlatch,lock,map,semaphore}.summary and provide the following child attributes:

NameUnitDescription
destroyed.countcountNumber of objects for the parent object type that have been destroyed. Also referred to as the tombstoned object count.
live.countcountNumber of objects for the parent object type that are in use.
Events
NameUnitDescription
event.eventQueueSizecountTotal number of events pending to be processed
event.eventsProcessedcountTotal number of processed events
event.listenerCountcountNumber of subscribed listeners for the specified service
event.publicationCountcountNumber of published events for the specified service
event.queueCapacitycountQueue capacity of the executor processing the events (this capacity is shared for all events)
event.rejectedCountcountNumber of rejected events; if the event is not accepted to the executor in hazelcast.event.queue.timeout.millis(see System Properties), it will be rejected and not processed
event.syncDeliveryFailureCountcountNumber of failures of sync event delivery
event.threadCountcountNumber of threads for the event service executor (the event thread count)
event.totalFailureCountcountNumber of events that fail to be published
Listeners

Scope: client-side only.

NameUnitDescription
listeners.eventsProcessedcountTotal number of processed events on the client listener service
listeners.eventQueueSizecountTotal number of tasks pending to be processed on the client listener service
Capacity
NameUnitDescription
file.partition.freeSpacebytesAmount of free space in the given directory, user.home
file.partition.totalSpacebytesAmount of total space in the given directory, user.home
file.partition.usableSpacebytesAmount of usable space in the given directory, user.home
Garbage Collection
NameUnitDescription
gc.majorCountcountTotal number of major garbage collections (GCs) that have occurred
gc.majorTimemsAccumulated elapsed time in major GCs
gc.minorCountcountTotal number of minor GCs that have occurred
gc.minorTimemsAccumulated elapsed time in minor GCs
gc.unknownCountcountNumber of unknown GCs that cannot be determined as minor or major (this is usually due to the lack of support of the used garbage collector)
gc.unknownTimemsAccumulated elapsed time in unknown GCs
Memory

The names in this table use canonical memory. metric names.

Prefix and scope variants:

  • Open Source member and client instance memory metrics: memory.
  • Enterprise member aggregate HD memory metrics: memory. or memorymanager.
  • Enterprise member per-memory-manager metrics: memorymanager.stats.

The same metric values can be exposed under multiple prefixes. Depending on the configured memory allocation type, memorymanager.stats. values can overlap with memorymanager. values.

NameUnitDescription
memory.committedHeapbytesAmount of heap memory that is committed for the JVM to use
memory.committedNativebytesAmount of native memory that is committed for current instance (member or client) to use
memory.freeHeapbytesAmount of free memory in the JVM of current instance (member or client)
memory.freeNativebytesAmount of free native memory in the current instance (member or client)
memory.freePhysicalbytesAmount of free physical memory available in OS
memory.maxHeapbytesMaximum amount of memory that the JVM will use for the heap
memory.maxMetadatabytesThe threshold for warning about metadata usage
memory.maxNativebytesMaximum amount of native memory that current instance (member or client) will use. This value is ignored by Hazelcast for metadata allocation and can increase from the initial configured value
memory.totalPhysicalbytesAmount of total physical memory available in OS
memory.usedHeapbytesAmount of used memory in the JVM of the current instance (member or client)
memory.usedMetadatabytesAmount of used metadata memory by the current instance (member or client)
memory.usedNativebytesAmount of used native memory by the current instance (member or client)
High-Density (HD) Memory Store

Scope: member-side only. Enterprise Edition only.

These metrics are provided for data structures using HD memory. They will be prefixed with the relevant data structure prefix (e.g. map.).

NameUnitDescription
usedMemorybytesCost of HD memory for the map to determine whether eviction of map entries is needed
forceEvictionCountbytesNumber of evictions performed for the HD memory store (when your eviction policy does not free enough native memory, operations that add entries to a map or cache such as map.put() trigger the forced eviction process.)
forceEvictedEntryCountbytesNumber of entries removed from the HD memory store
entryCountbytesNumber of entries in the HD memory store
Near Cache

Scope: member-side and client-side.

The names in this table use the client prefix form (nearcache.). For member-side Near Cache metrics, use map.nearcache. with the same suffix. The same metric values and semantics apply on both sides.

NameUnitDescription
nearcache.creationTimemsCreation time of this Near Cache on this instance (member or client)
nearcache.evictionscountNumber of evictions of Near Cache entries owned by this instance (member or client)
nearcache.expirationscountNumber of TTL and max-idle expirations of Near Cache entries owned by this instance (member or client)
nearcache.hitscountNumber of hits (reads) of Near Cache entries owned by this instance (member or client)
nearcache.invalidationRequestscountNumber of invalidations of Near Cache entries owned by this instance (member or client).
nearcache.invalidationscountNumber of invalidations of Near Cache entries owned by this instance (member or client).
nearcache.lastPersistenceDurationmsDuration of the last Near Cache key persistence
nearcache.lastPersistenceKeyCountcountNumber of persisted keys of the last Near Cache key persistence
nearcache.lastPersistenceTimemsTimestamp of the last Near Cache key persistence
nearcache.lastPersistenceWrittenBytesbytesWritten bytes of the last Near Cache key persistence
nearcache.missescountNumber of misses of Near Cache entries owned by this instance (member or client).
nearcache.ownedEntryCountcountNumber of Near Cache entries owned by this instance (member or client)
nearcache.ownedEntryMemoryCostbytesMemory cost of Near Cache entries owned by this instance (member or client)
nearcache.persistenceCountcountNumber of Near Cache key persistences (when the preload feature is enabled)
Operations
note

Within Hazelcast context, the priority operations are the ones that are important for the stability of cluster, for example heartbeats and migration requests. The normal operations are the ones that manipulate the data, for example map.get and map.put.

NameUnitDescription
operation.adhoc.executedOperationsCountcountNumber of executed adhoc operations
operation.asyncOperationscountNumber of current executing async operations on the operation service of the member
operation.callTimeoutCountcountNumber of operation call timeouts on the member
operation.completedCountcountNumber of completed operations
operation.failedBackupscountNumber of failed backup operations on the operation service of the member
operation.generic.executedOperationsCountcountNumber of executed generic operations
operation.genericPriorityQueueSizecountNumber of priority generic operations pending (waiting in the priority queue)
operation.genericQueueSizecountNumber of normal generic operations pending (waiting in the queue)
operation.genericThreadCountcountNumber of generic operation handler threads in the member
operation.invocations.backupTimeoutMillismsOperation backup timeout that specifies how long the invocation will wait for acknowledgements from the backup replicas (if acks are not received from some backups, there will not be any rollback on other successful replicas)
operation.invocations.backupTimeoutscountNumber of operation invocations that acknowledgment from backups has timeout
operation.invocations.delayedExecutionCountcountNumber of times that the operation invocations have delayed
operation.invocations.heartbeatBroadcastPeriodMillismsBroadcast period of operation heartbeats (this heartbeat packets sent to inform the other member about if the operation is still alive). The heartbeat period is configured to be 1/4 of the call timeout. So with default settings, every 15 seconds, every member in the cluster, will notify every other member in the cluster about all calls that are pending.
operation.invocations.heartbeatPacketsReceivedcountNumber of received heartbeat packets
operation.invocations.heartbeatPacketsSentcountNumber of sent heartbeat packets
operation.invocations.invocationScanPeriodMillismsPeriod for scanning over pending invocations for getting rid of duplicates, checking for heartbeat timeout, and checking for backup timeout
operation.invocations.invocationTimeoutMillismsTimeout for operation invocations
operation.invocations.lastCallIdcountLast issued invocation call ID
operation.invocations.normalTimeoutscountNumber of times that the operation invocations have timed out
operation.invocations.pendingcountNumber of pending invocations
operation.invocations.usedPercentagepercentUsage percentage of the operation invocation capacity (pending invocations/ max concurrent invocations)
operation.parker.parkQueueCountcountNumber of separate WaitSet (set of operations waiting for some condition)
operation.parker.totalParkedOperationCountcountTotal number of parked operations
operation.partition.executedOperationsCountcountNumber of executed partition operations on the specified partition
operation.partitionThreadCountcountNumber of partition operation handler threads for given member
operation.priorityQueueSizecountNumber of priority operations pending (priority partition ops. + priority generic ops.)
operation.queueSizecount

Number of normal operations pending (normal partition operations + normal generic operations).

It refers to the number of operations sent to the member that have yet to be consumed for processing by the partition operation threads. This is the most critical queue for partition aware operations such as map.put and map.remove. This value should be zero or very close to zero. Based on your latency tolerance in your business use case, you can define a threshold for alerts with your preferred alerting mechanism. For instance, triggering an alert if this value is above 100 for 15 seconds would be useful.

operation.responseQueueSizecountTotal number of pending responses (work queue for the response threads) to be processed.
operation.responses.backupCountcountNumber of backup acknowledgement responses
operation.responses.errorCountcountNumber of error responses
operation.responses.missingCountcountNumber of responses having missing invocations
operation.responses.normalCountcountNumber of normal responses
operation.responses.timeoutCountcountNumber of call timeout responses
operation.retryCountcountNumber of retried operations
operation.runningCountcountNumber of currently running operations (runningPartitionCount + runningGenericCount)
operation.runningGenericCountcountNumber of currently running generic (non partition specific) operations
operation.runningPartitionCountcountNumber of currently running partition operations
operation.thread.completedOperationCountcountNumber of completed operations by this operation thread
operation.thread.completedOperationBatchCountcountNumber of completed TaskBatch (a batch of tasks) by this operation thread
operation.thread.completedPacketCountcountNumber of packets that executed by this operation thread
operation.thread.completedPartitionSpecificRunnableCountcountNumber of PartitionSpecificRunnable tasks executed by this operation thread
operation.thread.completedRunnableCountcountTotal number of runnables executed by this operation thread
operation.thread.completedTotalCountcountTotal number of tasks (Operation + PartitionSpecificRunnable + Runnable + TaskBatch) completed on this operation thread
operation.thread.errorCountcountTotal number of failed tasks on this operation thread
operation.thread.normalPendingCountcountNumber of normal pending operations (tasks)
operation.thread.priorityPendingCountcountNumber of priority pending operations (tasks)
Operating System
NameUnitDescription
os.committedVirtualMemorySizebytesAmount of committed virtual memory (that is, the amount of virtual memory guaranteed to be available to the running process)
os.freePhysicalMemorySizebytesAmount of free physical memory
os.freeSwapSpaceSizebytesAmount of free swap space size
os.maxFileDescriptorCountcountMaximum number of open file descriptors (only for UNIX platforms)
os.openFileDescriptorCountcountNumber of open file descriptors (only for UNIX platforms)
os.processCpuLoadpercentRecent CPU usage for the JVM process; a negative value if not available
os.processCpuTimemsCPU time used by the process on which the JVM is running
os.systemCpuLoadpercentRecent CPU usage for the whole system; a negative value if not available
os.systemLoadAveragepercentSystem load average for the last minute, or a negative value if not available
os.totalPhysicalMemorySizebytesTotal amount of physical memory
os.totalSwapSpaceSizebytesTotal amount of swap space
Partitions
NameUnitDescription
partitions.activePartitionCountcountNumber of partitions assigned to the member
partitions.completedMigrationscountNumber of completed migrations on the latest repartitioning round
partitions.elapsedDestinationCommitTimensTotal elapsed time of commit operations' executions to the destination endpoint on the latest repartitioning round
partitions.elapsedMigrationOperationTimensTotal elapsed time of migration & replication operations' executions from source to destination endpoints on the latest repartitioning round
partitions.elapsedMigrationTimensTotal elapsed time from the start of migration tasks to their completion (successful or otherwise) on the latest repartitioning round
partitions.lastRepartitionTimemsLatest time that repartition took place
partitions.localPartitionCountcountNumber of partitions currently owned by given member
partitions.maxBackupCountcountMaximum allowed backup count according to current cluster formation and partition group configuration
partitions.memberGroupsSizecountNumber of the member groups to be used in partition assignments
partitions.migrationActivebooleanWhether there are any currently active migration tasks
partitions.migrationQueueSizecountNumber of migration tasks in the migration queue
partitions.partitionCountcountTotal partition count
partitions.plannedMigrationscountNumber of planned migrations on the latest repartitioning round
partitions.replicaSyncRequestsCountercountNumber of replica sync requests
partitions.replicaSyncSemaphorecountPermits count of this replica sync semaphore
partitions.stateStampcountStamp value for the current partition table; stamp is calculated by hashing the individual partition versions using MurmurHash3 (if stamp has this initial value, 0L, then that means partition table is not initialized yet)
partitions.totalCompletedMigrationscountTotal number of completed migrations
partitions.totalElapsedDestinationCommitTimensTotal elapsed time of commit operations' executions to the destination endpoint since the beginning
partitions.totalElapsedMigrationOperationTimensTotal elapsed time of migration & replication operations' executions from source to destination endpoints since the beginning
partitions.totalElapsedMigrationTimensTotal elapsed time from the start of migration tasks to their completion (successful or otherwise) since the beginning
Persistence
NameUnitDescription
persistence.liveTombstonescountNumber of live tombstones in the persistent store
persistence.liveValuescountNumber of live values in the persistent store
persistence.tombGarbagebytesApproximate size of the garbage within the tombstone chunks (it does not account for data in the active chunk, it is incremented when a record is retired or an active chunk is turned into a stable one)
persistence.tombOccupancybytesApproximate size of tombstone chunks (it does not account for data in the active chunk, it is incremented when the active chunk is turned into a stable one)
persistence.valGarbagebytesApproximate size of the garbage within the value chunk (it does not account for data in the active chunk, it is incremented when a record is retired or an active chunk is turned into a stable one)
persistence.valOccupancybytesApproximate size of value chunks (it does not account for data in the active chunk, it is incremented when the active chunk is turned into a stable one)
Proxies
NameUnitDescription
proxy.createdCountcountNumber of created proxies for a given service
proxy.destroyedCountcountNumber of destroyed proxies for a given service
proxy.proxyCountcountNumber of active proxies for a given service
Raft Algorithm
NameUnitDescription
raft.destroyedGroupIdscountNumber of destroyed raft node group IDs
raft.group.availableLogCapacitycountAvailable log capacity for this CP group
raft.group.commitIndexcountCommit index of this CP group
raft.group.lastAppliedcountLast applied index of this CP group
raft.group.lastLogIndexcountLast log index of this CP group
raft.group.lastLogTermcountLast log term of this CP group
raft.group.leadership.averageTimeMsmsAverage time this CP group member has held the leader role for
raft.group.leadership.electedCountcountNumber of times this CP group member has been elected leader
raft.group.memberCountcountNumber of members in this CP group
raft.group.snapshotBuild.countcountNumber of local snapshots built so far by this CP group member, including failed attempts
raft.group.snapshotBuild.durationMsmsDuration of the most recently completed local snapshot build for this CP group member, including failed attempts
raft.group.snapshotIndexcountRaft snapshot index of this CP group
raft.group.snapshotTransfer.countcountNumber of snapshot transfers this CP group member has installed from its leader
raft.group.snapshotTransfer.durationMsmsDuration of the most recently completed snapshot transfer this CP group member installed from its leader
raft.group.termcountRaft term of this CP group
raft.metadata.activeMembersCommitIndexcountCommit index of the active CP members
raft.metadata.activeMemberscountNumber of active CP members
raft.metadata.groupscountNumber of CP groups
raft.missingMemberscountNumber of missing CP members
raft.nodescountNumber of local Raft nodes
raft.terminatedRaftNodeGroupIdscountNumber of terminated raft node group IDs
Runtime
NameUnitDescription
runtime.availableProcessorscountNumber of processors available to the JVM
runtime.freeMemorybytesAmount of free memory in the JVM
runtime.maxMemorybytesMaximum amount of memory that the JVM will attempt to use
runtime.totalMemorybytesTotal amount of memory in the JVM, the value of this metric may vary over time, depending on the host environment
runtime.upTimemsUptime of the JVM
runtime.usedMemorybytesApproximation to the total amount of memory currently used
TCP

Protocol-specific TCP byte metrics are available only when Advanced Networking is enabled. {PROTOCOL} is one of MEMBER, CLIENT, WAN, REST, or MEMCACHE; WAN includes all WAN protocol endpoints on the reporting member.

NameUnitDescription
tcp.acceptor.eventCountcountTotal number of the connections accepted by TcpServerAcceptor
tcp.acceptor.exceptionCountcountNumber of thrown exceptions on this TcpServerAcceptor
tcp.acceptor.idleTimeMillismsIdle time that measures how long this TcpServerAcceptor has not received any events
tcp.acceptor.selectorRecreateCountcountNumber of times the selector was recreated
tcp.balancer.imbalanceDetectedCountcountNumber of times the IOBalancer detects the imbalance of loads on NioThread s
tcp.balancer.migrationCompletedCountcountNumber of completed NioPipeline migrations by the IOBalancer (these migrations are performed to fix the load imbalance problem on the NioThreads)
tcp.bytesReceivedbytesNumber of bytes received over all connections (active and closed)
tcp.bytesReceived.{PROTOCOL}bytesNumber of bytes received over all connections for the protocol (Advanced Networking only)
tcp.bytesSendbytesNumber of bytes sent over all connections (active and closed)
tcp.bytesSend.{PROTOCOL}bytesNumber of bytes sent over all connections for the protocol (Advanced Networking only)
tcp.connection.acceptedSocketCountcountNumber of accepted socket channels
tcp.connection.activeCountcountNumber of active connections
tcp.connection.clientCountcountNumber of the active client connections
tcp.connection.closedCountcountNumber of closed connections
tcp.connection.connectionListenerCountcountNumber of active connection listeners
tcp.connection.countcountNumber of TcpServerConnection
tcp.connection.inProgressCountcountNumber of connection establishments in progress
tcp.connection.openedCountcountNumber of opened connections
tcp.connection.textCountcountNumber of connections used by text-based protocols (REST, Memcache)
tcp.connection.in/out.completedMigrationscountNumber of completed migrations on this pipeline (migrates this pipeline to a different NioThread)
tcp.connection.in/out.opsInterestedcount
tcp.connection.in/out.opsReadycount
tcp.connection.in/out.ownerIdcountOwner ID of this NioPipeline, -1 if the pipeline is being migrated (owner is null)
tcp.connection.in/out.processCountcountNumber of time the NioPipeline.process() method has been called
tcp.connection.in/out.startedMigrationscountNumber of started migrations on this pipeline
tcp.connection.in.bytesReadbytesTotal size of frames read on this inbound pipeline
tcp.connection.in.idleTimeMsmsIdle time that indicates how long since the last read on this inbound nio pipeline
tcp.connection.in.normalFramesReadcountNumber of priority frames read on this inbound nio pipeline
tcp.connection.in.priorityFramesReadcountNumber of priority frames read
tcp.connection.out.bytesWrittenbytesTotal amount of written frames on this outbound pipeline
tcp.connection.out.idleTimeMillismsIdle time that indicates how long since the last write on this outbound nio pipeline
tcp.connection.out.normalFramesWrittencountNumber of written normal frames on this outbound nio pipeline
tcp.connection.out.priorityFramesWrittencountNumber of priority frames written into this nio pipeline
tcp.connection.out.priorityWriteQueuePendingBytesbytesTotal size of priority frames pending in the write queue
tcp.connection.out.priorityWriteQueueSizecountNumber of priority frames pending in the write queue
tcp.connection.out.scheduledcountOrdinal of enum state of this outbound pipeline: 0 → UNSCHEDULED, 1 → SCHEDULED, 2 → BLOCKED, 3 → RESCHEDULE
tcp.connection.out.writeQueuePendingBytesbytesTotal size of normal frames pending in the write queue
tcp.connection.out.writeQueueSizecountNumber of normal frames pending in the write queue
tcp.inputThread/outputThread.bytesTransceivedbytesAmount of transceived data on this NioThread
tcp.inputThread/outputThread.completedTaskCountcountTotal number of completed tasks on this NioThread
tcp.inputThread/outputThread.eventCountcountTotal number of the connections accepted by TcpServerAcceptor
tcp.inputThread/outputThread.framesTransceivedcountNumber of transceived frames on this NioThread
tcp.inputThread/outputThread.idleTimeMillismsIdle time that indicates the duration since the last read/write
tcp.inputThread/outputThread.ioThreadIdcountThread ID of this NioThread
tcp.inputThread/outputThread.priorityFramesTransceivedcountNumber of transceived priority frames
tcp.inputThread/outputThread.processCountcountNumber of processed `NioPipeline`s on this NioThread
tcp.inputThread/outputThread.selectorIOExceptionCountcountNumber of times that I/O exceptions are thrown during selection
tcp.inputThread/outputThread.selectorRebuildCountcountNumber of times the selector was recreated on this NioThread
tcp.inputThread/outputThread.taskQueueSizecountNumber of pending tasks on the queue of NioThread
Threads
NameUnitDescription
thread.daemonThreadCountcountCurrent number of live daemon thread in the JVM
thread.peakThreadCountcountPeak live thread count since the JVM started
thread.threadCountcountCurrent number of live threads including both daemon and non-daemon threads in the JVM
thread.totalStartedThreadCountcountTotal number of threads started since the JVM started
Transactions
NameUnitDescription
transactions.commitCountcountNumber of committed transactions
transactions.rollbackCountcountNumber of rollbacked transactions
transactions.startCountcountNumber of started transactions
WAN Replication
NameUnitDescription
wan.ackDelayCurrentMillismsDuration of ongoing delaying, -1 if there is no current delaying
wan.ackDelayLastEndmsTimestamp of the last end of delaying the acknowledgments; if this value is bigger than wan.ackDelayLastStart, then there is no delaying
wan.ackDelayLastStartmsTimestamp of the last start of delaying the acknowledgments
wan.ackDelayTotalCountcountTotal number of the triggering delaying the WAN acknowledgments (exceeding the invocation threshold)
wan.ackDelayTotalMillismsTotal amount of time delaying the WAN acknowledgments was taking place
wan.consistencyCheck.lastCheckedPartitionCountcountNumber of checked partitions on the last WAN consistency check
wan.consistencyCheck.lastCheckedLeafCountcountNumber of checked partitions on the last WAN consistency check
wan.consistencyCheck.lastDiffLeafCountcountNumber of different Merkle tree leaves on the last WAN consistency check
wan.consistencyCheck.lastDiffPartitionCountcountNumber of partitions found to be inconsistent on the last WAN consistency check
wan.consistencyCheck.lastEntriesToSynccountNumber of entries to synchronize to get the clusters into sync on the last WAN consistency check
wan.droppedCountcountNumber of dropped entry events
wan.outboundQueueSizecountTotal number of WAN events currently placed in the WAN queues of primary partitions on this member
wan.queueFillPercentpercentPercentage of the WAN replication queue that is filled
wan.removeCountcountNumber of entry remove events
wan.syncCountcountNumber of entry sync events
wan.sync.avgEntriesPerLeafcountAverage of the number of records belong the synchronized Merkle tree nodes have
wan.sync.maxLeafEntryCountcountMaximum of the number of records belong the synchronized Merkle tree nodes have
wan.sync.minLeafEntryCountcountMinimum of the number of records belong the synchronized Merkle tree nodes have
wan.sync.nodesSyncedcountNumber of the synchronized Merkle tree nodes
wan.sync.partitionsSyncedcountNumber of synchronized partitions
wan.sync.partitionsToSynccountNumber of partitions to synchronize
wan.sync.recordsSyncedcountNumber of synchronized records
wan.sync.syncDurationNanosnsDuration of the last synchronization
wan.sync.stdDevEntriesPerLeafcountStandard deviation of the number of records belong the synchronized Merkle tree nodes have
wan.sync.syncStartNanosnsStart time of this WAN synchronization
wan.totalPublishLatencymsTotal latency of published WAN events from this member
wan.totalPublishedEventCountcountTotal number of published WAN events from this member
wan.updateCountcountNumber of entry update events
wan.connectionHealthbooleanThe health of an individual WAN target endpoint, where 1 is healthy and 0 is not
wan.failedTransmitCountcountNumber of attempted WAN replication transmissions that have failed
User Code Namespaces
NameUnitDescription
ucn.updateTimemsUpdate time of a user code namespace configuration
ucn.resourceCountcountNumber of resources contained in a namespace
ucn.resource.resourceTypeenumThe Ordinal of the enum type of the resource contained in a namespace
ucn.resource.resourceSizeBytesbytesThe size in bytes of a resource contained in a namespace