Clustered REST
The clustered REST API can be enabled to allow you to monitor clustered statistics of distributed objects through HTTP endpoints.
Enabling Clustered REST
To enable clustered REST in Management Center, use the
hazelcast.mc.rest.enabled
property at startup. This property is disabled by default.
Root Endpoint
The entry point for the clustered REST API is /rest
. This
resource does not have any attributes.
All parameters that are used in the REST API URLs, like
cluster names and distributed data structure names, must be
URL encoded when composing
a valid request for clustered REST.
Such parameters are marked in braces ({
and }
) in the URL description
for each endpoint. As an example, name.with/special@chars
parameter value would be encoded as name.with%2Fspecial%40chars
.
All endpoints return HTTP status code 404 if no data about a cluster, member, client or data structure can be found in the Management Center.
Issuing User Authentication Tokens
Unless the security provider is set to dev mode, users need an authentication (auth) token to send requests to the REST API.
The auth token must be included in the following header:
-
Authorization: Bearer <user_auth_token>
The way in which you generate an auth token depends on the security provider that you use. See Security Providers.
All auth tokens inherit roles from the user. |
Security Provider | Method for issuing tokens |
---|---|
Local Security Provider |
Use the Send a |
All other security providers other than dev mode |
Send a |
To use the /api/tokens endpoint for security providers except SAML and OIDC, run the following command. Replace the <username> and <password> placeholders with the credentials of a user logged in to the Management Center.
curl -u <username>:<password> -d '{"username":"<username>"}' -H "Content-Type: application/json" -X POST http://localhost:8080/api/tokens
To use the /api/tokens endpoint for SAML and OIDC, run the following command.
curl -d '{"username":"<username>"}' -H "X-CSRF-TOKEN: <x-csrf-token>" -H "Cookie: <cookie>" -H "Content-Type: application/json" -X POST http://localhost:8080/api/tokens
Here, replace the <username>
placeholders with the username of a user logged in to the Management Center, and replace the <x-csrf-token>
and <cookie>
with X-CSRF-TOKEN and Cookie headers. You can get the headers from any request on Network tab in Developer Tools when you are logged into Management Center:
To use the mc-conf
tool, do the following:
mc-conf.sh user issue-token -n <username>
mc-conf.bat user issue-token -n <username>
The created token is displayed in the output:
Successfully issued a token for user 'test_user'.
Token: 'mJMMDfaSWZ1MuqhmGhA8m4erCNZtPi_A4_VyR_y8eH0'
Label: 'test_user_2021-07-07T17:24 EEST'
OpenAPI Definitions
In addition to this documentation, the Clustered REST endpoints are also documented in OpenAPI format. This REST API documentation is accessible at the /swagger-ui/index.html
endpoint. For example http://localhost:8080/swagger-ui/index.html
.
Retrieving Management Center License Expiration Time
This endpoint returns the expiration time in milliseconds (since epoch)
of the license key assigned for the Management Center.
Returns -1
if no license is assigned.
-
Request Type:
GET
-
URL:
/rest/license
-
Request:
curl http://localhost:8080/rest/license
-
Response:
200
(application/json) -
Body:
{ "licenseExpirationTime": 4099755599515 }
Health Check
This resource returns a list of health information for each cluster configured on Management Center. See the Cluster Resource section for detailed explanation on the returned information.
-
Request Type:
GET
-
URL:
/rest/health
-
Request:
curl http://localhost:8080/rest/health
-
Response:
200
(application/json) -
Body:
[ { "cluster": "Cluster A", "connection": "Active", "state": "ACTIVE", "safe": true, "partitionMigrationQueueSize": 0, "members": 3, "numberOfConnectedClients": 120, "masterAddress": "192.168.1.24:5701", "licenseExpirationTime": 4099755599515 }, { "cluster": "Cluster B", "connection": "Error", "state": null, "safe": null, "partitionMigrationQueueSize": null, "members": null, "numberOfConnectedClients": null, "masterAddress": null, "licenseExpirationTime": null }, { "cluster": "Cluster C", "connection": "Disabled", "state": null, "safe": null, "partitionMigrationQueueSize": null, "members": null, "numberOfConnectedClients": null, "masterAddress": null, "licenseExpirationTime": null } ]
Clusters Resource
This resource returns a list of clusters that are connected to the Management Center.
-
Request Type:
GET
-
URL:
/rest/clusters
-
Request:
curl http://localhost:8080/rest/clusters
-
Response:
200
(application/json) -
Body:
["dev", "qa"]
Cluster Resource
This resource returns information related to the provided cluster name:
-
cluster
: name of the cluster -
connection
: cluster connection status (Active
,Error
orDisabled
) -
state
: state of the cluster -
safe
: whether the cluster is safe, i.e., whether it has any ongoing partition migrations -
partitionMigrationQueueSize
: partition migration queue size (sum of partition migration queue sizes of all cluster members) -
members
: number of members -
numberOfConnectedClients
: number of connected clients (excluding any Management Center clients) -
masterAddress
: address of the oldest cluster member -
licenseExpirationTime
: the expiration time in milliseconds (since epoch) of the license key assigned for the cluster
Fields other than the name of the cluster and cluster connection status are returned as null if there’s an error
connecting to the cluster, i.e., connection status is Error or connection to the cluster is disabled, i.e., connection
status is Disabled .
|
Following is an example that shows how to get this resource.
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>
-
Request:
curl http://localhost:8080/rest/clusters/dev/
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "connection": "Active", "state": "ACTIVE", "safe": true, "partitionMigrationQueueSize": 0, "members": 3, "numberOfConnectedClients": 120, "masterAddress": "192.168.1.24:5701", "licenseExpirationTime": 4099755599515 }
Members Resource
This resource returns a list of the members belonging to the provided clusters.
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members
-
Request:
curl http://localhost:8080/rest/clusters/dev/members
-
Response:
200
(application/json) -
Body:
[ "192.168.2.78:5701", "192.168.2.78:5702", "192.168.2.78:5703", "192.168.2.78:5704" ]
Member Resource
This resource returns the following information related to the provided member:
See the following example responses for each of the above.
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "address": "192.168.2.78:5701", "uuid": "11adba52-e19d-4407-a9e9-e0a271cef14a", "cpMemberUuid": "f5a8f8a4-f278-4a13-a23e-5accf5b02f42", "maxHeapMemory": 129957888, "ownedPartitionCount": 68, "usedHeapMemory": 60688784, "freeHeapMemory": 24311408, "committedHeapMemory": 85000192, "connectedClientCount": 1, "master": true }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/connectionManager
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/connectionManager
-
Response:
200
(application/json) -
Body:
{ "clientConnectionCount": 2, "activeConnectionCount": 5, "connectionCount": 5 }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/operationService
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/operationService
-
Response:
200
(application/json) -
Body:
{ "responseQueueSize": 0, "operationExecutorQueueSize": 0, "runningOperationsCount": 0, "remoteOperationCount": 1, "executedOperationCount": 461139, "operationThreadCount": 8 }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/eventService
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/eventService
-
Response:
200
(application/json) -
Body:
{ "eventThreadCount": 5, "eventQueueCapacity": 1000000, "eventQueueSize": 0 }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/partitionService
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/partitionService
-
Response:
200
(application/json) -
Body:
{ "partitionCount": 271, "activePartitionCount": 68 }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/proxyService
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/proxyService
-
Response:
200
(application/json) -
Body:
{ "proxyCount": 8, "createdCount": 13, "destroyedCount": 5 }
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/managedExecutors
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701/managedExecutors
-
Response:
200
(application/json) -
Body:
["hz:system", "hz:async", "hz:scheduled", "hz:client", "hz:client-query", "hz:client-blocking-tasks", "hz:query", "hz:io", "hz:offloadable", "hz:map-load", "hz:map-loadAllKeys", "hz:mc"]
Retrieving information of a single Managed Executor:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/members/<member>/managedExecutors/<managedExecutor>
-
Request:
curl http://localhost:8080/rest/clusters/dev/members/192.168.2.78:5701 /managedExecutors/hz:system
-
Response:
200
(application/json) -
Body:
{ "name": "hz:system", "queueSize": 0, "poolSize": 0, "remainingQueueCapacity": 2147483647, "maximumPoolSize": 4, "completedTaskCount": 12 }
Client Endpoints Resource
This resource returns a list of the client endpoints belonging to the provided cluster. Consider using the newly added Client Statistics Resource as it contains more detailed information about the clients.
Retrieving the list of Client Endpoints:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/clients
-
Request:
curl http://localhost:8080/rest/clusters/dev/clients
-
Response:
200
(application/json) -
Body:
["192.168.2.78:61708"]
Retrieving Client Endpoint Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/clients/<client>
-
Request:
curl http://localhost:8080/rest/clusters/dev/clients/192.168.2.78:61708
-
Response:
200
(application/json) -
Body:
{ "uuid": "6fae7af6-7a7c-4fa5-b165-cde24cf070f5", "address": "192.168.2.78:61708", "clientType": "JAVA", "name": "hz.client_1", "labels": [ "label1" ], "ipAddress": "192.168.2.78", "canonicalHostName": "localhost" }
Maps Resource
This resource returns a list of maps belonging to the provided cluster.
Retrieving the list of Maps:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/maps
-
Request:
curl http://localhost:8080/rest/clusters/dev/maps
-
Response:
200
(application/json) -
Body:
["customers", "orders"]
Retrieving Map Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/maps/<mapname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/maps/customers
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "customers", "ownedEntryCount": 5085, "backupEntryCount": 5076, "ownedEntryMemoryCost": 833940, "backupEntryMemoryCost": 832464, "heapCost": 1666668, "lockedEntryCount": 2, "dirtyEntryCount": 0, "hits": 602, "lastAccessTime": 1532689094579, "lastUpdateTime": 1532689094576, "creationTime": 1532688789256, "putOperationCount": 5229, "getOperationCount": 2162, "removeOperationCount": 150, "setOperationCount": 100, "otherOperationCount": 3687, "events": 10661, "maxPutLatency": 48, "maxGetLatency": 35, "maxRemoveLatency": 18034, "maxSetLatency": 10, "totalPutLatency": 1715433, "totalGetLatency": 945421, "totalRemoveLatency": 66558323, "totalSetLatency": 786421 }
MultiMaps Resource
This resource returns a list of multimaps belonging to the provided cluster.
Retrieving the list of MultiMaps:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/multimaps
-
Request:
curl http://localhost:8080/rest/clusters/dev/multimaps
-
Response:
200
(application/json) -
Body:
["customerAddresses"]
Retrieving MultiMap Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/multimaps/<multimapname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/multimaps/customerAddresses
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "customerAddresses", "ownedEntryCount": 4862, "backupEntryCount": 4860, "ownedEntryMemoryCost": 0, "backupEntryMemoryCost": 0, "heapCost": 0, "lockedEntryCount": 1, "dirtyEntryCount": 0, "hits": 22, "lastAccessTime": 1532689253314, "lastUpdateTime": 1532689252591, "creationTime": 1532688790593, "putOperationCount": 5125, "getOperationCount": 931, "removeOperationCount": 216, "otherOperationCount": 373570, "events": 0, "maxPutLatency": 8, "maxGetLatency": 1, "maxRemoveLatency": 18001, "totalPutLatency": 487805, "totalGetLatency": 14931, "totalRemoveLatency": 16388472 }
Replicated Maps Resource
This resource returns a list of replicated maps belonging to the provided cluster.
Retrieving the list of Replicated Maps:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/replicatedmaps
-
Request:
curl http://localhost:8080/rest/clusters/dev/replicatedmaps
-
Response:
200
(application/json) -
Body:
["replicated-map-1"]
Retrieving Replicated Map Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/replicatedmaps/<replicatedmapname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/replicatedmaps/replicated-map-1
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "replicated-map-1", "ownedEntryCount": 10955, "ownedEntryMemoryCost": 394380, "hits": 15, "lastAccessTime": 1532689312581, "lastUpdateTime": 1532689312581, "creationTime": 1532688789493, "putOperationCount": 11561, "getOperationCount": 1051, "removeOperationCount": 522, "otherOperationCount": 355552, "events": 6024, "maxPutLatency": 1, "maxGetLatency": 1, "maxRemoveLatency": 1, "totalPutLatency": 64, "totalGetLatency": 12, "totalRemoveLatency": 11 }
Caches Resource
This resource returns a list of caches belonging to the provided cluster.
Retrieving the list of Caches:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/caches
-
Request:
curl http://localhost:8080/rest/clusters/dev/caches
-
Response:
200
(application/json) -
Body:
["cache-1"]
Retrieving Cache Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/caches/<cachename>
-
Request:
curl http://localhost:8080/rest/clusters/dev/caches/cache-1
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "cache-1", "creationTime": 1532688789493, "hits": 100, "misses": 11, "getOperationCount": 231, "putOperationCount": 42, "removeOperationCount": 1, "evictions": 3, "averageGetTime": 10.5, "averagePutTime": 21.12, "averageRemoveTime": 23, "lastAccessTime": 1403602693411, "lastUpdateTime": 1403602693411, "ownedEntryCount": 300 }
Queues Resource
This resource returns a list of queues belonging to the provided cluster.
Retrieving the list of Queues:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/queues
-
Request:
curl http://localhost:8080/rest/clusters/dev/queues
-
Response:
200
(application/json) -
Body:
["messages"]
Retrieving Queue Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/queues/{queuename>
-
Request:
curl http://localhost:8080/rest/clusters/dev/queues/messages
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "messages", "ownedItemCount": 55408, "backupItemCount": 55408, "minAge": 0, "maxAge": 0, "averageAge": 0, "offerOperationCount": 55408, "rejectedOffers": 0, "pollOperationCount": 0, "emptyPolls": 0, "otherOperationCount": 0, "events": 0, "creationTime": 1403602694196 }
Topics Resource
This resource returns a list of topics and reliable topics belonging to the provided cluster.
Retrieving the list of Topics:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/topics
-
Request:
curl http://localhost:8080/rest/clusters/dev/topics
-
Response:
200
(application/json) -
Body:
["news"]
Retrieving Topic Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/topics/{topicname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/topics/news
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "news", "publishOperationCount": 56370, "receiveOperationCount": 56370, "creationTime": 1403602693411 }
Retrieving the list of Reliable Topics:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/reliabletopics
-
Request:
curl http://localhost:8080/rest/clusters/dev/reliabletopics
-
Response:
200
(application/json) -
Body:
["news"]
Retrieving Reliable Topic Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/reliabletopics/<reliableTopicname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/reliabletopics/news
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "news", "publishOperationCount": 56370, "receiveOperationCount": 56370, "creationTime": 1403602693411, }
Executors Resource
This resource returns a list of executors belonging to the provided cluster.
Retrieving the list of Executors:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/executors
-
Request:
curl http://localhost:8080/rest/clusters/dev/executors
-
Response:
200
(application/json) -
Body:
["order-executor"]
Retrieving Executor Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/executors/<executorname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/executors/order-executor
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "order-executor", "creationTime": 1403602694196, "pendingTaskCount": 0, "startedTaskCount": 1241, "completedTaskCount": 1241, "cancelledTaskCount": 0, "totalExecutionTime": 1000, "totalStartLatency": 400 }
PN Counters Resource
This resource returns a list of PN counters belonging to the provided cluster.
Retrieving the list of PN Counters:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/pncounters
-
Request:
curl http://localhost:8080/rest/clusters/dev/pncounters
-
Response:
200
(application/json) -
Body:
["order-pncounter"]
Retrieving PN Counter Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/pncounters/<pnCountername>
-
Request:
curl http://localhost:8080/rest/clusters/dev/pncounters/order-pncounter
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "order-pncounter", "creationTime": 1403602694196, "statsPerMember": { "192.168.2.78:5701": { "value": 1, "incOperationCount": 1, "decOperationCount": 0 }, "192.168.2.79:5701": { "value": 1, "incOperationCount": 0, "decOperationCount": 0 } } }
Flake ID Generators Resource
This resource returns a list of flake ID generators belonging to the provided cluster.
Retrieving the list of Flake ID Generators:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/flakeidgenerators
-
Request:
curl http://localhost:8080/rest/clusters/dev/flakeidgenerators
-
Response:
200
(application/json) -
Body:
["order-idgenerator"]
Retrieving Flake ID Generator Information:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/flakeidgenerators/{flakeIdGeneratorname>
-
Request:
curl http://localhost:8080/rest/clusters/dev/flakeidgenerators/order-idgenerator
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "name": "order-idgenerator", "creationTime": 1403602694196, "statsPerMember": { "192.168.2.78:5701": { "batchRequests": 1, "idCount": 100 }, "192.168.2.79:5701": { "batchRequests": 0, "idCount": 0 } } }
Client Statistics Resource
This resource returns a list of clients belonging to the provided cluster.
Retrieving the list of Client UUIDs:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/clientStats
-
Request:
curl http://localhost:8080/rest/clusters/dev/clientStats
-
Response:
200
(application/json) -
Body:
[ "f3b1e0e9-ea67-41b2-aba5-ea7480f02a93", "cebf4dc9-852c-4605-a181-ffe1cca371a4", "2371eed5-26e0-4470-92c1-41ea17110ef6", "139990b3-fbc0-43a8-9c12-be53913333f7", "d0364a1e-8665-46a8-af1d-be1af5580d07", "7f337f8a-3538-4b5c-8ffc-9d4ae459e956", "6ef9b6e5-5add-40d9-9319-ce502f55b5fc", "fead3a99-19de-431c-9dd0-d6ecc4a4b9c8", "e788e04e-2ded-4992-9d76-52c1973216e5", "654fc9fb-c5c1-48a0-9b69-0c129fce860f" ]
Retrieving Detailed Client Statistics:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/clientStats/<clientUuid>
-
Request:
curl http://localhost:8080/rest/clusters/dev/clientStats/2371eed5-26e0-4470-92c1-41ea17110ef6
-
Response:
200
(application/json) -
Body:
{ "type": "JAVA", "name": "hz.client_7", "address": "127.0.0.1", "clusterConnectionTimestamp": 1507874427419, "enterprise": true, "lastStatisticsCollectionTime": 1507881309434, "committedVirtualMemorySize": 12976173056, "freePhysicalMemorySize": 3615662080, "freeSwapSpaceSize": 8447324160, "maxFileDescriptorCount": 1000000, "openFileDescriptorCount": 191, "processCpuTime": 252980000000, "systemLoadAverage": 83.0, "totalPhysicalMemorySize": 16756101120, "totalSwapSpaceSize": 8447324160 "availableProcessors": 12, "freeHeapMemory": 135665432, "maxHeapMemory": 3724541952, "totalHeapMemory": 361234432, "usedHeapMemory": 225569000, "uptime": 6894992, "memberConnection": "ALL", "version": "UNKNOWN", "nearCacheStats":{ "CACHE":{ "cache-1":{ "clientUuid":"805a6342-eebb-412d-aeba-21c55fadedc3", "creationTime":1588752664324, "evictions":0, "expirations":0, "hits":148, "misses":164, "ownedEntryCount":4, "ownedEntryMemoryCost":420, "lastPersistenceDuration":0, "lastPersistenceKeyCount":0, "lastPersistenceTime":0, "lastPersistenceWrittenBytes":0 } }, "MAP":{ "map-1":{ "clientUuid":"805a6342-eebb-412d-aeba-21c55fadedc3", "creationTime":1588752664321, "evictions":0, "expirations":0, "hits":92, "misses":116, "ownedEntryCount":109, "ownedEntryMemoryCost":7701, "lastPersistenceDuration":0, "lastPersistenceKeyCount":0, "lastPersistenceTime":0, "lastPersistenceWrittenBytes":0 } } } }
WAN Publisher Resource
This resource returns information of a WAN replicated cluster’s publisher.
Retrieving WAN Publisher Statistics:
-
Request Type:
GET
-
URL:
/rest/clusters/<clustername>/wanStats/<wanReplication>/publishers/<publisher>
-
Request:
curl http://localhost:8080/rest/clusters/dev/wanStats/devWanConfig/publishers/devPublisher
-
Response:
200
(application/json) -
Body:
{ "cluster": "dev", "configName": "devWanConfig", "publisherId": "devPublisher", "totalPublishedEventCount": 1023, "totalPublishLatency": 14200, "outboundQueueSize": 10 }