A newer version of IMDG is available.

View latest

Want to try Hazelcast Platform?

We’ve combined the in-memory storage of IMDG with the stream processing power of Jet to bring you the all new Hazelcast Platform.

Durable Executor Service

4.2 4.1 4.0 3.12 3.11 3.10 3.9 3.8 3.7 3.6 3.5

Hazelcast’s durable executor service is a data structure which is able to store an execution task both on the executing Hazelcast member and its backup member(s), if configured. By this way, you do not lose any tasks if a member goes down or any results if the submitter (member or client) goes down while executing the task. When using the durable executor service you can either submit or execute a task randomly or on the owner of a provided key. Note that in executor service, you can submit or execute tasks to/on the selected member(s).

Processing of the tasks when using durable executor service involves two invocations:

  1. Sending the task to primary Hazelcast member (primary partition) and to its backups, if configured, and executing the task.

  2. Retrieving the result of the task.

As you may already know, Hazelcast’s executor service returns a future representing the task to the user. With the above two-invocations approach, it is guaranteed that the task is executed before the future returns and you can track the response of a submitted task with a unique ID. Hazelcast stores the task on both primary and backup members, and starts the execution also.

With the first invocation, a Ringbuffer stores the task and a generated sequence for the task is returned to the caller as a result. In addition to the storing, the task is executed on the local execution service for the primary member. By this way, the task is now resilient to member failures and you are able to track the task with its ID.

After the first invocation has completed and the sequence of task is returned, second invocation starts to retrieve the result of task with that sequence. This retrieval waits in the waiting operations queue until notified, or it runs immediately if the result is already available.

When task execution is completed, Ringbuffer replaces the task with the result for the given task sequence. This replacement notifies the waiting operations queue.

Configuring Durable Executor Service

This section presents example configurations for durable executor service along with the descriptions of its configuration elements and attributes.

Declarative Configuration:

Declarative Configuration:

<hazelcast>
    ...
    <durable-executor-service name="myDurableExecSvc">
        <pool-size>8</pool-size>
        <durability>1</durability>
        <capacity>1</capacity>
        <quorum-ref>quorumname</quorum-ref>
    </durable-executor-service>
    ...
</hazelcast>

Programmatic Configuration:

        Config config = new Config();
        config.getDurableExecutorConfig( "myDurableExecSvc" )
                .setPoolSize ( 8 )
                .setDurability( 1 )
                .setCapacity( 1 )
                .setQuorumName( "quorumname" );

        HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance(config);
        DurableExecutorService durableExecSvc = hazelcast.getDurableExecutorService("myDurableExecSvc");

The following are the descriptions of each configuration element and attribute:

  • name: Name of the executor task.

  • statistics-enabled: Specifies whether the statistics gathering is enabled. If set to false, you cannot collect statistics.

  • pool-size: Number of executor threads per member for the executor.

  • durability: Number of backups in the cluster for the submitted task. Its default value is 1.

  • capacity: Executor’s task queue capacity; the number of tasks this queue can hold.

  • quorum-ref: Name of quorum configuration that you want this Durable Executor Service to use. See the Split-Brain Protection for Durable Executor Service section.

Split-Brain Protection for Durable Executor Service

Durable Executor Service can be configured to check for a minimum number of available members before applying its operations (see the Split-Brain Protection section). This is a check to avoid performing successful queue operations on all parts of a cluster during a network partition.

The following is a list of methods, grouped by quorum type, that support split-brain protection checks:

  • WRITE, READ_WRITE:

    • disposeResult

    • execute

    • executeOnKeyOwner

    • retrieveAndDisposeResult

    • shutdown

    • shutdownNow

    • submit

    • submitToKeyOwner

  • READ, READ_WRITE:

    • retrieveResult

Configuring Split-Brain Protection

Split-brain protection for Durable Executor Service can be configured programmatically using the method setQuorumName(), or declaratively using the element quorum-ref. Following is an example declarative configuration:

<hazelcast>
    ...
    <durable-executor-service name="myDurableExecSvc">
        <quorum-ref>quorumname</quorum-ref>
    </durable-executor-service>
    ...
</hazelcast>

The value of quorum-ref should be the quorum configuration name which you configured under the quorum element as explained in the Split-Brain Protection section.

lifebuoy

Help and support