Evaluating 5.x (For IMDG 3.x Users)

Hazelcast offers an evaluation tool for IMDG 3.x users. Using it, you can try out some of the Hazelcast 5.x features on your existing data in the IMDG 3.x cluster.

With this tool, you can:

use existing map data in 3.x and 5.x clusters as sources and sinks so that you can transfer data between them
use map or replicated map data from a 3.x cluster to perform stream data enrichment in a 5.x cluster

See Sources and Sinks and Enrich a Stream to learn more about these functionalities.

The evaluation tool is basically a connector which can be used to perform the above tasks.

The evaluation tool is not intended and recommended for production usage. It supports only the legacy Hazelcast IMDG 3.x versions between 3.6.x and 3.12.x and Platform 5.x versions.

Get the Evaluation Tool

The tool is included in the Hazelcast full distribution package; see here for information on the packages.

You can also have it using Maven under com.hazelcast with the name "Hazelcast 3 Connector Interface". You need to add the following modules as dependencies to your pom.xml:

hazelcast-3-connector-common
hazelcast-3-connector-interface
hazelcast-3-connector-impl

Copy Data From 3.x to 5.x

Using the evaluation tool, you can read data from a map in a 3.x cluster, i.e., it can be a source for 5.x. By this way, it is possible to copy data from a 3.x map to a 5.x map. In this case, the 3.x cluster behaves as the remote cluster. Follow the below steps.

You need to connect to the remote (3.x) cluster from 5.x. For this, a client configuration should be provided to identify the 3.x cluster. This configuration must conform to the 3.x schema; it should be provided as an XML document in a string. See the example below.

String HZ3_CLIENT_CONFIG =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
   + "<hazelcast-client xmlns=\"http://www.hazelcast.com/schema/client-config\"\n"
   + "                  xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n"
   + "                  xsi:schemaLocation=\"http://www.hazelcast.com/schema/client-config\n"
   + "                  http://www.hazelcast.com/schema/client-config/hazelcast-client-config-3.12.xsd\">\n"
   + "\n"
   + "    <network>\n"
   + "        <cluster-members>\n"
   + "            <address>127.0.0.1:3210</address>\n"
   + "        </cluster-members>\n"
   + "    </network>\n"
   + "</hazelcast-client>\n";

Use the tool’s remoteMap() method to specify the map in the 3.x cluster (source) by providing its name and the client configuration mentioned above:
```
BatchSource<Entry<Object, Object>> source = Hz3Sources.remoteMap(sourceMap, HZ3_CLIENT_CONFIG);
```
Specify the map in the 5.x cluster (target) to which the 3.x map data will be copied:
```
Sink<Entry<Object, Object>> sink = Sinks.map(targetMap);
```

Create the pipeline to copy the 3.x map data to 5.x map:

Pipeline p = Pipeline.create();
p.readFrom(source)
 .writeTo(sink);

Configure the custom classpaths to separate the libraries for the 3.x member/client and evaluation tool (connector):
```
List<String> jars = new ArrayList<>();
jars.add("hazelcast-3.12.12.jar");
jars.add("hazelcast-client-3.12.12.jar");
jars.add("hazelcast-3-connector-impl.jar");
JobConfig config = new JobConfig();
config.addCustomClasspaths(source.name(), jars)
```
Above libraries (JAR files) should exist in the directory specified by the hazelcast.jet.custom.lib.dir property. This is already set up for the distributions of 3.x member/client and connector.
Copy the map data from 3.x to 5.x using a Jet engine job:
```
Job job = hz.getJet().newJob(p, config);
job.join();
```

Copy Data From 5.x to 3.x

Using the evaluation tool, you can also copy data from a 5.x map to a 3.x map. Follow the below steps.

You need to connect to the remote (3.x) cluster from 5.x; for this, a client configuration should be provided to identify the 3.x cluster. This configuration must conform to the 3.x schema; it should be provided as an XML document in a string. See the example below.

String HZ3_CLIENT_CONFIG =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
   + "<hazelcast-client xmlns=\"http://www.hazelcast.com/schema/client-config\"\n"
   + "                  xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n"
   + "                  xsi:schemaLocation=\"http://www.hazelcast.com/schema/client-config\n"
   + "                  http://www.hazelcast.com/schema/client-config/hazelcast-client-config-3.12.xsd\">\n"
   + "\n"
   + "    <network>\n"
   + "        <cluster-members>\n"
   + "            <address>127.0.0.1:3210</address>\n"
   + "        </cluster-members>\n"
   + "    </network>\n"
   + "</hazelcast-client>\n";

Specify the map in the 5.x cluster (source):

BatchSource<Entry<Object, Object>> source = Sources.map(sourceMap);

Specify the map in the 3.x cluster (target) to which the 5.x map data will be copied, by providing its name and the client configuration mentioned above:
```
Sink<Entry<Object, Object>> sink = Hz3Sinks.remoteMap(targetMap, HZ3_CLIENT_CONFIG);
```

Create the pipeline to copy the 5.x map data to 3.x map:

Pipeline p = Pipeline.create();
p.readFrom(source)
 .writeTo(sink);

Configure the custom classpaths to separate the libraries for the 3.x member/client and evaluation tool (connector):
```
List<String> jars = new ArrayList<>();
jars.add("hazelcast-3.12.12.jar");
jars.add("hazelcast-client-3.12.12.jar");
jars.add("hazelcast-3-connector-impl.jar");
JobConfig config = new JobConfig();
config.addCustomClasspaths(source.name(), jars)
```
Above libraries (JAR files) should exist in the directory specified by the hazelcast.jet.custom.lib.dir property. This is already set up for the distributions of 3.x member/client and connector.
Copy the map data from 5.x to 3.x using a Jet engine job:
```
Job job = hz.getJet().newJob(p, config);
job.join();
```

Enrich Stream Data from Hazelcast 3.x

Using the evaluation tool, you can enrich a stream data in a 5.x cluster using a map or replicated map from a 3.x cluster. See Enrich a Stream for information on this functionality. Follow the below steps.

String HZ3_CLIENT_CONFIG =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
   + "<hazelcast-client xmlns=\"http://www.hazelcast.com/schema/client-config\"\n"
   + "                  xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n"
   + "                  xsi:schemaLocation=\"http://www.hazelcast.com/schema/client-config\n"
   + "                  http://www.hazelcast.com/schema/client-config/hazelcast-client-config-3.12.xsd\">\n"
   + "\n"
   + "    <network>\n"
   + "        <cluster-members>\n"
   + "            <address>127.0.0.1:3210</address>\n"
   + "        </cluster-members>\n"
   + "    </network>\n"
   + "</hazelcast-client>\n";

Create a ServiceFactory for the map in the 3.x cluster by providing its name and the client configuration mentioned above. This factory provides functions to create and destroy objects to be used in the stream enrichment pipeline:
```
ServiceFactory<Hz3MapAdapter, AsyncMap<Integer, String>> hz3MapSF =
    hz3MapServiceFactory("testMap", HZ3_CLIENT_CONFIG);
```

Use this factory in a pipeline’s computational step (stage) which reads data from the 3.x map:

Pipeline p = Pipeline.create();
BatchStage<String> mapStage = p.readFrom(TestSources.items(1, 2, 3))
 .mapUsingService(
   hz3MapSF,
   mapUsingIMap(FunctionEx.identity(), (Integer i, String s) -> s)
 );
mapStage.writeTo(Sinks.list(results));

Configure the custom classpaths to separate the libraries for the 3.x member/client and evaluation tool (connector):
```
List<String> jars = new ArrayList<>();
jars.add("hazelcast-3.12.12.jar");
jars.add("hazelcast-client-3.12.12.jar");
jars.add("hazelcast-3-connector-impl.jar");
JobConfig config = new JobConfig();
config.addCustomClasspaths(mapStage.name(), jars)
```
Above libraries (JAR files) should exist in the directory specified by the hazelcast.jet.custom.lib.dir property. This is already set up for the distributions of 3.x member/client and connector.
Create the Jet engine job with the pipeline stage that reads from the 3.x map:
```
hz.getJet().newJob(p, config);
```

Serialization Considerations

Since the connector sends/receives data over a network to/from a store, the data objects go through a (de)serialization.

If your data objects use the following serialization interfaces on your 3.x cluster, then you need to implement and register the related object factories/serializers on the 5.x client/member side:

See the above links for information on how you can implement and register object factories or serializers.

Implementation and/or registration of factories and custom serializers should be done only on the Hazelcast Platform side; there is no need to to register a factory/serializer in the 3.x client configuration which is used by the connector.

If your data objects use the following serialization interfaces, you do not need to perform additional steps as for the above ones; Hazelcast takes care of the serializations out of the box:

Code Sample

See also the Hazelcast 3 Connector code sample.

The code sample does the following:

Start a 3.x member which represents an existing cluster with your data.
Run a Jet engine job CopyMapFromHz3Example to copy the data from a map in 3.x cluster to a map in 5.x cluster.
Start an empty 3.x member and run a Jet engine job to copy data from a map in 5.x cluster back to a map on 3.x cluster.
Run a Jet engine job which uses a map in 3.x cluster to enrich data in a stream job running in 5.x cluster.

Edit this page Report an issue