A newer version of Hazelcast Platform is available.

View latest

Upgrading from Jet

6.0-SNAPSHOT 5.5 5.4 5.3 5.2 5.1 5.0

This guide covers the most important changes that affect users of Jet 4.x who want to upgrade to Hazelcast.

This guide is not an overview of the new features in Hazelcast Platform. This release includes other enhancements that don’t need any special attention during an upgrade.

Before you Upgrade

If you’re using Jet Enterprise version 4.x, you can run your existing jobs on an upgraded cluster. Just cancel any running jobs on your Jet cluster before upgrading. Then, resubmit the job to the upgraded cluster.

You will lose the state of any running jobs that have not yet completed.

For help canceling jobs, see Managing Jobs.

See Submitting Jobs.

Compatibility

Hazelcast Platform is compatible only with the Job and Pipeline APIs of Jet 4.x. As a result, you can resubmit jobs that were running on Jet 4.x clusters to a 5.1.7 cluster. But, you cannot do the following:

  • Upgrade to 5.1.7 using rolling upgrades.

  • Connect to a 5.1.7 cluster using version 4.x of clients.

  • Resubmit jobs from snapshots that were made on Jet 4.x clusters.

  • Use versions of Management Center before 5.1.7.

Jet Engine Security

In embedded mode (Hazelcast in the same JVM as your application), you cannot use Jet APIs or SQL by default. These features are disabled for security reasons. For more information about security in the Jet engine and how to enable/disable the Jet engine, see Securing Jobs.

Configuration

By default, Hazelcast Platform looks for configurations in the following order and applies only the first one that it finds:

  1. A Config object

  2. A YAML or XML file whose path is given in the hazelcast.config system property

  3. A hazelcast.xml file in the working directory

  4. A hazelcast.xml file on the classpath

  5. A hazelcast.yaml file in the working directory

  6. A hazelcast.yaml file on the classpath

You can also override these configurations, using system properties or environment variables. See Member Configuration Files.

Jet 4.x did not offer an option to configure it using XML. As a result, you must do one of the following:

  • Add your configurations to the Config object.

    Some of the JetConfig API has been moved to the Config object. See IMDG Configuration, InstanceConfig, and Configuration Loaders, and Jet Properties on this page.

  • Pass the path of your YAML file to the hazelcast.config system property.

    For the Jet engine related configuration fields, see the example YAML file. NOTE: If your YAML file is invalid, the member will not start. Hazelcast Platform checks and validates your YAML configurations during a cluster startup. See YAML File Validation on this page.

  • Add your configurations to the hazelcast.xml file.

    For the Jet engine related configuration elements, see the example XML file on GitHub.

IMDG Configuration

The following methods for configuring the underlying IMDG instance of Jet have been removed from the JetConfig object:

  • getHazelcastConfig()

  • configureHazelcast()

  • setHazelcastConfig()

If you used the JetConfig object to configure IMDG, you should replace instances of JetConfig with Config.

  • Jet 4.x

  • Hazelcast

JetConfig config = new JetConfig().configureHazelcast(c -> {
  c.getNetworkConfig().setPort(8000);
  c.setClusterName("jet-dev");
  });

JetInstance jet = Jet.newJetInstance(config);
Config config = new Config() ;
config.setPort(8000);
config.setClusterName("jet-dev");

HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);

InstanceConfig

The InstanceConfig object has been deprecated and all its methods have been added to the JetConfig object.

  • Jet 4.x

  • Hazelcast

JetConfig jetConfig = new JetConfig();
jetConfig.getInstanceConfig().setCooperativeThreadCount(4);
Config config = new Config();
config.getJetConfig().setCooperativeThreadCount(4);

Configuration Loaders

All configuration loader methods have been moved from the JetConfig object to the Config (Hazelcast configuration) object. Some of these methods have been renamed. If you used Jet configuration loaders, use the following table to find out what you should replace in your code:

Jet method Hazelcast method

JetConfig.loadDefault()

Config.loadDefault() or Config.load()

JetConfig.loadFromClasspath()

Config.loadFromClasspath()

JetConfig.loadFromFile()

Config.loadFromFile()

JetConfig.loadXmlFromStream()

Config.loadFromStream()

JetConfig.loadXmlFromString()

Config.loadFromString()

JetConfig.loadYamlFromStream()

Config.loadFromStream()

JetConfig.loadYamlFromString()

Config.loadFromString()

Jet Properties

In the Java API, properties in the JetProperties object have been merged into the ClusterProperty object.

The following Jet properties have been removed:

  • jet.home

  • jet.imdg.version.mismatch.check.disabled

All Jet system properties are now prefixed with hazelcast. For example jet.job.scan.period is now hazelcast.jet.job.scan.period. Any Jet system properties that do not include this prefix have been deprecated.

If you use the Java API to set system properties, you must now use the Config object instead of the JetConfig object.

  • Jet 4.x

  • Hazelcast

JetConfig config = new JetConfig() ;
config.setProperty( "hazelcast.property.foo", "value" );
Config config = new Config() ;
config.setProperty( "hazelcast.property.foo", "value" );

See Configuring with System Properties and Environment Variables.

YAML File Validation

Hazelcast Platform checks and validates your YAML configurations during a cluster startup. According to this validation:

  • The top-level hazelcast object must exist.

  • Client and member YAML configurations must be separate (not in the same file).

  • There must be no case insensitive enum values.

While upgrading to Hazelcast Platform, if a YAML configuration violates any of these rules, the cluster will not start. You need to either edit and update your YAML configuration files or disable the validation by setting the hazelcast.config.schema.validation.enabled property to false.

API Entry Points

The Jet class, which was the main entry point of Jet 4.x, has been deprecated and replaced by the HazelcastInstance class.

The JetInstance class, which represented an instance of a Jet member or client has been been deprecated and replaced by the JetService class. To access Jet related services, you should now use the HazelcastInstance.getJet() method to get an instance of the JetService object.

  • Jet 4.x

  • Hazelcast

JetInstance jet = Jet.newJetInstance();
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
JetService jet = hz.getJet();

Use the following table to find out which new class to use instead of JetInstance`.

Table 1. JetInstance Replacements
Usage New class

Submitting streaming/batch jobs to the cluster and managing them.

JetService

Accessing Hazelcast data structures.

HazelcastInstance

The only exception is Jet observables. An observable is a Jet data structure that is part of the JetService class.

Performing cluster operations such as shutting down the cluster.

HazelcastInstance

The JetInstance.bootstrappedInstance() method has also been deprecated and replaced by HazelcastInstance.bootstrappedInstance().

  • Jet 4.x

  • Hazelcast

JetInstance jet = Jet.bootstrappedInstance();
HazelcastInstance hz = Hazelcast.bootstrappedInstance();
JetService jet = hz.getJet();

Lossless Cluster Restart

The hot-restart-persistence configuration has been renamed to persistence. If you use lossless cluster restart, you must update your configuration with this change.

  • XML

  • YAML

<hazelcast>
  <persistence enabled="true">
    <base-dir>/mnt/persistence</base-dir>
    <backup-dir>/mnt/hot-backup</backup-dir>
  </persistence>
  <jet>
    <instance>
      <lossless-restart-enabled>true</lossless-restart-enabled>
    </instance>
  </jet>
</hazelcast>
hazelcast:
  persistence:
    enabled: true
    base-dir: /mnt/persistence
    backup-dir: /mnt/hot-backup
  jet:
    instance:
      lossless-restart-enabled: true

Persistence files are now saved to the directory that’s set in the persistence.base-dir field. In Jet 4.x, these files were saved to the directory that was set in the jet.home property. However, this property has been removed. See the changes to Jet properties on this page.

See Configuring Persistence.

SQL

The slim distribution of Hazelcast Platform does not include the SQL module. If you use the slim distribution with SQL, download the hazelcast-sql module. For a complete list of contents in each distribution of Hazelcast Platform, see Full and Slim Packages.

In the information_schema.mappings table, the following column names have been changed to make them consistent with the ANSI SQL standard. If you queried this table in Jet, make sure to use the correct column names in Hazelcast Platform.

Table 2. Changes to column names in the information_schema.mapping tables
Column name in Jet Column name in Hazelcast Platform

mapping_catalog

table_catalog

mapping_schema

table_schema

mapping_name

table_name

The JSON serialization format has been renamed to json-flat, and the JSON_FILE() table function has been renamed to JSON_FLAT_FILE. If you queried JSON with SQL in Jet, make sure to replace json with json-flat.

  • Jet 4.x

  • Hazelcast

CREATE MAPPING my_files
TYPE File
OPTIONS (
    'path' = '/path/to/directory',
    'format' = 'json'
)
SELECT * FROM TABLE(
  JSON_FILE(path => '/path/to/directory')
);
CREATE MAPPING my_files
TYPE File
OPTIONS (
    'path' = '/path/to/directory',
    'format' = 'json-flat'
)
SELECT * FROM TABLE(
  JSON_FLAT_FILE(path => '/path/to/directory')
);

See SQL.

Code Samples

Jet code samples have been moved to the Hazelcast code samples repository. See GitHub.

Scripts

Most scripts in the bin directory have been renamed. If you have any automated processes that use these scripts, update them to use the new scripts.

Table 3. Comparison of scripts in Jet 4.x and Hazelcast Platform
Jet 4.x Hazelcast Platform

/bin

 — common.sh

 — jet

 — jet-cluster-admin

 — jet-cluster-cp-admin

 — jet-start

 — jet-start.bat

 — jet-stop

 — jet-stop.bat

 — jet.bat

/bin

 — common.sh

 — hz-cli

 — hz-cluster-admin

 — hz-cluster-cp-admin

 — hz-start

 — hz-start.bat

 — hz-stop

 — hz-stop.bat

 — hz-cli.bat

 — hz-healthcheck
lifebuoy

Help and support