Upgrading from Jet

This guide covers the most important changes that affect users of Jet 4.x who want to upgrade to Hazelcast.

This guide is not an overview of the new features in Hazelcast Platform. This release includes other enhancements that don't need any special attention during an upgrade.

Before you Upgrade

If you're using Jet Enterprise version 4.x, you can run your existing jobs on an upgraded cluster. Just cancel any running jobs on your Jet cluster before upgrading. Then, resubmit the job to the upgraded cluster.

warning

You will lose the state of any running jobs that have not yet completed.

For help canceling jobs, see Managing Jobs.

See Submitting Jobs.

Compatibility

Hazelcast Platform is compatible only with the Job and Pipeline APIs of Jet 4.x. As a result, you can resubmit jobs that were running on Jet 4.x clusters to a 6.0.0-SNAPSHOT cluster. But, you cannot do the following:

• Upgrade to 6.0.0-SNAPSHOT using rolling upgrades.
• Connect to a 6.0.0-SNAPSHOT cluster using version 4.x of clients.
• Resubmit jobs from snapshots that were made on Jet 4.x clusters.
• Use versions of Management Center before 5.13-snapshot.

Jet Engine Security

In embedded mode (Hazelcast in the same JVM as your application), you cannot use Jet APIs or SQL by default. These features are disabled for security reasons. For more information about security in the Jet engine and how to enable/disable the Jet engine, see Securing Jobs.

Configuration

By default, Hazelcast Platform looks for configurations in the following order and applies only the first one that it finds:

1. A Config object
2. A YAML or XML file whose path is given in the hazelcast.config system property
3. A hazelcast.xml file in the working directory
4. A hazelcast.xml file on the classpath
5. A hazelcast.yaml file in the working directory
6. A hazelcast.yaml file on the classpath

note

You can also override these configurations, using system properties or environment variables. See Member Configuration Files.

Jet 4.x did not offer an option to configure it using XML. As a result, you must do one of the following:

• Add your configurations to the Config object.

  Some of the JetConfig API has been moved to the Config object. See IMDG Configuration, InstanceConfig, and Configuration Loaders, and Jet Properties on this page.

• Pass the path of your YAML file to the hazelcast.config system property.

  For the Jet engine related configuration fields, see the example YAML file. NOTE: If your YAML file is invalid, the member will not start. Hazelcast Platform checks and validates your YAML configurations during a cluster startup. See YAML File Validation on this page.

• Add your configurations to the hazelcast.xml file.

  For the Jet engine related configuration elements, see the example XML file on GitHub.

IMDG Configuration

The following methods for configuring the underlying IMDG instance of Jet have been removed from the JetConfig object:

• getHazelcastConfig()
• configureHazelcast()
• setHazelcastConfig()

If you used the JetConfig object to configure IMDG, you should replace instances of JetConfig with Config.

Jet 4.x

java

JetConfig config = new JetConfig().configureHazelcast(c -> {
  c.getNetworkConfig().setPort(8000);
  c.setClusterName("jet-dev");
  });

JetInstance jet = Jet.newJetInstance(config);

Hazelcast

java

Config config = new Config() ;
config.setPort(8000);
config.setClusterName("jet-dev");

HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);

InstanceConfig

The InstanceConfig object has been deprecated and all its methods have been added to the JetConfig object.

Jet 4.x

java

JetConfig jetConfig = new JetConfig();
jetConfig.getInstanceConfig().setCooperativeThreadCount(4);

Hazelcast

java

Config config = new Config();
config.getJetConfig().setCooperativeThreadCount(4);

Configuration Loaders

All configuration loader methods have been moved from the JetConfig object to the Config (Hazelcast configuration) object. Some of these methods have been renamed. If you used Jet configuration loaders, use the following table to find out what you should replace in your code:

Jet method	Hazelcast method
JetConfig.loadDefault()	Config.loadDefault() or Config.load()
JetConfig.loadFromClasspath()	Config.loadFromClasspath()
JetConfig.loadFromFile()	Config.loadFromFile()
JetConfig.loadXmlFromStream()	Config.loadFromStream()
JetConfig.loadXmlFromString()	Config.loadFromString()
JetConfig.loadYamlFromStream()	Config.loadFromStream()
JetConfig.loadYamlFromString()	Config.loadFromString()

Jet Properties

In the Java API, properties in the JetProperties object have been merged into the ClusterProperty object.

The following Jet properties have been removed:

• jet.home
• jet.imdg.version.mismatch.check.disabled

All Jet system properties are now prefixed with hazelcast. For example jet.job.scan.period is now hazelcast.jet.job.scan.period. Any Jet system properties that do not include this prefix have been deprecated.

If you use the Java API to set system properties, you must now use the Config object instead of the JetConfig object.

Jet 4.x

java

JetConfig config = new JetConfig() ;
config.setProperty( "hazelcast.property.foo", "value" );

Hazelcast

java

Config config = new Config() ;
config.setProperty( "hazelcast.property.foo", "value" );

See Configuring with System Properties and Environment Variables.

YAML File Validation

Hazelcast Platform checks and validates your YAML configurations during a cluster startup. According to this validation:

• The top-level hazelcast object must exist.
• Client and member YAML configurations must be separate (not in the same file).
• There must be no case-insensitive enum values.

While upgrading to Hazelcast Platform, if a YAML configuration violates any of these rules, the cluster will not start. You need to either edit and update your YAML configuration files or disable the validation by setting the hazelcast.config.schema.validation.enabled property to false.

API Entry Points

The Jet class, which was the main entry point of Jet 4.x, has been deprecated and replaced by the HazelcastInstance class.

The JetInstance class, which represented an instance of a Jet member or client has been deprecated and replaced by the JetService class. To access Jet related services, you should now use the HazelcastInstance.getJet() method to get an instance of the JetService object.

Jet 4.x

java

JetInstance jet = Jet.newJetInstance();

Hazelcast

java

HazelcastInstance hz = Hazelcast.newHazelcastInstance();
JetService jet = hz.getJet();

Use the following table to find out which new class to use instead of JetInstance`.

Usage	New class
Submitting streaming/batch jobs to the cluster and managing them.	JetService
Accessing Hazelcast data structures.	HazelcastInstance The only exception is Jet observables. An observable is a Jet data structure that is part of the JetService class.
Performing cluster operations such as shutting down the cluster.	HazelcastInstance

The JetInstance.bootstrappedInstance() method has also been deprecated and replaced by HazelcastInstance.bootstrappedInstance().

Jet 4.x

java

JetInstance jet = Jet.bootstrappedInstance();

Hazelcast

java

HazelcastInstance hz = Hazelcast.bootstrappedInstance();
JetService jet = hz.getJet();

Lossless Cluster Restart

The hot-restart-persistence configuration has been renamed to persistence. If you use lossless cluster restart, you must update your configuration with this change.

XML

xml

<hazelcast>
  <persistence enabled="true">
    <base-dir>/mnt/persistence</base-dir>
    <backup-dir>/mnt/hot-backup</backup-dir>
  </persistence>
  <jet>
    <instance>
      <lossless-restart-enabled>true</lossless-restart-enabled>
    </instance>
  </jet>
</hazelcast>

YAML

yaml

hazelcast:
  persistence:
    enabled: true
    base-dir: /mnt/persistence
    backup-dir: /mnt/hot-backup
  jet:
    instance:
      lossless-restart-enabled: true

Persistence files are now saved to the directory that's set in the persistence.base-dir field. In Jet 4.x, these files were saved to the directory that was set in the jet.home property. However, this property has been removed. See the changes to Jet properties on this page.

See Configuring Persistence.

SQL

The slim distribution of Hazelcast Platform does not include the SQL module. If you use the slim distribution with SQL, download the hazelcast-sql module. For a complete list of contents in each distribution of Hazelcast Platform, see Full and Slim Packages.

In the information_schema.mappings table, the following column names have been changed to make them consistent with the ANSI SQL standard. If you queried this table in Jet, make sure to use the correct column names in Hazelcast Platform.

Column name in Jet	Column name in Hazelcast Platform
mapping_catalog	table_catalog
mapping_schema	table_schema
mapping_name	table_name

The JSON serialization format has been renamed to json-flat, and the JSON_FILE() table function has been renamed to JSON_FLAT_FILE. If you queried JSON with SQL in Jet, make sure to replace json with json-flat.

Jet 4.x

sql

CREATE MAPPING my_files
TYPE File
OPTIONS (
    'path' = '/path/to/directory',
    'format' = 'json'
)

sql

SELECT * FROM TABLE(
  JSON_FILE(path => '/path/to/directory')
);

Hazelcast

sql

CREATE MAPPING my_files
TYPE File
OPTIONS (
    'path' = '/path/to/directory',
    'format' = 'json-flat'
)

sql

SELECT * FROM TABLE(
  JSON_FLAT_FILE(path => '/path/to/directory')
);

See SQL.

Code Samples

Jet code samples have been moved to the Hazelcast code samples repository. See GitHub.

Scripts

Most scripts in the bin directory have been renamed. If you have any automated processes that use these scripts, update them to use the new scripts.

Jet 4.x	Hazelcast Platform
/bin — common.sh — jet — jet-cluster-admin — jet-cluster-cp-admin — jet-start — jet-start.bat — jet-stop — jet-stop.bat — jet.bat	/bin — common.sh — hz-cli — hz-cluster-admin — hz-cluster-cp-admin — hz-start — hz-start.bat — hz-stop — hz-stop.bat — hz-cli.bat — hz-healthcheck