Release Notes

For Google Cloud Storage (GCS) file sources, the SecuredFunction interface now returns a list of permissions instead of a single permission. Previously, Hazelcast was returning only the permission for the bucket to be read, and this was causing a source to be possibly exploited for gaining information whether some file exists in the system. Now, Hazelcast also returns a permission for the keyfile that is required to read a bucket on GCS.

The "Hot Restart Persistence" feature name has been changed to "Persistence"; with this, we’ve performed the following minor changes in its API:

After the unification of cloud discovery plugins (see here), the exception class for the failure in generating SSLSocketFactory is changed from KubernetesClientException to RestClientException for the Kubernetes plugin. #19132

Changed the type of YEAR field in the SQL client protocol from short to int. #18984

Since the HOST and RACK metadata were deprecated in the previous Hazelcast release, these information are removed from the ZONE_AWARE partition grouping configuration. #18780

Since DAG printing was not reflecting the real queue size, the DAG.toDotString(int defaultParallelism) method signature is changed as DAG.toDotString(int defaultLocalParallelism, int defaultQueueSize). You, now, need to supply the queue size that will be shown if it is not overriden on the edge. #18475

Removed the following properties that were used in the former Jet product, since it is now part of Hazelcast Platform:

Hazelcast distribution is now available in two versions: full and slim. While the full one includes Hazelcast and all of its modules along with SQL, Jet engine extensions and Management Center, the slim one only includes the Hazelcast runtime, default configurations and scripts. See here for details. #18990, #18989

Consolidated the scripts delivered with the former IMDG and Jet distributions. See the Changes in Distribution Packaging section. #18999

The former hazelcast-sql, hazelcast-sql-core and hazelcast-jet-sql Maven modules in the distributions have been merged into a single hazelcast-sql module as a part of the Hazelcast Platform distribution. #18584

Removed the jackson-core dependency from pom.xml since it was breaking the extensions that depend on jackson. #18665

Moved the former Hazelcast Jet code samples into the Hazelcast code samples repository. #475

Added support of rolling upgrades for SQL. #19026

Added support of reading data from High-Density Memory Store backed maps via the SQL engine. #19328

Added support of regular and index (migration tolerant) scan of High-Density Memory Store via SQL. #19227

Tables were called mapping in information_schema.mappings and table in information_schema.columns. This inconsistency has been fixed by renaming mapping as table. #19210

Added support of the CONCAT_WS function, which takes variable sized VARCHAR (minimum 3) and returns a VARCHAR that consists of the concatenation of the arguments except the first one using the first argument as a separator. #19094

The 'SELECT' statement now also supports queries without the FROM clause so that you can submit queries like SELECT rand() without this clause. #19030

The Jet engine jobs submitted in a Hazelcast cluster are now cancelled when you upgrade your Hazelcast version since the Jet engine doesn’t provide backwards compatibility. #19012

Implemented the partition-tolerant index scan processor for Hazelcast maps: during a partition migration, this processor searches all the migrated partitions on all available cluster members. #18968

Added support of the putIfAbsentAsync() method for maps on the member side; which is required for the usage of INSERT INTO statements in SQL queries. #18946

Added support of returning nested fields without having to deserialize them, which enables you to use Portable in client/server deployments without touching the server side; for example, SQL queries can now return columns without having the class on the server-side classpath. #18922

Standardized the TIME and TIMESTAMP temporal formats for the SQL engine: You can now use TIME without leading zeroes and TIMESTAMP with space instead of the T symbol. Also added support of leading non-zero characters for the DATE formats. #18881, #18842

Added support of OFFSET for SQL queries. #18866

Implemented IdentifiedDataSerializable for SQL schema objects. #18851

Changed the since tags in Jet engine API and its extension modules from @since x.y to @since Jet x.y. #18832

Implemented the OnHeapMapScanP class to read the Hazelcast maps directly by the SQL engine. #18685

Implemented a basic memory management for the SQL engine so that number of records accumulated by it can be limited to avoid out of memory failures. You can use the max-processor-accumulated-records configuration element for this purpose. #18671

Added support of dynamic parameters for the SQL engine and file table functions. #18613, #18522

Introduced QueryDataType.MAP and QueryDataTypeFamily.MAP to support map operand checks for file table functions. #18602

Added support of EXTRACT(field FROM source) for the SQL engine. The function computes date parts from the source field. The supported types for source argument are as follow:

Added support of the LIMIT <n> and ORDER BY clauses for the streaming engine. #18479

Implemented the following functions for the SQL engine:

Added support of plan caching for Jet engine based queries. #18446

Added support of plus and minus operations for interval types (date, time, etc.) for the SQL engine. #18390

Added support of various new Portable types for the SQL engine. #18115

Added support of IN and BETWEEN operators for the SQL queries. #18483, #18422, #18067

The previous Replicated Map implementation was iterating all the values while calculating the size of map; this was causing latencies and performance issues as the entries in a Replicated Map grows. The related size() method has been refactored to eliminate the aforementioned situation. #19005

In Kubernetes, Hazelcast resolves its public addresses by finding an individual service that points to the given Hazelcast pod. If there are multiple services pointing to one pod, then the discovery could not work or might have chosen the wrong service. The following changes have been made to address this:

The code of the AWS, Azure, Kubernetes and GCP discovery plugins' in their own Github repos have been moved into the hazelcast/hazelcast repo. Their documentation also has been merged and unified into Hazelcast documentation. #19132

Added Kubernetes plugin’s configuration file for role based access control into the hazelcast/hazelcast Github repository as kubernetes-rbac.yaml. #19093

Added support of default serializers for the following classes which has been necessary for non-Java clients to use these:

Added an example Hazelcast configuration file (hazelcast-security-hardened.yaml) focused on hardened security to the distribution packages; it lists configuration options with their descriptions which may help securing your Hazelcast deployment. #18843

Introduced the simple authentication configuration; it allows to have users and their assigned roles stored together with other Hazelcast configurations. See the example:

hazelcast:
  security:
    enabled: true
    realms:
      - name: simpleRealm
        authentication:
          simple:
            users:
              - username: test
                password: 'a1234'
                roles:
                  - monitor
                  - hazelcast
              - username: root
                password: 'secret'
                roles:
                  - admin

Unless you explicitly disable them, the Merkle Trees are now enabled automatically when your cluster has a map or cache whose persistence is enabled; this is to improve a single member recovery from a crash, and it does not have a high memory overhead. #19502

Added the expose-externally configuration parameter for objects that expose an external IP address In. Kubernetes. See Configuring Kubernetes for its description.

The properties provided in former JetProperties are now merged into ClusterProperty. Also added the hazelcast prefix to the former Jet property names, e.g., jet.job.scan.period has become hazelcast.jet.job.scan.period and the former one is deprecated. #19146

Added a configuration option to enable/disable resource uploading for Jet engine jobs. See here for details.

Even when the factory configuration is missing on the member but the map is configured to have the the in-memory format as OBJECT, Hazelcast now can store portables as PortableGenericRecord and still query them without needing to convert them to Object/Data. #18891

Introduced the following properties:

Introduced a warning when the users create a job JAR to submit via hz-cli and Hazelcast detects hazelcast or hazelcast-enterprise is packaged in it. #19512

Added queueFillPercent metric to show how full the WAN replication queue is, in percentage. #19431

The README of hazelcast/hazelcast GitHub repository has been completely rewritten to reflect the unification of former Hazelcast IMDG and Jet products. #19061

The hazelcast-sql module is now covered by the Hazelcast Community License; before, it was Apache License, Version 2. #18957

Added the merkle tree support for caches to speed up the migration process during a cluster rebalancing. #18898

Added the client console entry point to the Hazelcast command line interface; you can now use the hazelcast console command to start the client console application. #18857

Enhanced the getPartitionGroupStrategy() method to have cluster members as arguments so that useful partitioning strategies can be implemented by accessing the members using this method. #18794

The log message for infinite cluster connection timeout is clearer now. Previously, it was represented as the value of Long.MAX_VALUE. #18642

Introduced a new mechanism in the background expiration tasks; now a thread local array controls the allocations for these tasks otherwise which may cause increased garbage collection pressure and CPU usage spikes when you use aggressive expiration configurations, e.g., low time-to-live values. #18633

The license key is, now, not shown while starting a member on Docker with overriding configurations. #18568

Limited the number of parallel partition reads (to a fixed value of five) for maps and caches to prevent out of memory failures. #18499

Added a comprehensive documentation for metrics produced by Hazelcast. See List of Metrics for the full list of metrics with their descriptions. #17880

Improved the speed of connection by a member when it joins the cluster, by removing the unnecessary sleep statements in the code. #17428