5.3.0 Release Notes
The minimal supported Java version is 11 starting with this release of Hazelcast Platform. |
New Features
-
Connector for Kafka Connect source (BETA): Import data from an external system directly into a Hazelcast data pipeline without the need of a Kafka cluster.
-
Connector for MongoDB (BETA): Read and write from/to MongoDB via this connector and execute SQL queries on Mongo collections directly from Hazelcast.
-
Partition-Aware SQL Client: Send the SQL commands only to the members having the relevant data, which reduces the network hops and improves the query performances.
-
Data Connection Support via SQL: Data connections and mappings using them can be managed using SQL commands.
Breaking Changes
-
Renamed the
DataLinkFactory
interface asDataConnection
. #24224 -
Removed the
TO_ROW
function as it is obsolete, you can useCAST (udtObj AS JSON)
instead. #23808 -
SQL mappings for internal maps (
sql.catalog
andjet.*
) cannot be created anymore. #24282 -
Changed the default cloud coordinator URL from
coordinator.hazelcast.cloud
toapi.viridian.hazelcast.com
. The default configuration now connects to Hazelcast Viridian Cloud instead of Hazelcast Cloud. If you want to continue accessing your Hazelcast Cloud clusters, you need to set thehazelcast.client.cloud.url
property tohttps://coordinator.hazelcast.cloud
in your configuration. #23290
Enhancements
Cloud
-
Hazelcast was sending requests to Kubernetes API when deploying an application with embedded Hazelcast and
service-dns
(DNS lookup mode) specified to a Kubernetes cluster. This was causing the requests to be unsuccessful and the application not to start. This mechanism has been improved by creating Kubernetes client only for the DNS lookup mode. #23883 -
When advanced networking is enabled, the Kubernetes discovery plugin might have been discovering several endpoints (per each port) for each member’s pod. The discovery plugin now matches only the private IP per endpoint, ignoring the port values. #23766
-
Added support of IMDSv2 for Hazelcast’s AWS Discovery plugin. #23545
-
Added the support of discovering ECS and EC2 members on AWS. See Deploying a Cluster on Amazon AWS. #22411
-
Enterprise Enhanced the experience of using Hazelcast with persistence under Kubernetes by removing requirements for manual interventions. Previously, when there is a restart on such a cluster, the cluster states were needed to be changed manually. For some cases, this might have led to data loss. Now, the shutdown process is automated. See Configuring Kubernetes for details. #21844
Distribution
-
Hazelcast no longer depends on JRuby; the JRuby
ScriptFactory
dependency must be explicitly added to the application. #23355 -
Added the Kafka Connect extension to the distribution. #23312
-
Shaded dependencies for Hazelcast Platform have been combined in a dedicated package (
com.hazelcast.shaded
). #23124
Networking
-
Added socket options for per-socket keep-alive configuration:
keep-count
,keep-idle-seconds
, andkeep-interval-seconds
. You can set these options using either the advanced network configurations or Hazelcast system properties. See Configuring TCP Keep-Alive. #23816, #24050
SQL Engine
-
The
CREATE MAPPING
statement for Compact Serialization now supports nullable primitives. #24300 -
Added support of fault tolerance and processing guarantees for stream-to-stream JOINs. #22979, #23462
-
Implemented the
GET_DDL
function. #23595 -
Implemented the
SHOW DATA CONNECTIONS
andSHOW RESOURCES
statements to show the connections and resources created by theCREATE CONNECTIONS
statement. #24028, #23844 -
Added support of providing declarative partitioning strategies based on attributes. These attributes represent specific partition keys for a map. You can also configure this strategy dynamically. #23535, #23954
-
Added
attributes
anduser_defined_types
tables toinformation_schema
. #23267 -
Made all streaming SQL jobs to be suspending on failure. #23178
-
Implemented the
TO_CHAR
function which is capable of formatting dates, timestamps, and numbers. #22529 -
Implemented
JSON=VARCHAR
comparison. #22746 -
Enterprise Added support of configuring SQL Catalog persistence. #24383
Jet Engine
-
Added support of modifying the job configuration of suspended jobs so that jobs can run without overflow failures by adding more resources. #23862
-
Each Jet engine job was creating a client for connecting to the remote cluster, resulting in an excessive number of clients. This has been fixed by introducing a single data connection which can be reused for all job instances. #23472, #23849
-
Added MongoDB read (source) and write (sink) processors. #23348
-
Introduced
JobStatusListener
as an alternative to repeatedly retrieve a job status via theJob.getStatus()
method. #23193 -
Updated the job API to add the ability to distinguish the user-cancelled jobs from the failed ones. #22924
-
Added
flock
to guard all the concurrentpip
executions (upgradingpip
andprotobuf
versions) in the Jet-to-Python script. #22829
Security
-
Enterprise Added the TLS configuration property
keyMaterialDuration
which allows using a configurable timeout for caching keyStores and trustStores. See TLS Configuration for the details on how to configure this property. #23307
Serialization
-
Hazelcast now allows serializing unsupported types via zero-configuration provided that there is an explicit serializer implemented for the unsupported type. #23449
-
Setting a generic record now fails if the value is not of the same type of generic record. #23522
-
Introduced clearer exception messages for Generic MapStore, instead of
UndefinedErrorCodeException
. #22765
WAN Replication
-
Added the REST endpoint
/hazelcast/rest/wan/sync/progress
which allows tracking the WAN synchronization progress. #22958
Connectors
-
Added support of predicates in DML (Data Manipulation Language) in JDBC SQL connector. #24521
-
Added the ability to provide host, username, password, and authDB when connecting to a MongoDB, as an alternative to specifying a connection string. #24238
-
Added the ability to create a mapping with a MongoDB data connections that provides unbounded (streaming) results. #24356
-
Added compound identifier for the
externalName()
method. #23772 -
Added an option for the MongoDB source to perform reading with a single processor instance. Usually distributing the operations is a best practice; however, the
$function
aggregation method is not supported by the Atlas Serverless instances. In this case, this newly introduced option can be used to to query such instances in one processor. See the option’s documentation. #24198 -
Added additional permission checks to JDBC connectors (source and sink). The previous checks in some cases used only the
jdbc:
name inConnectorPermission
. The newly introduced checks get the JDBC URL from existing connection metadata and check if the permission is granted. If want to grant aConnectorPermission
to all JDBC URLs, you may use wildcards, e.g.,jdbc:*
. #23716 -
Added
DataConnection
for Kafka consumers and producers, and for MongoDB #23886, #23837 -
Jet engine’s connector for MongoDB is now a part of the Hazelcast Platform distribution. #22565
-
Introduced a way to control the initial partitions offsets when consuming records from Kafka via the Jet engine: by passing an additional topics configuration object that contains definitions of initial offsets for specific partitions. #21546
Metrics
-
Added name of the job to the job specific metrics. #23744
-
Added Tiered Storage compactor metrics to be exposed by the tools and interfaces such as Management Center, JMX, and diagnostics. #23643
-
Added a new tag to the CP Subsystem metrics to display the client names for CP sessions on Hazelcast Management Center. #23387
-
Added entry eviction and expiration counts to local map statistics and metrics. #23318
Other Enhancements
-
External Data Store references have been renamed as _Data Connections. This includes the renaming of the
external-data-store
configuration element asdata-connection
. #24224 -
Added the ability to test whether the connection with a JDBC data link is valid (
testConnection
method forJdbcDataConnection
). #23405 -
Upgraded Apache Calcite to 1.32.0. #22849
-
Added confirmation that Hazelcast Platform has been tested on x86_64 and AArch64/ARM64 system architectures from Platform 5.3.
Fixes
-
Enterprise Fixed an issue where a member with persistence can be restarted even though the persistence file structure is not valid (e.g., the persistence directory does not contain the required files). #5861
-
Enterprise Fixed an issue where multiple Hazelcast members were able to use the same base directory for Tiered Storage, causing crashes. #5770
-
Enterprise Fixed an issue where the members were crashing while they are loading data into the NATIVE memory and using JSON or Kryo serialization. #5384
-
Enterprise Fixed an issue where Jet job snapshots could be prematurely deleted after a restart of a cluster, having lossless restart enabled. #24576
-
Fixed an issue where
map.clear()
was timing out when the map is configured with a generic map store. #24534 -
Fixed an issue where the
SELECT COUNT(DISTINCT COLUMN)
query for maps was producing incorrect results. #24464 -
Fixed an issue where there was a memory leak when Jet engine metrics are enabled. #23492
-
Fixed an issue where the
DELETE FROM
orUPDATE
functions, when used with fully-qualified names, was failing in a JDBC table. #23476 -
Fixed an issue where SQL statements were failing when a class (to determine the fields of a key/value pair) no longer exists but the mapping is still valid. #24003
-
Fixed an issue where
lastAccessTime
andexpirationTime
was not updated when an entry is accessed viaexecuteOnEntries
. #23926 -
Fixed an issue where the Hazelcast configuration was modified by the Jet engine, if the wildcard configuration is used and Jet is enabled; meaning the configurations made by the user was not being reflected. #23848
-
Fixed an issue where Hazelcast was failing to connect to a database when using Debezium, and the key of a record is null. #23768
-
Fixed an issue where the
SELECT
query was returning duplicated rows for the mappings in AWS S3 bucket. #23723 -
Enabled smart client connectivity to Hazelcast clusters deployed on Kubernetes, when advanced network configuration is enabled. #23707
-
Fixed an issue where Debezium connector for MongoDB could not keep the connection after some time. #23689
-
Enhanced the handling of errors coming from the Kubernetes API:
-
Not detecting the intent correctly whenever the cluster is shutdown, so the cluster was not restarted successfully.
-
Extensive amount of Kubernetes API calls were being logged when Automatic State Management feature is enabled.
-
-
Fixed an issue where Hazelcast transaction manager was timing out before the timeout value set for the transaction commit. #23471
-
Fixed an issue where the unordered
mapUsingServiceAsync
was losing items when there are no watermarks. #23271 -
Fixed an issue where the state of a job was being corrupted if a member is forcefully shut down and the job is restarted from snapshot. #23201
-
Fixed an issue where disabling metrics and JMX on a Hazelcast client was failing the members to start. #23360
-
Fixed an issue where a JDBC related exception message was mentioning data source reference instead of the table name. #23122
-
Fixed an issue where the beginning time of a job was being recorded after the job is submitted, which was causing a race condition. #23065
-
Fixed an issue where the
hz-cli
shell script was failing to run in bash on Windows systems since it uses a colon-separatedCLASSPATH
where each path is a Linux-style path. However, Java on Windows expects a semicolon-separatedCLASSPATH
where each path is a Windows-style path. As a consequence, thehz-cli
could not be used from bash on Windows. #22884 -
Fixed a memory leak due to incomplete clean-up of backup replica sync operations. #22769
-
Fixed an issue where a map was loading old values with map.set() when map store offloading is enabled. #22602
-
Fixed an issue where a CP Subsystem object could be destroyed without a permission even the object has security and permission configurations enabled. #22475
-
Fixed an issue where external data store configurations could not be added dynamically. #22450
-
Fixed an issue where replication over WAN was failing on the source cluster members, when there are multiple batch publishers configured in a single WAN replication. #22437
-
Fixed an issue where creating a JDBC SQL mapping without specifying the columns was causing the column order to be reversed compared to that in the remote source table (affecting statements such as
SELECT * FROM …
andINSERT INTO <mapping> VALUES
. #22410 -
Fixed an issue where the map entries recovered from persistence were not expiring after their time-to-live durations. #22279
-
Fixed an issue where Hazelcast members in a Spring Boot application could not establish a connection to each other when deployed in Kubernetes cluster with Istio Envoy Proxy enabled. #22256
-
Eliminated the usage of
ForkJoinPool#commonPool
for internal Hazelcast code, improving stability and avoiding potential deadlocks. #18190
Upcoming Deprecations
-
Hazelcast will end support for Elasticsearch 6 in the upcoming release (5.4). This will affect you if you are using the version 6 module of the Elasticsearch connector.
-
Hazelcast will deprecate transactions in the upcoming release (5.4). An improved version of this feature is under consideration. If you are already using transactions, get in touch and share your use case. Your feedback will help us to develop a solution that meets your needs.