Connector Guides
Hazelcast comes with a wide range of built-in connectors for accessing data in other platforms. You can also use Hazelcast with pre-built connectors for importing data from external systems. Both types of connector are configured to read and write data in the most efficient way for their respective platform.
Built-in connectors are available as either sources or sinks. Sources allow Hazelcast to read data from the required platform either for queries or as part of a larger data pipeline. Sinks allow Hazelcast to write data, such as query results, to a target system.
You can also use pre-built Kafka Connect Source connectors with Hazelcast. These low-code connectors allow you to stream events from non-Kafka external systems into a data pipeline ready for stream processing. Kafka Connect Source connectors are available for many popular platforms, including RabbitMQ, Neo4j, Couchbase, Scylla, SAP and Redis.
Explore the available connectors and choose one for your use case.
Processing Guarantees
To ensure that all data is processed correctly, connectors that support streaming have either an at-least-once or exactly-once processing guarantee.
Connectors that do not support streaming do not have any processing guarantees. Instead, during intermittent failures, you must restart any queries or jobs that use these connectors.
Connectors for SQL
You can use the following connectors in SQL by configuring them in the CREATE MAPPING statement.
Sources
| Connector | Module | Batch/Streaming | Processing guarantee |
|---|---|---|---|
hazelcast-jet-kafka |
streaming |
exactly-once |
|
Depends on the file format and/or storage system. |
Depends on the storage system. |
Depends on the storage system. |
|
hazelcast-sql |
batch |
N/A |
|
hazelcast-sql |
batch |
N/A |
|
hazelcast-sql and hazelcast-jet-mongodb |
batch, streaming |
exactly-once |
Kafka Connect Connectors
Source
Integrate the Kafka Connect Source connector for your external system with Hazelcast by calling the KafkaConnectSources.connect() method in a pipeline.
| Connector | Module | Batch/Streaming | Processing Guarantee |
|---|---|---|---|
|
streaming |
at-least-once |
Connectors for the Jet API
The Jet API supports more connectors than SQL.
Sources
| Connector | Module | Batch/Streaming | Processing guarantee |
|---|---|---|---|
|
batch |
N/A |
|
|
streaming |
at-least-once |
|
|
batch |
N/A |
|
|
batch |
N/A |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
batch |
exactly-once |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
streaming |
N/A |
|
|
streaming |
exactly-once |
|
|
batch |
N/A |
|
|
batch |
N/A |
|
|
streaming |
exactly-once |
|
|
batch |
N/A |
|
|
streaming |
none |
|
|
batch |
N/A |
|
|
streaming |
none |
|
|
batch |
N/A |
|
|
streaming |
exactly-once |
|
|
batch |
N/A |
|
|
batch |
N/A |
|
|
streaming |
exactly-once |
|
|
streaming |
none |
|
|
batch |
N/A |
|
|
streaming |
none |
Sinks
| Connector | Module | Batch/Streaming | Processing Guarantee |
|---|---|---|---|
|
batch |
N/A |
|
|
streaming |
at-least-once |
|
|
streaming |
at-least-once |
|
|
batch |
N/A |
|
|
streaming |
exactly-once |
|
|
streaming |
at-least-once |
|
|
batch, streaming |
exactly-once |
|
|
streaming |
at-least-once |
|
|
batch |
N/A |
|
|
streaming |
at-least-once |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
streaming |
exactly-once |
|
|
batch |
N/A |
|
|
streaming |
at-least-once |
|
|
streaming |
at-least-once |
|
|
streaming |
at-least-once |
|
|
streaming |
at-least-once |
