Hazelcast Jet

Hazelcast Jet is a distributed batch and stream processing framework based on Hazelcast IMDG. It allows you to write, currently, modern Java code that focuses purely on data transformation while it does all the heavy lifting of getting the data flowing and computation running across a cluster of members. It supports working with both bounded (batch) and unbounded (streaming) data.

You can follow the Getting Started Guide in the Hazelcast Jet documentation to see a simple example.

Jet supports a rich set of data transformations such as windowed aggregations. For example, if your data is GPS location reports from millions of users, Jet can compute every user’s velocity vector by using a sliding window and just a few lines of code. Jet also supports at-least-once and exactly-once processing.

Jet can be used to import/export data from/to Hazelcast IMDG using a very wide variety of data sources including Hadoop, S3, Apache Kafka, Elasticsearch, JDBC and JMS. For example, you can read data from Kafka and write to IMap with just a few lines of code. You can stream changes from an IMap and write it to an external system or you can join to a stream reference data that is already stored in IMap.

For a full list of external systems that Jet integrates with, see the Sources and Sinks section of Jet’s documentation.