Jet jobs (publish streaming endpoints)

Flow Jet jobs

Flow uses Jet jobs when you create and enable streaming endpoints. These jobs are long-lived and run continuously in the background for each endpoint. The Jet job is stopped when you disable the streaming endpoint.

Publish queries describes how to create and publish queries as HTTP or WebSocket endpoints.

After creating a query, it is visible in the list of endpoints in the Flow UI under "Endpoints".

stream endpoints

View Jet Jobs in Management Center

To view the Jet jobs in Management Center, navigate to the Jet Jobs tab. Here you can see the status of each job, such as running, completed, or failed.

jet jobs

Job Graph

The pipeline for a streaming endpoint consists of a series of stages that are executed in order. The Job Graph tab in Management Center shows the stages of the pipeline and the data flow between them.

Example Job Graph

jet pipeline

As shown in the example Job Graph, the pipeline consists of the following stages:

  • Ingestion: The data is ingested from the source

  • Validation: The data is validated mandatory fields

  • Map: Transform the input type to the output type

  • Sink: The data is written to a Hazelcast Topic as sink