To ensure the reliability of streaming jobs and their workloads, try implementing some basic monitoring. With Hazelcast, you can monitor the status of streaming jobs to quickly identify job failures for troubleshooting.
Monitoring for Status Changes
There are two ways to monitor for job status changes. You can either poll the
Job.getStatus() method separately for each job or add a
JobStatusListener to your jobs. In both cases, you can write custom code for execution on status change. For example, triggering an alert or writing the changed status to a dashboard.
Using a JobStatusListener
JobStatusListener has several advantages over polling for the job status.
The listener is notified immediately when a status change occurs. To achieve a similar result with polling, you might increase the polling frequency, which would decrease performance.
You can identify jobs in an error state. By default, all streaming jobs are suspended on failure. When a job moves to the suspended status, the listener is passed a suspension cause; either an error description or requested by a user.
|The reasons for job suspensions and failures are also displayed on the Jobs page in Management Center.|
Even though you can attach a
Adding a JobStatusListener
To register the listener for a job, add the following to your client code:
statusListenerId = myJob.addStatusListener(JobStatusListener)
The following example creates a new job, registers the
JobStatusListener, and uses a lambda expression to print out status changes to the stdout console, including whether they were requested by a user.
JetService jet1 = hz1.getJet(); JobConfig jobConfig = new JobConfig(); String jobName = "sampleJob"; jobConfig.setName(jobName); Job job = jet1.newJob(pipeline, jobConfig); job.addStatusListener(event -> System.out.printf("Job status changed: %s -> %s. User requested? %b%n", event.getPreviousStatus(), event.getNewStatus(), event.isUserRequested()) );