Deploy in development

To get up and running quickly, you can bring up a fully working Flow environment locally using Docker Compose to start testing or developing your integrations.

Development setup

This guide will walk you through setting up Flow in a development environment. This setup pre-configures Management Center in Dev Mode, without needing any security credentials for logging in or using the REST API. Dev Mode should not be used in production. For more information about Dev Mode, see https://docs.hazelcast.com/management-center/latest/deploy-manage/dev-mode

Prerequisites

Before you get started, you’ll need the following:

CLC version 5.5.0 or later is recommended for use with the starter template.

Create your project

The easiest way to get started with Flow is by using Hazelcast’s command-line tool clc. This gives you a:

  • Workspace for developing your queries

  • docker-compose.yml file to be able to run locally

Set up your Flow workspace with:

clc project create -t hazelcast-flow-starter

You have now created a copy of our example Flow project containing the workspace and Docker Compose file.

You can navigate to the newly created project directory:

cd hazelcast-flow-starter

The next step is to setup your Hazelcast License Key. Copy the template file as shown below and add your license key to the .env file.

cp .env.example .env

The .env file contains the following variables:

  • MC_LICENSE contains the Hazelcast license

  • MC_APPLICATION_CLIENT_SECRET is used as client credentials for authenticated API access

Flow will not start if the MC_LICENSE environment variable is not set and passed to it. See license management for Flow.

Take a look at Obtain an Access Token for details on setting this secret and using client credentials to authenticate programmatically.

Start Flow

Next you can simply start your local Flow using:

docker compose up

The following containers will now be running:

Name Purpose Interface

Flow

Runs the main Flow server

http://localhost:9021

Postgres

Data store for your projects

psql 25432

Management Center

An interface into the workings of Hazelcast

https://localhost:8080

After the application starts and everything is set up, navigate to: http://localhost:9021 and start using Flow.

Live reload

The developer environment of Flow is configured with live reload, so any changes you make in your Taxi project are automatically picked up.

Advanced setup

Deploy without CLC

A sample docker-compose.yml file is maintained at:

https://github.com/hazelcast/hazelcast-flow-docker-compose

With examples for:

  • Running with an external Hazelcast cluster

  • Using external IDP config

  • Configuring to include metrics

File Watcher Settings

Flow watches the project directories in the file system to automatically load the changes. It uses the inotify mechanism on Linux. Linux systems have a limited number of watches available by default. Sometimes this limit is too low, so you may have to set it to a higher value.

To check the current watch limit, run the following:

sysctl fs.inotify.max_user_watches

The ideal number of watches depends on the number of Flow projects you have, and the number of other apps that add file watches. For many use cases, 250 000 would be a sensible choice.

You can set the number of file watches limit temporarily using the following commands:

sudo sysctl fs.inotify.max_user_watches=250000
sudo sysctl -p

To set the limit permanently, you can run the following:

echo fs.inotify.max_user_watches=250000 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Continue reading

Continue learning about Flow by setting up your workspace or investigating how to deploy in production.

lifebuoy

Help and support