Work with XML and JSON
This tutorial shows how to link services that publish a mixture of XML and JSON. It explains how to combine these services and then expose them as a REST API.
How to begin a Hazelcast Flow project is described from scratch so, if you’re unfamiliar with Flow, this is the perfect starting point.
Typically, you’ll use Flow to stitch together services in different locations from across your organization but, to keep it simple, these are deployed below as a single service.
Overview
This tutorial shows how to connect the following three services:
// A service that returns a list of Films (in XML)
operation findAllFilms():FilmList
// A service that returns the cast of a film (in XML)
operation findCast(FilmId):ActorList
// A service that returns the list of awards a film has won (in JSON)
operation findAwards(FilmId):Award[]
You’ll combine these together to create a single endpoint that exposes a REST API to return the data from all three services.
Prerequisites
You should have Docker and Docker Compose.
Step 1 - Start a local instance of Flow
In this tutorial, we’re going to deploy Flow, as well as a demo project that we’ll use throughout this tutorial.
Everything you need is packaged up in a Docker Compose file to make getting started easier.
To launch, clone the repository and follow the instructions in the README file:
git clone https://github.com/hazelcast/hazelcast-flow-public-demos.git
cd hazelcast-flow-public-demos/xml-and-jsonAfter about a minute, Flow should be available at http://localhost:9021.
To confirm everything is ready to go, head to the Projects Explorer to make sure that some schemas are registered. It should look like the screenshot below. If you see a message saying there’s nothing registered, wait a few moments longer.
You now have Flow running locally, along with an example REST service which we’ll use in our next steps.
If you run docker ps, you should see a collection of Docker containers now running.
| Container Name | Part of Flow stack or Demo? | Description |
|---|---|---|
flow |
Flow |
The core Flow server, which provides our UI, and runs all our integration for us |
postgres |
Flow |
A Postgres DB, used by flow to store its metadata |
management-center |
Flow |
The Management Center, for monitoring and managing Flow |
films-xml |
Demo |
A REST API, which exposes information about films in XML and JSON format |
With Flow, instead of using integration code, schemas are used to describe data sources which Flow uses to link everything together. Some services publish their own schemas (e.g. XSDs or WSDLs) but the following example shows how to do this without schemas, with everything written in Taxi.
Step 2 - Describe the Film Service
First, write Taxi code that describes the Films Service. This is an HTTP operation that returns a list of films.
| Use Flow’s Designer to create and edit your Taxi code. On the left-hand side, input a data sample or upload a CSV, TSV, PSV, JSON or XML source file. On the right-hand side, create a Taxi schema based on this data. The parsed results appear below as you work. When you are happy with your Taxi model, you can copy it from the Designer into your project. |
In the Taxi model, you’ll notice that the name of the field is composed of special types that will be used to share data across services and connect them together. This means you are not coupled to field names, which are free to change without impacting the model.
The Taxi code that describes the FilmsService is shown in the films.taxi tab below, and an example of the XML source for the list of films is shown in the films-data.xml tab.
To add the taxi definitions to your project, copy the code into a file called called films.taxi in the directory taxi/src in your project.
import flow.formats.Xml
// The @Xml annotation tells Flow how to read this object
@Xml
model FilmList {
item: Film[]
}
model Film {
id: FilmId inherits Int
title: FilmTitle inherits String
yearReleased: YearReleased inherits Int
}
service FilmsService {
@HttpOperation(url = "http://films-xml:80/films", method = "GET")
operation getAllFilms(): FilmList
}<List>
<item id="0">
<title>ACADEMY DINOSAUR</title>
<yearReleased>2005</yearReleased>
</item>
<item id="1">
<title>ACE GOLDFINGER</title>
<yearReleased>1975</yearReleased>
</item>
</List>| Once you have exposed this service, you will be able to see it in the Services diagram in Flow’s Catalog to view how everything links together. |
Step 3 - Integrate the Cast Service
Repeat this process to define a Taxi schema for the film cast.
The CastService takes the FilmId you created previously and uses this to return a list of actors.
The Taxi code that describes the CastService is shown in the actors.taxi tab below, and an example of the XML source for the cast list is shown in the actor-data.xml tab.
Again, create a file called actors.taxi in the directory taxi/src in your project.
import flow.formats.Xml
@Xml
model ActorList {
item : Actor[]
}
model Actor {
id : ActorId inherits Int
name : ActorName inherits String
}
service CastService {
@HttpOperation(url = "http://films-xml:80/film/{filmId}/cast", method = "GET")
operation fetchCastForFilm(@PathVariable("filmId") filmId : FilmId):ActorList
}<List>
<item>
<id>34</id>
<name>JUDY DEAN</name>
</item>
<item>
<id>21</id>
<name>ELVIS MARX</name>
</item>
</List>What connects it together
Flow can now link the services together - there’s no need to write any integration code or resolvers as there is enough information contained in the schemas.
| Use the Services diagram in Flow’s Catalog to view how everything links together. |
// The FilmId from the Film model...
model Film {
id : FilmId inherits Int
...
}
// ... is used as an input to the fetchCastForFilm operation:
operation fetchCastForFilm(FilmId):ActorList
More Taxi has been written here than normal because you’re not working with the service’s XSD directly (e.g., it wasn’t available, or it didn’t exist). If the services published XSDs or WSDLs, you could have leveraged those and only declared the Taxi scalars, such as FilmId.
|
Step 4 - Write Data Queries
Next, using Flow’s Query editor, write a query using TaxiQL.
Fetch the list of films
// Just fetch the ActorList
find { FilmList }Which returns:
{
"item": [
{
"id": 0,
"title": "ACADEMY DINOSAUR",
"yearReleased": 2005
},
{
"id": 1,
"title": "ACE GOLDFINGER",
"yearReleased": 1975
},
// snip
]
}Restructure the result
To remove the item wrapper (which is carried over from the XML format), change the query to just ask for a Film[]:
find { FilmList } as Film[]Which returns:
[
{
"id": 0,
"title": "ACADEMY DINOSAUR",
"yearReleased": 2005
},
{
"id": 1,
"title": "ACE GOLDFINGER",
"yearReleased": 1975
}
]Define a custom response object
You can define a data contract of the exact data you want back, specifying the field names you like, with the data type indicating where the data is sourced from. This means you are not bound to the source system’s descriptions.
find { FilmList } as (Film[]) -> {
filmId : FilmId
nameOfFilm : FilmTitle
} []Link the Actor Service
To include data from the CastService, just ask for the actor information:
find { FilmList } as (Film[]) -> {
filmId : FilmId
nameOfFilm : FilmTitle
cast : Actor[]
} []Which now gives us:
{
"filmId": 0,
"nameOfFilm": "ACADEMY DINOSAUR",
"cast": [
{
"id": 18,
"name": "BOB FAWCETT"
},
{
"id": 28,
"name": "ALEC WAYNE"
},
//..snip
]
}Step 5 - Add the Awards Service
You can also define a schema and service for the awards information, which is returned in JSON.
The Taxi code that describes the AwardsService is shown in the awards.taxi tab below, and an example of the awards returned is shown in the awards-data.json tab.
You’ll need to add this to your project by creating a file called awards.taxi in the directory taxi/src.
model Award {
title: AwardTitle inherits String
yearWon: YearWon inherits Int
}
service AwardsService {
@HttpOperation(url = "http://films-xml:80/film/{filmId}/awards", method = "GET")
operation fetchAwardsForFilm(@PathVariable("filmId") filmId: FilmId): Award[]
}[
{
"title": "Best Makeup and Hairstyling",
"yearWon": 2020
},
{
"title": "Best Original Score",
"yearWon": 2020
},
// snip\...
]Enrich your query
Finally, to include this awards data, you just add it to the query:
find { FilmList } as (Film[]) -> {
filmId: FilmId
nameOfFilm: FilmTitle
cast: Actor[]
awards: Award[]
} []Which gives us:
{
"filmId": 0,
"nameOfFilm": "ACADEMY DINOSAUR",
"cast" : [] // omitted
"awards": [
{
"title": "Best Documentary Feature",
"yearWon": 2020
},
{
"title": "Best Supporting Actress",
"yearWon": 2020
}
]
}Step 6 - Publish your query
The following shows how to publish a query as a REST API, and as an endpoint using the UI.
Publish a query as a REST API
Now that you have response data you’re happy with, you can publish this query as a REST API.
-
Wrap the query in a
query { ... }block, and save it asquery_filmsAndAwards.taxiin the directorytaxi/src -
Add an
@HttpOperation(...)annotation
@HttpOperation(url = '/api/q/filmsAndAwards', method = 'GET')
query filmsAndAwards {
find { FilmList } as (Film[]) -> {
filmId : FilmId
nameOfFilm : FilmTitle
awards : Award[]
cast : Actor[]
} []
}Your query is now available at http://localhost:9021/api/q/filmsAndAwards
$ curl http://localhost:9021/api/q/filmsAndAwards | jqWhich gives:
[
{
"filmId": 0,
"nameOfFilm": "ACADEMY DINOSAUR",
"awards": [
{
"title": "Best Animated Feature",
"yearWon": 2020
},
{
"title": "Best Original Score for a Comedy",
"yearWon": 2020
},
{
"title": "Best Documentary Feature",
"yearWon": 2020
},
// .... snip
]
}
]Publish a query using the UI
To publish a query as an endpoint using the UI:
-
Choose Query editor and in the editor, write your query
-
Click Run to make sure the query runs with no errors
-
Click the Save query to project button, choose a project (this must be editable), give your query a name and then save it
-
Click the Publish endpoint button and publish it as an HTTP or WebSocket endpoint, depending on the query
-
Choose Endpoints and make sure the query is running (you can disable/enable the endpoint if necessary)
