A newer version of Platform is available.

Mapping to MongoDB

To query MongoDB data connections, you can create a mapping to them with the Mongo connector.

What is the Mongo Connector

The Mongo connector allows you to read from/write to a MongoDB database, and to execute SQL queries on Mongo collections directly from Hazelcast.

Supported SQL Statements

For the batch mode:

For the streaming mode:

SELECT

Installing the Connector

The Mongo Connector artifacts are published on the Maven repositories. Add the following lines to your pom.xml to include it as a dependency to your project:

<dependency>
    <groupId>com.hazelcast.jet</groupId>
    <artifactId>hazelcast-jet-mongodb</artifactId>
    <version>$5.3.7</version>
</dependency>

or if you are using Gradle:

compile group: 'com.hazelcast.jet', name: 'hazelcast-jet-mongodb', version: $5.3.7.

To be able to use SQL over MongoDB, you have to include hazelcast-sql as well as a dependency.

Permissions

Enterprise

The Mongo connector does not yet support permissions.

Before you Begin

Before you can create a mapping to a MongoDB, you must have the following:

A $jsonSchema validation in the collection (see the schema documentation), or you have to have at least one element in the collection you want to create mapping for (for property type validation).
Enabled operations log (oplog) if you want to use streaming mappings.

Creating a MongoDB Mapping

The following example creates a mapping to a MongoDB database.

In a MongoDB database, create a people collection. For example in Java, you would run the following command.

CreateCollectionOptions options = new CreateCollectionOptions();
ValidationOptions validationOptions = new ValidationOptions();
validationOptions.validator(BsonDocument.parse(
       "{\n" +
               "    $jsonSchema: {\n" +
               "      bsonType: \"object\",\n" +
               "      title: \"Object Validation\",\n" +
               "      properties: {" +
               "        \"personId\": { \"bsonType\": \"int\" },\n" +
               "        \"name\": { \"bsonType\": \"string\" }\n" +
               "      }\n" +
               "    }\n" +
               "  }\n"
));
options.validationOptions(validationOptions);
database.createCollection(collectionName, options);

The ValidationOptions are not required, but recommended.

Configure the data connection so that the client can be reused by multiple mappings.

XML
YAML
Java

<hazelcast>
    <data-connection name="myMongo">
        <type>Mongo</type>
        <properties>
            <property name="connectionString">stringForMongo</property> (1)
        </properties>
        <shared>false</shared> (2)
    </data-connection>
</hazelcast>

data-connection:
  name: myMongo
  type: Mongo
  properties:
    connectionString: stringforMongo (1)
  shared: false (2)

DataConnectionConfig dataConnectionConfig = new DataConnectionConfig()
        .setName("myMongo")
        .setType("Mongo")
        .setProperty("connectionString", connectionStringToMongo) (1)
        .setShared(false); (2)
config.addDataConnectionConfig(dataConnectionConfig);

1 Your connection string.

Set to true if the connection is reusable.

Instead of providing a single connectionString parameter, you may also want to provide host, username, password and (optionally) authDb.

Instead of providing the configuration in YAML or XML, you can also run the following SQL query.

CREATE DATA CONNECTION myMongo type Mongo SHARED
OPTIONS (
	‘connectionString’ = ‘<your connection string>’
)

Create the mapping.

CREATE MAPPING people
TYPE MONGO (1)
DATA CONNECTION myMongo; (2)

1 The type of the connector.

The name of the data connection configuration on your members (see Step 2 above).

In the above case, automatic schema inference will be used. You may also want to provide the schema explicitly as shown below.

CREATE MAPPING people (
    firstName VARCHAR(100),
    lastName VARCHAR(100),
    age INT
)
DATA CONNECTION myMongo

Notice that there is no mention of TYPE MONGO this time; it’s automatically assumed by the SQL engine when you provide MongoDB data connection. This works with both schema provided or not.

Type Mapping

The type system in MongoDB and SQL is not exactly the same. That leads to potential confusions and the need of type coercion.

Table 1. MongoDB Type Conversion
BSON Type	SQL Type	Java Type
`DOUBLE`	`DOUBLE`	`DOUBLE`
`STRING`	`VARCHAR`	`STRING`
`OBJECT`	`OBJECT`	`org.bson.Document`
`ARRAY`	`OBJECT`	`LIST`
`BINDATA`	-	-
`UNDEFINED`	-	-
`OBJECTID`	`OBJECT`	`org.bson.ObjectId`
`BOOL`	`BOOLEAN`	`BOOLEAN`
`DATE` This represents seconds from Unix epoch in UTC timezone. Therefore, it’s not mapped to pure `DATE` SQL type nor `LOCALDATE` in Java (nor any formats with timezones).	`DATE_TIME` or `TIMESTAMP`	`LOCALDATETIME`
`TIMESTAMP`	`DATE_TIME` or `TIMESTAMP`	`LOCALDATETIME`
`NULL`	-	-
`REGEX`	`OBJECT`	`org.bson.BsonRegularExpression`
`DBPOINTER`	-	-
`JAVASCRIPT`	`VARCHAR`	`STRING`
`JAVASCRIPTWITHSCOPE`	`OBJECT`	`org.bson.CodeWithScope`
`SYMBOL`	-	-
`INT (32 BIT)`	`INT`	`INT`
`LONG (64 BIT)`	`BIGINT`	`LONG`
`DECIMAL (128 BIT)`	`DECIMAL`	`BIGDECIMAL`
`MINKEY`	`OBJECT`	`org.bson.MinKey`
`MAXKEY`	`OBJECT`	`org.bson.MaxKey`

The Java Type column represents an object returned by the SQL query if the object put into the collection is of given BSON type.

Note that, while Hazelcast is able to convert MongoDB type to the requested SQL type in the projection, the argument binding will not always work the same due to technical limitations. For example, you can have an object with the type TIMESTAMP represented as DATE_TIME, that after execution of SELECT it will give you LocalDateTime in Java client. However, binding LocalDateTime as an argument will not work, as only native MongoDB types will work for arguments. Same applies to, for example, having BSON column of type STRING mapped to INTEGER in SQL.

Type Coercion

The following table shows the possible and supported type coercions. All the default mappings from the previous section are always valid.

Table 2. MongoDB Type Conversion
Type of Provided Argument	Resolved Insertion Type
`LOCALDATETIME`	`BSONDATETIME`
`OFFSETDATETIME`	`BSONDATETIME`
`HazelcastJsonValue (JSON column)`	`DOCUMENT`