Release Notes: MongoDB Connector Release Notes

MongoDB is a cross-platform document-oriented database. Classified as a NoSQL database, MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), making the integration of data in certain types of applications easier and faster.

Introduction

MongoDB Connector is a closed source connector which provides a connection between Mule and third-party software MongoDB server. It implements the MongoDB v3.0 API modules and it also includes different types of transformers for data management.

The latest MongoDB Connector is an SDK-based connector developed with DevKit 3.9.0.

The Select versions of the connector are MongoDB connector MongoDB v4 and later. The Community versions of the connector are MongoDB connector v3 and earlier.

Prerequisites

This document assumes that you are familiar with MongoDB API. To use the this connector you need the following:

  • MongoDB instance

  • Set of valid credentials, including the required MongoDB endpoints, pointing to your instance

  • MuleSoft Enterprise License

Requirements

See hardware and software requirements.

Dependencies

The MongoDB connector requires the following dependencies:

Application/Service Version

Anypoint Studio

3.7

Mule Runtime

EE 3.6.0 and above

MongoDB

3.0 and above

Java

JDK 7 and above

Installing and Configuring

Installing

To install the MongoDB Connector:

  1. Open Anypoint Studio and got to Help > Install New Software

  2. Select Anypoint Connectors Update Site.

  3. Expand Select and scroll down to locate the MongoDB Connector. Note that MongoDB version 4 connector is in the Select connector category, and the MongoDB version 3.x connector is in the Community connector category.

    Anypoint Studio Install Window

  4. Click Next, Next, and accept the license agreement.

  5. Restart Studio when prompted.

For more information, see Installing Connectors.

After installing the connector, your Studio palette contains the following.

Anypoint Studio palette - MongoDB Connector

Updating from an Older Version

  1. Uninstall the previous MongoDB Connector.

  2. Install the latest version of MongoDB Connector from the update site. Click Help > Install New Software. In the Work with drop-down menu, click Anypoint Connectors Update Site. Expand the Select category and click the checkbox for the Mongo DB Connector version 4.1.x.

  3. Ensure that the Maven dependencies have been updated correctly from Mongo DB[v3.X.X] to Mongo DB[v4.1.0].

  4. Update flows that are using Phased Out or Deprecated operations.

    1. Most operation names have been changed to comply with Mongo Driver v3, so if you had an InsertObject operation, it now appears as InsertDocument.

    2. WriteResult objects have been removed. Now your operations only return Strings, Documents, and DBObjects.

    3. MongoDB transformers have been replaced by implicit ones. So, if your input data is a Map and, you want to perform an Insert operation, just drag a MongoDB connector into the workspace, choose the InsertDocument operation and you’re done! You no longer require the use of the transformer.

    4. Remember that mongo:config does NOT support the port field anymore. Specify the port inside the host field of your mongo:config using the format "host:port". For more information see the MongoDB Connector Migration Guide.

Using This Connector

For a list of all the operations refer to the latest MongoDB Connector API Reference.

Example Application Setup

  • Start by inserting your MongoDB credentials and endpoints in a .properties file and load them using the Property Placeholder component.

  • Search for and drag the HTTP connector to the Anypoint Studio Canvas

  • Search for and drag the Transform Message component to the Canvas.

  • Search for and drag the MongoDB connector to the Canvas.

    MongoDB Connector initial flow

Setting up the Connector Global Element

To set up the global element:

  1. Go to the Global Element tab and click the Create button.

  2. Inside the filter text box type "Mongo", and select MongoDB, from the Connector Configuration directory.

  3. When you select the global element of your choice, you are presented with the global element properties to enter.

    1. The Username field refers to the username you use to access your MongoDB instance.

    2. The Password field contains the corresponding password.

    3. Specify the Database used to send and retrieve data from your MongoDB instance. Before running your application, make sure that the user has the necessary permissions to do so.

    4. You may also need to set the Servers (host:port) property, which is composed of a host:port comma-separated list (in case you have a replica set). This attribute is optional, and if left empty, it defaults to localhost:27017.

    5. If needed, click the SSL checkbox to access the database over SSL. For more information, see Setting up the Global Configuration with SSL (optional).

    6. You may adjust the values for the Connect Timeout, Connections Per Host, Max Wait Time, and the Socket Timeout for requests that are sent to MongoDB. The default value for Connect Timeout is 30000ms. This means that any connection requests that take longer than 30 seconds to be sent and received throw an exception. You can override this behavior by setting their values to 0, meaning that the MongoDB connector waits indefinitely until a request is successfully sent and a response is received.

The following screenshot shows the MongoDB configuration with Spring property placeholders for the Connection field values. For more information, see Configuring Properties. This is the recommended approach when you plan to deploy your application to Runtime Manager or to a Mule server. However, you may hard code your connection credentials in the Global Element Properties shown below if you are in the development stage and simply want to speed up the process.

MongoDB Connector Global Elements

Fields Description

Username

MongoDB username. Input your username using Mule property placeholder syntax.

Password

MongoDB password. Input your password using Mule property placeholder syntax.

Database

MongoDB database. Input your database using Mule property placeholder syntax.

Servers

This location points to the default MongoDB instance, Input your endpoint using Mule property placeholder syntax.

Setting up the Global Configuration with SSL (optional)

As of MongoDB Connector v4.0.0, we support the use of SSL.

To set up the Global Configuration:

  • In the Global Element tab for your MongoDB connection, check the SSL property field.

  • Remember, your instance must support this feature. For more information check MongoDB Configure SSL documentation.

Invoking an Operation

To invoke a simple operation such as the Insert Document operation, follow these steps:

  1. Locate, and drag and drop the HTTP connector, Transform Message, and MongoDB connector onto the Anypoint Studio Canvas.

  2. Configure the MongoDB connector by selecting the Connector Configuration you created in the previous section and choosing the operation to invoke.

    MongoDB Connector

  3. Click Transform Message and enter these two key-value pairs:

    
  
    1
2
3
4
5
6
7

    		
  
    %dw 1.0
%output application/java
---
{
        name:"Peter",
        age:"42"
}

    Transform Message

Example XML Flow

The following example provides a DataWeave transform and the use of the Mongo DB connector.

MongoDB v4 does an automatic conversion for "Document to JSON" and "JSON to Document". 

  
1
2
3
4
5
6
7
8
9
10
11
12
13


  
<flow name="insert-document-flow">
  <http:listener config-ref="HTTP_Listener_Configuration" path="/" doc:name="HTTP"/>
  <dw:transform-message doc:name="Transform Message">
    <dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
{
    name:"Peter",
    age:"42"
}]]></dw:set-payload>
  </dw:transform-message>
  <mongo:insert-document config-ref="Mongo_DB__Configuration" collection="People" doc:name="Insert Document"/>
</flow>

Batch Support

This is feature added for the Insert Documents operation only in Mongo Connector 4.2.0.

The main benefits of supporting batch are:

  • Handling of large quantities of incoming data from an API into a legacy system.

  • Parallel processing of records.

  • Synchronizing data sets between business applications.

Below is a basic example of a Mongo application implementing Batch that takes a Transform Message as input and an Object to Json* transformer as output:


  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


  
<batch:job name="batch_Flow">
        <batch:input>
            <dw:transform-message doc:name="Transform Message">
                <dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[{
  key: "value"
} as :object {
  class : "org.bson.Document"
},
{
  key2: "value2"
} as :object {
  class : "org.bson.Document"
}
]]]></dw:set-payload>
            </dw:transform-message>
        </batch:input>
        <batch:process-records>
            <batch:step name="Batch_Step">
                <batch:commit size="2" doc:name="Batch Commit">
                    <mongo:insert-documents config-ref="Mongo_DB__Configuration" collection="${collection}" ordered="true" doc:name="Batch Bulk Insert"/>
                </batch:commit>
            </batch:step>
        </batch:process-records>
        <batch:on-complete>
            <json:object-to-json-transformer doc:name="Object to JSON"/>
        </batch:on-complete>
    </batch:job>

To avail yourself of this new feature, you can download a demo available here.

For more information, refer to MuleSoft’s Batch Processing documentation.

See Also