MongoDBReader formatting issues

Support/help with CloverETL implementation problems

sshrestha
Posts: 15
Joined: Tue Jul 11, 2017 6:50 pm

MongoDBReader formatting issues

Postby sshrestha » Fri Mar 30, 2018 6:25 pm

Hi,

I am trying to read some data from MongoDB. An example data is below:

Code: Select all

{
   _id: "XXXXXXXXXXXX",
   name: "Foo",
   description: "bar",
   metadata: {
      info: "Hello World"
   }
}


If I read a document as shown above from MongoDB the document is returned as JSON Object which is a map field.
But the metadata field is returned as string.

From the above example, is there a way to get the metadata field as a map field as well? If not, how do I convert the string field into map field?

Sincerely,
Shubha

bartonv
Posts: 79
Joined: Wed May 03, 2017 12:10 pm

Re: MongoDBReader formatting issues

Postby bartonv » Wed Apr 11, 2018 8:02 am

Hi Shubha,
unfortunately, extracting nested elements as json objects (map[string,string]) is currently not supported in CloverETL. We do plan to incorporate such feature in the future but due to its complexity and resource demandingness, it has not been developed yet.
However, there are a couple of workaround options of how to deal with this issue. Attached is a project where I demonstrate 2 useful approaches to nested elements:
  1. If you need to extract the ‘metadata’ nested element as json, you would have to deconstruct the original json into individual fields and then reconstruct the ‘metadata’ part again. In my example, I am using JSONExtract to parse the original json into individual fields and the JSONWriter component to build up the nested element part. Note: due to the way how JSONWriter works, the output json is a simple string field (not a map).
  2. If you do insist on having the output json as a map[string,string] field, review the Reformat component in my example. The source code will show you have to construct a map from the given individual fields.
Kind regards,
Attachments
nestedElements.zip
(5.68 KiB) Downloaded 16 times
---
Vladimir Barton
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support