Posts: 20
Joined: Mon Feb 23, 2015 8:57 am

Is there a Data Writer equivalent of ComplexDataReader?

Postby dflkid » Wed Apr 12, 2017 2:50 am

I'm importing a heterogeneous flat file using the ComplexDataReader component, but I also need to write out a subset of that data to a flat file that will also be heterogeneous, but I don't see an equivalent Writer component. How do you write a flat file with differing meta data in each row? My particular scenario involves an Order file that contains order header rows with line item rows, each with different metadata.


Posts: 7
Joined: Wed Nov 23, 2016 10:17 am

Re: Is there a Data Writer equivalent of ComplexDataReader?

Postby krizovav » Thu Apr 13, 2017 12:25 pm


You are right, there is no Writer equivalent to ComplexDataReader. If you need a flat file on the output, you can use FlatFileWriter, but it needs the data to be prepared. It all depends on what data format you are using during the transformation in the graph and what data you are expecting on the output.

For example, if you need to write the content of a map or a list field, convert the field into string using Reformat first.

If you are using multiple metadata on multiple edges instead of map or list field, use Reformat for each edge and convert each metadata into one string. Then there are two possibilities how to write this strings into one flat file.
1. Use SimpleGather for gathering these strings and then use one FlatFileWriter.
2. Use one FlatFileWriter for each edge, set the “Append” attribute to true and the right Phase for each Writer. Then you can write into one flat file using multiple FlatFileWriters.

You could also use CustomJavaWriter and set the component behavior by yourself.

I would also like to bring to your attention the “StructuredDataWriter” component. You can define the structure of your output file using this Writer, but it can be used for a maximum of three inputs. For more information about this component, check the documentation, please.

I hope this helps.
Vladimira Krizova
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support