Need simple example with quotedStrings

Discussion on developing CloverETL engine, transformation components etc.

ericl
Posts: 5
Joined: Thu Mar 28, 2013 9:07 pm

Need simple example with quotedStrings

Postby ericl » Thu Mar 28, 2013 9:26 pm

Hi,

I'm a beginner at using Clover. I'm currently using the code from DataParsing.java (provided in the examples/JavaExamples section) and it works perfectly except for the case where my data fields are surrounded by quotes.

My source file has this format: "John", "Smith", etc.

I tried to set the quotedStrings to true at the record level, hence something like this:

Code: Select all

<Record fieldDelimiter="," recordDelimiter="\r\n" name="Person" recordSize="-1" type="delimited" skipFirstLine="false" quotedStrings="true">


but the resulting values after being parsed still end up with the double quotes (I end up with "John" with the double quotes from the call to record.getField(0).toString()).

After reading a bit on the forum I discovered that the quotedStrings attribute seems to apply only to the DataReader.

If that is the case, would someone have a simple example on how to use the DataReader in regards of the DataParsing.java example ?

Thank you very much!

Eric

slechtaj
Posts: 192
Joined: Wed Aug 15, 2012 8:18 am

Re: Need simple example with quotedStrings

Postby slechtaj » Fri Mar 29, 2013 2:09 pm

Hi Eric,

A similar issue has been reported in earlier versions. What version do you use?
Jan Slechta
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

How to speed up communication with CloverCARE support

ericl
Posts: 5
Joined: Thu Mar 28, 2013 9:07 pm

Re: Need simple example with quotedStrings

Postby ericl » Mon Apr 01, 2013 3:02 pm

I'm using version 3.3.1.

Am I doing it correctly by using the quotedStrings attribute at the "Record" level in the fmt file ?

Any alternative to make this work would be greatly appreciated!

Thank you!

Eric

slechtaj
Posts: 192
Joined: Wed Aug 15, 2012 8:18 am

Re: Need simple example with quotedStrings

Postby slechtaj » Tue Apr 02, 2013 2:59 pm

Hi Eric,

DataParsing.java does not use any external FMT file. The metadata are created inside the code. Due to a logic in the code, it is not possible to set qutedStrings in the java file, since this would be ignored.
However, you may sligthly modify the code. All you need to do is to replace parser

Code: Select all

Parser parser = TextParserFactory.getParser(metadata);

with the following code:

Code: Select all

 final TextParserConfiguration parserCfg = new TextParserConfiguration();
       parserCfg.setMetadata(metadata);
       parserCfg.setCharset("UTF8");
       parserCfg.setVerbose(true);
       parserCfg.setTreatMultipleDelimitersAsOne(false);
       parserCfg.setQuotedStrings(metadata.isQuotedStrings());
       parserCfg.setQuoteChar(metadata.getQuoteChar());
       parserCfg.setSkipLeadingBlanks(false);
       parserCfg.setSkipTrailingBlanks(false);
       parserCfg.setTryToMatchLongerDelimiter(DataRecordUtils.containsPrefixDelimiters(parserCfg.getMetadata()));
       parserCfg.setTrim(null);
       Parser parser = TextParserFactory.getParser(parserCfg);


Besides this, you will also need to adjust the metadata accordingly to your data source.

Please note that you should be working on component level (create your own graphs) instead of pareser level.
Jan Slechta
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

How to speed up communication with CloverCARE support

ericl
Posts: 5
Joined: Thu Mar 28, 2013 9:07 pm

Re: Need simple example with quotedStrings

Postby ericl » Tue Apr 02, 2013 4:00 pm

Thank you very much Jan, that solved my problem!

I will read more on processing the data at the component level, any simple example you could suggest (GraphSort.java maybe) ?

Thanks again,

Eric

slechtaj
Posts: 192
Joined: Wed Aug 15, 2012 8:18 am

Re: Need simple example with quotedStrings

Postby slechtaj » Wed Apr 03, 2013 3:01 pm

Hi Eric,

Yes, this is correct, GraphSort.java is the file you may use. There is also another example (XMLGraph.java), which is used to run the graph from XML definition.
Jan Slechta
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

How to speed up communication with CloverCARE support