Delimited FlatFile reader - Question from New User

Support/help with CloverETL implementation problems

mohan@datki.com
Posts: 1
Joined: Thu Mar 15, 2007 12:53 am

Delimited FlatFile reader - Question from New User

Postby mohan@datki.com » Sun Mar 25, 2007 5:31 pm

I am exporting an excel file into a csv text and trying to parse this data using delimited flatfile reader. The file is , delimited (dumped from excel using the save as feature).

One of the string inside has , by itself as it is a long comment string.

I need an optional way of specifying the fact that these strings are within double quotes ie "STRING" and STRING itself will contain some commas which should be skipped.

e.g. orderid, comment, itemname are the three fields
File is delimited by ,
comment in some cases is within comma and double quotes (when there is a comment which contains comma) or it is delimited just by comma from orderid.

I am wondering whether the format parameter for fields will help here.

Any pointers will be greatly appreciated.

thanks
mohan s.

jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

Postby jausperger » Wed Apr 04, 2007 3:37 pm

Hello,

delimited reader defines only line and field delimiter, more complicated structure isn't possible. To be able to skip some substring you need append a reformat component.

Try something like this:

<Node id="REF" type="REFORMAT">
<attr name="transform">
import org.jetel.*;

public class reformatOrders extends DataRecordTransform{
DataRecord source,target;
int counter;
String message;

public boolean transform(DataRecord[] _source, DataRecord[] _target){
source=_source[0]; target=_target[0];
try{
SetVal.setInt(target,"orderid",GetVal.getInt(source,"orderid"));
message = GetVal.getString(source,"messagecomment")
// some operation over message
SetVal.setString(target,"messagecomment",message);
}catch(Exception ex){
message=ex.getMessage()+" ->occured with record :"+counter;
throw new RuntimeException(message);
}
counter++;
return true;
}
}
</attr>
</Node>

ja

mzatopek
Posts: 76
Joined: Fri May 11, 2007 9:49 am

Postby mzatopek » Fri May 11, 2007 9:54 am

Maybe you are searching for our Universal data reader, check out 'quotedStrings' attribute in the documentation.

Martin Zatopek

mukunku
Posts: 5
Joined: Wed Nov 01, 2017 3:49 pm

Re: Delimited FlatFile reader - Question from New User

Postby mukunku » Tue Feb 27, 2018 8:59 pm

I find it disappointing that the Flat File Reader can't read a RFC 4180 compliant CSV. The reader requires commas to be quoted themselves where as the standard recommends enclosing the entire string in quotes.

bartonv
Posts: 63
Joined: Wed May 03, 2017 12:10 pm

Re: Delimited FlatFile reader - Question from New User

Postby bartonv » Mon Mar 05, 2018 3:19 pm

Hi Mukunku,
I have noticed your post concerning this topic and it appears that this thread is more than 10 years old. The good news is that Clover has evolved since then and the functionality that handles RFC 4180 compliant CSV files is now there. Feel free to review the attached example graph as a demonstration of how this can be approached. Worth noting is that FlatFileReader is capable of extracting the metadata automatically with no additional manual changes needed. Hope this helps.
Kind regards,
Attachments
parsingCSVfile.zip
(1.06 KiB) Downloaded 12 times
---
Vladimir Barton
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support


cron