Loading all records of a Lookup Table in Custom Java Transformer

Discussion on developing CloverETL engine, transformation components etc.

pareshjoshi
Posts: 7
Joined: Thu Jan 21, 2016 6:48 pm

Loading all records of a Lookup Table in Custom Java Transformer

Postby pareshjoshi » Tue Apr 12, 2016 5:11 pm

Hi,
I have created a custom java transformer and want to read all the data from a look up table.

In the graph, I first populate look up table from an excel sheet. It has 3 records and I confirmed that all the 3 records exists in look up table. However in my java code, I can only see one record when I try to read it.

The Key for look up table is called "Key" field and has 0 as value in all 3 records. Here is my Java code

Code: Select all

   private void loadAllLookUpRecords() {
      try {
         System.out.println("Loading loadAllLookUpRecords");

         LookupTable lt;
         lt = getGraph().getLookupTable(LOOKUP_TABLE_ID);
         DataRecord patternRecord = DataRecordFactory.newRecord(lt
               .getMetadata());
         patternRecord.getField(0).setValue(0);
         String[] lookupFields = { "Key" };
         RecordKey recordKey = new RecordKey(lookupFields, lt.getMetadata());

         Lookup lookup;
         lookup = lt.createLookup(recordKey, patternRecord);

         lookup.seek();
         System.out.println("lookup Size :: " + lookup.getNumFound());

         while (lookup.hasNext()) {
            System.out.println("Look up records Found");
            DataRecord record = lookup.next();
            allLookupDataRecords.add(record);
         }
         System.out.println("AllLookUpRecords Size :: " + allLookupDataRecords.size());

      } catch (Exception ex) {
         throw new JetelRuntimeException(ex);

      }
   }
   


lookup.getNumFound() always returns 1 record. I will really appreciate if someone can point in right direction to load all data from lookup table. Thanks, Paresh

vazquezrosariop
Posts: 101
Joined: Mon Feb 29, 2016 5:33 pm

Re: Loading all records of a Lookup Table in Custom Java Transformer

Postby vazquezrosariop » Mon Apr 18, 2016 6:28 pm

Hi,

Could you please tell me what kind of LookUp Table you are utilizing? If it’s a Simple lookup table, please make sure to check if the Allow key duplicates is enabled in your lookup table settings.

Regards,
Attachments
^60F7584418840E7380AEA521390CA2993C6B35926D5C102F28^pimgpsh_thumbnail_win_distr.jpg
^60F7584418840E7380AEA521390CA2993C6B35926D5C102F28^pimgpsh_thumbnail_win_distr.jpg (7.64 KiB) Viewed 1211 times
---
Pedro Vazquez Rosario
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

pareshjoshi
Posts: 7
Joined: Thu Jan 21, 2016 6:48 pm

Re: Loading all records of a Lookup Table in Custom Java Transformer

Postby pareshjoshi » Tue May 03, 2016 5:40 pm

Hi,
Thank you. As you suggested just had to check " Allow key duplicates" and that fixed the issue. Thanks