How to read data from db subsequently (incremental reading)?

How-to(s), solutions to common problems, pearls of wisdom

avackova
Posts: 841
Joined: Fri Jul 20, 2007 9:28 am

How to read data from db subsequently (incremental reading)?

Postby avackova » Thu Oct 23, 2008 8:35 am

It is possible to create graph, that reads only "new" (added from the last run of graph) data from database.
Look at following DBInputTable definition:

Code: Select all

<Node dbConnection="Connection0" id="INPUT" incrementalFile="dbInc.txt" incrementalKey="key=max(employee_id)" sqlQuery="select * from employee where employee_id>#key" type="DB_INPUT_TABLE"/>

In the above node, new records are defined by clause where employee_id>#key and key is stored as maximum value from the last database reading. You have also to define the file, that stores the incremental key value. For the first reading you must define the key by hand (create file, where are keys and theirs values eg.

Code: Select all

key = -1
).

The incremental key can consist of more then one field (sqlQuery="select * from employee where hire_date > #key1 or employee_id > #key2", incrementalKey="key1=first(hire_date);key2=max(employee_id)") and available value in key definition are: first,last,max and min.
Agata Vackova
Javlin a.s.
agata.vackova@javlin.eu