mason6787
Posts: 3
Joined: Mon Nov 02, 2015 2:48 pm

How to increase CPU load on DBInputTable components

Postby mason6787 » Mon Aug 01, 2016 4:55 pm

Hello,

I have several large Clover jobs which include multiple queries that return millions of rows. When running them within the DBInputTable components, I've noticed the the CPU load (indicated in the tracking tab in the clover portal) seems to vary quite a bit form day to day.

Can anyone give me a good idea on what I can do to maximize the CPU utilization on the specific threads that are running these components?

Thank you,

cholastal
Posts: 103
Joined: Tue Sep 01, 2015 1:22 pm

Re: How to increase CPU load on DBInputTable components

Postby cholastal » Tue Aug 02, 2016 2:38 pm

Hi,

Here are some options how to possibly increase the CPU load:

    1. Increase the value of the Fetch size property of the DBInputTable component.
    2. Use more DBInputTable components (in single graph) and load only some part of the table in each of them (parallel loading). When the loading is done, you can merge the results by for example a SimpleGather component.

However, I would recommend to focus more on the speed of the loading instead of the CPU load as a higher CPU load isn't always the same as faster input from a database. You can try setting different values to the Fetch size property and play with the number of DBInputTables in order to achieve the best possible speed. Also, if you have a Cluster license and at least two cluster nodes, you can use the Cluster components to distribute the reading process to these nodes (in order to increase the speed). You can find more information about clustering here and here.

Hope this helps.
---
Lukas Cholasta
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support