Search found 13 matches

by hewills
Thu Dec 14, 2017 7:36 pm
Forum: CloverETL Support
Topic: CTL function example
Replies: 0
Views: 1

CTL function example

Hi, I'm wanting to create my own simple function. I believe I can use CTL to do it, and don't need Java. Basically I'm hoping to wrap some CTL code into a function, so we can use it in many jobs easily. Example : function handle_null(record) is record a string? return '-' is record a number? return ...
by hewills
Wed Nov 15, 2017 6:32 pm
Forum: CloverETL Support
Topic: zeroDate returning 12/31/1969
Replies: 2
Views: 67

Re: zeroDate returning 12/31/1969

Ah, can't believe I missed that! Thanks, that worked great.
by hewills
Tue Nov 14, 2017 9:34 pm
Forum: CloverETL Support
Topic: Different results using str2date
Replies: 1
Views: 44

Different results using str2date

I get different dates returned depending on what date format I use in str2date. I'm using a str2date in Reformat, and it's being loaded into a date metadata field. The metadata is being formatted as yyyy-MM-dd HH:mm:ss , and my default local timezone. If I use str2date('01.01.1753','MM.dd.yyyy') It ...
by hewills
Tue Nov 14, 2017 7:00 pm
Forum: CloverETL Support
Topic: zeroDate returning 12/31/1969
Replies: 2
Views: 67

zeroDate returning 12/31/1969

Hi, I'm trying to use the function zeroDate(), which returns 1.1.1970. I'm using it in a Reformat object, then passing it into the metadata as a data field. When I inspect the data being passed into the metadata, it reads 12.31.1969. Do you know what could be causing this? The timezone and Locale is...
by hewills
Fri May 26, 2017 6:09 pm
Forum: CloverETL Server
Topic: Tomcat user running CloverETL
Replies: 1
Views: 340

Tomcat user running CloverETL

Currently we are using tomcat as our application server, and as such the tomcat user is running CloverETL. Since it's normally recommended to have tomcat as a user without login, it makes it tricky having Clover run shell scripts. Basically we have to have tomcat sudo as another user. Is there a rec...
by hewills
Thu May 25, 2017 7:54 pm
Forum: CloverETL Support
Topic: skip graph initialization
Replies: 1
Views: 242

skip graph initialization

I know there is a way to skip the graph configuration, using skip_check_config . But is there a way to have a graph skip the initialization, and just start running? Basically, I have a child graph with dynamic metadata using a sql query - SELECT * FROM SOURCE_TBL WHERE 1=0; Before running the graph,...
by hewills
Tue May 02, 2017 5:02 pm
Forum: CloverETL Server
Topic: Using allocation with DBInputTable
Replies: 5
Views: 553

Re: Using allocation with DBInputTable

Thanks Pedro! I had a chance to use your design, and it sped up the graph as expected. As you said, using the 'ParallelPartition' seems to be key.
Using 'PARTITION' wouldn't be an option for the database we are currently working with. But I'll keep it in mind for the future.
by hewills
Tue Apr 25, 2017 11:42 pm
Forum: CloverETL Server
Topic: Setting number of executors to a jobflow
Replies: 5
Views: 550

Re: Setting number of executors to a jobflow

Now that I understand how the executors work, I don't think the 'max_running_concurrently' parameter is needed. But it's nice to know in case we need it for a different scenario.
I setup the ExecuteGraph with dynamic table parameters and metadata, and so far it's working great. Thanks!
by hewills
Mon Apr 24, 2017 7:50 pm
Forum: CloverETL Server
Topic: Setting number of executors to a jobflow
Replies: 5
Views: 550

Re: Setting number of executors to a jobflow

I decided to try setting up the "dynamic table load", where I feed a list of table names and parameters to one ExecuteGraph. This looks like a better solution for us.
by hewills
Fri Apr 21, 2017 6:07 pm
Forum: CloverETL Server
Topic: Using allocation with DBInputTable
Replies: 5
Views: 553

Re: Using allocation with DBInputTable

Hi, thanks for taking a look. How many records are you processing? ~ 3,000,000 How long does this process usually take? 7.5 minutes What Database Server are you utilizing (vendor, version)? Oracle 12c database, on Red Hat 7.2 Clover Server is running on CentOS I don't have access to the source serve...
by hewills
Fri Apr 21, 2017 5:51 pm
Forum: CloverETL Server
Topic: Setting number of executors to a jobflow
Replies: 5
Views: 550

Re: Setting number of executors to a jobflow

Thanks for the reply Eva, that makes sense. You're right that in my example the jobs could be run in parallel, except if too many run at a time, it causes issues. We have about 500 graphs that need to be executed, so I'm trying to figure out the best way to organize them. I was hoping that there was...
by hewills
Tue Apr 18, 2017 5:00 pm
Forum: CloverETL Server
Topic: Using allocation with DBInputTable
Replies: 5
Views: 553

Using allocation with DBInputTable

I'm using Corp Plus Server, and saw that the data partition objects are available to use, including the Allocation parameter. I've been testing how to use it on a DBInputTable object, where each allocation pulls a different set of data from our source table, and I use ParallelSimpleGather to bring t...
by hewills
Tue Apr 18, 2017 4:46 pm
Forum: CloverETL Server
Topic: Setting number of executors to a jobflow
Replies: 5
Views: 550

Setting number of executors to a jobflow

I have a jobflow with 8 graphs, that I'm running on Clover Server. Each graph is truncating a target table, selecting data from my source, and loading the target table. They are connected and running synchronously in the jobflow. I created a "global" parameter called EXECUTORS, with a valu...