Search found 31 matches

by starsailor
Mon Oct 31, 2016 11:37 pm
Forum: CloverETL Support
Topic: Apache Kafka reader component.
Replies: 5
Views: 1436

Re: Apache Kafka reader component.

Hi Cholastal, Unfortunately we haven't had the opportunity to look into the Kafka sub-graphs that you provided in this thread primarily because we were stuck on an older version of CloverETL Designer and Server that didn't support sub-graphs, v3.5.1.018 to be specific. Fortunately, we're finally sen...
by starsailor
Tue Jun 07, 2016 7:01 pm
Forum: CloverETL Support
Topic: Apache Kafka reader component.
Replies: 5
Views: 1436

Re: Apache Kafka reader component.

Thanks Lukas,

Is there a (good) way to integrate an existing groovy Kafka consumer into a generic data flow component such that the data flow can process the messages in a Kafka topic in a data flow?
by starsailor
Wed May 18, 2016 6:18 pm
Forum: CloverETL Support
Topic: Apache Kafka reader component.
Replies: 5
Views: 1436

Apache Kafka reader component.

Is there anyway any of the existing reader components can read from an Apache Kafka topic?

If not, has anyone developed a Kafka reader component that's open source or otherwise available?
by starsailor
Tue Apr 05, 2016 12:23 am
Forum: CloverETL Support
Topic: Disable CloverETL Server Simple HTTP API endpoint.
Replies: 1
Views: 545

Disable CloverETL Server Simple HTTP API endpoint.

Is there a mechanism in the CloverETL Server configuration to disable the Simple HTTP API endpoint from accepting any requests? Specifically, we'd like to disable remote job execution for a specific period of time on the CloverETL Server and then re-enable it after the time period.
by starsailor
Wed Mar 30, 2016 3:52 pm
Forum: CloverETL Support
Topic: ListFiles component not listing directory contents on AWS S3.
Replies: 5
Views: 1475

Re: ListFiles component not listing directory contents on AWS S3.

Can you confirm or deny that this works in the current CloverETL version? That would help motivate us to upgrade soon, as we've putting it off for quite a while. In the meantime, I've coded a workaround using the SystemExecute component to execute "s3cmd ls ..." in a Bash shell. It's a pai...
by starsailor
Mon Mar 28, 2016 6:57 pm
Forum: CloverETL Support
Topic: ListFiles component not listing directory contents on AWS S3.
Replies: 5
Views: 1475

Re: ListFiles component not listing directory contents on AWS S3.

When I change the protocol from HTTP:// to S3:// I get the following error (note actual keys and path changed for security purposes) ------------------------------------------------------------------------------------------ Error details --------------------------------------------------------------...
by starsailor
Mon Mar 21, 2016 9:34 pm
Forum: CloverETL Support
Topic: ListFiles component not listing directory contents on AWS S3.
Replies: 5
Views: 1475

ListFiles component not listing directory contents on AWS S3.

I trying to use the ListFiles component to list the files in a sub-directory of an AWS S3 bucket. Regardless of the URL that I create to the sub-directory in the ListFiles "FileURL" property using the supplied editor all I can seem to get is one record output of the sub-directory it self. ...
by starsailor
Fri Mar 18, 2016 7:07 pm
Forum: CloverETL Support
Topic: CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).
Replies: 7
Views: 1375

Re: CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).

Hi Lukas, After making the phase modification that you suggested the connections to the AWS SES SMTP server no longer regularly times out when the phase (not the whole jobflow) the EmailSender component is in does less work that takes elapsed time since the phase initialization. As long as the Email...
by starsailor
Tue Mar 15, 2016 4:16 pm
Forum: CloverETL Support
Topic: CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).
Replies: 7
Views: 1375

Re: CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).

Hi Lukas,

Yes, the jobflow only has one phase, phase 0. We'll split the EmailSender component into 2 phases with the EmailSender in the final phase at the end of the jobflow. I'll report the results back here.

Thanks for the suggestion.
by starsailor
Fri Mar 11, 2016 8:15 pm
Forum: CloverETL Support
Topic: CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).
Replies: 7
Views: 1375

CloverETL EmailSender component incompatible with AWS Simple Email Service (SES).

Hi, All our DW systems, including CloverETL Servers, are deployed on AWS EC2. We have many CloverETL jobflows that need to be able to send status emails to based on data available in the jobflows. To do so we are using the EmailSender component in the jobflow. It seems to work for the quickest and s...
by starsailor
Tue Mar 08, 2016 9:54 pm
Forum: CloverETL Support
Topic: ExecuteScript evaluating expressions in Bash script.
Replies: 1
Views: 701

ExecuteScript evaluating expressions in Bash script.

I'm using the ExecuteScript component to execute an inline script, i.e. the Bash script text is in the "Script" field of the component. I'm getting WARNings like: 2016-03-08 18:20:54,767 WARN 589063 [http-8080-19] Cannot evaluate expression: date '+%F %T %Z' from the component trying to ev...
by starsailor
Thu Apr 30, 2015 11:38 pm
Forum: CloverETL Server
Topic: MonitorJobflow or MonitorGraph component examples?
Replies: 2
Views: 906

Re: MonitorJobflow or MonitorGraph component examples?

I probably should have mention this when I started the topic... We have 2 jobflows that are executed independently from 2 separate scheduled and we'd like to find a way to start a third job to monitor them, i.e. the monitoring job is not the parent of either schedlued job so that pretty much rules o...
by starsailor
Wed Apr 29, 2015 8:45 pm
Forum: CloverETL Server
Topic: MonitorJobflow or MonitorGraph component examples?
Replies: 2
Views: 906

MonitorJobflow or MonitorGraph component examples?

Does anyone have an example of using the MonitorJobflow or MonitorGraph components where multiple jobs/graphs are being scheduled or otherwise executed in parallel and then monitored for completion by another job?

I'd appreciate any concrete example as I have the need to implement the same ASAP.
by starsailor
Thu Feb 05, 2015 12:28 am
Forum: CloverETL Server
Topic: Changing CloverETL Server "server properties"
Replies: 2
Views: 955

Re: Changing CloverETL Server "server properties"

Hi Jan,

Your instruction worked fine. Thanks.
by starsailor
Wed Feb 04, 2015 12:38 am
Forum: CloverETL Server
Topic: Changing CloverETL Server "server properties"
Replies: 2
Views: 955

Changing CloverETL Server "server properties"

I need to change my CloverETL Server v3.5.1 server properties, specifically to change the SMTP configuration. My server is running under Apache Tomcat. I don't see a way to do this in the Server Console GUI, so I'm assuming I need to change the server properties using one of the methods document in ...