Running CloverETL from Java

Discussion on developing CloverETL engine, transformation components etc.

vsvoboda@trask.cz
Posts: 6
Joined: Mon Jun 04, 2007 11:25 am

Running CloverETL from Java

Postby vsvoboda@trask.cz » Thu Jul 12, 2007 10:08 am

Hi,
can you tell me how should I CloverETL run from Java? Can you tell me how Application call CloverETL? Where can I find the sample of configuration file for running CloverETL from Java?


thanks

jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

Postby jausperger » Fri Jul 20, 2007 10:07 am


jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

Postby jausperger » Fri Jul 20, 2007 1:53 pm

Hi,

maybe I wrong understood the question. Of course you can run graph from java, some examples how to run cloverETL graph from java, can be found in

http://www.cloveretl.org/download/examp ... -2-1-1.zip
cloverExamples.rel-2-1-1.zip/cloverETL/examples/javaExamples

I recommend to see testGraphSort.java. You can inspire with runGraph.java which is a main class that runs clover ETL graphs.

Basic informations are on wiki pages http://wiki.clovergui.net/doku.php?id=t ... on_concept

Regard,

ja

bcervante2007
Posts: 2
Joined: Wed Nov 28, 2007 8:28 pm

I need your help!!!

Postby bcervante2007 » Wed Nov 28, 2007 9:15 pm

Hello jausperger:
In spite of having visited the connections I continue with the doubt, me pregonto if maybe you could give me a bigger explanation, and to give me an example.
It is that I am new working with the CloverETL, and dont know a lot.
If you have something that can serve it you can send to: bcervante@estudiantes.uci.cu
Muchas gracias.

It excuses any error in the writing, English is not my language.

Greetings:
Betty.

jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

two ways

Postby jausperger » Mon Dec 10, 2007 1:57 pm

Hello,

there are 2 ways how to import clover etl project without gui. The site "http://www.cloveretl.org/download/clover-etl/" contains some releases and source codes.

1) If you want to use or re-extend clover etl faetures, please use the release. There are jars files which add to your project. You can write even own components. It is very easy. Download cloverETL.rel-2-3-x.zip and create new java project and add all jars from zip to java-build-path in a project properties folder. And then you can use some example in javaExamples...

2) If you want to make deep changes then there is better to use source codes. You have to create several project, each for plugin and one for engine and you have to copy src directories from zip(and relevant plugin) to your source project directries. And then you must to set the class paths.

ja

CrazyBird
Posts: 8
Joined: Mon May 19, 2008 1:45 pm

Outdated examples

Postby CrazyBird » Thu Jun 05, 2008 1:20 pm

We are thinking about using CloverETL within our projects in java.

Unfourtunately the examples like here http://wiki.clovergui.net/doku.php?id=t ... on_concept are oudated. They are using deprecated methods and there are always missing some details so that just simple "copy-and-paste" wouldn't work.



In general I think that there is not enough docuemntation for JasperETL, also there are too less java- examples compared to the many XML- bases ones.

It would really help to have some COMPLETE (in full java code) examples using best practises.

jurban
Posts: 162
Joined: Fri Jul 20, 2007 9:25 am

Postby jurban » Fri Jun 06, 2008 4:10 pm

Hi!
thank you for your interest in Clover. We will update the Java Clover examples so they work with the newest Clover and then we'll let you know.

Jaro

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Sun Aug 24, 2014 6:25 pm

>>We will update the Java Clover examples so they work with the newest Clover and then we'll let you know.

Are there any updated examples of this available?

I am completely new at this - have used CloverETL designer frequently on my local PC but this is my first time trying to execute a graph on a remote Linux box from a command line. I have downloaded the CloverETL engine to my local PC, and want to understand how to do this step-by-step, for example:

1) I assume that I need to unzip the CloverETL engine file on my local PC and then copy the extracted folders/files to the remote Linux computer. If so, to where? See attached screen print of the folder structure on the remote Linux computer.
2) Do I need to update any Linux environment variables, e.g. PATH?
3) I have created a simple graph on my local PC; where do I copy the file to on the remote Linux computer?
4) What is the basic syntax to execute the graph that I have copied to the remote Linux computer?
5) How do I structure the execution statement to pass parameters to the graph to specify the source and target file(s)?

Once I understand these basic questions, I believe I can continue from there.

Thank you.
Attachments
VPS Folder Listing.jpg
VPS Folder Listing.jpg (103.12 KiB) Viewed 7060 times

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: Running CloverETL from Java

Postby kubosj » Tue Aug 26, 2014 2:02 pm

Hi Marlon,

I am afraid that there is no up-to-date documentation on topic of running Engine.

1) Engine does not care of location of installation. Good place for such software is /opt on linux systems.
2) No need to update ENV variables if you will execute Engine by full path.
3) Yes just copy your graphs, you may pick arbitrary folder.
4) Try to run shell script cloverETL/bin/clover from unzipped engine on your linux machine to see description of command line parameters.
5) Same as 4)

I hope this helps.
Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Thu Aug 28, 2014 1:19 am

Hi Jaroslav,

Your suggested instructions have helped me, yes.

I have not completed this successfully yet, but I am getting closer. Very close, I believe!

When I try to run the conversion using the following command:

~/opt/cloverETL/bin/clover runGraph -plugins ~/opt/cloverETL/plugins ~/opt/cloverETL/graph/web.grf

...then the result is what is captured in the attached text file (CloverETL.log). Can you advise how to correct this error? Note that I am correctly specifying the full path whenever I can to hopefully avoid any problems.

Thank you.
Attachments
CloverETL.log
(21.05 KiB) Downloaded 76 times

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: Running CloverETL from Java

Postby kubosj » Thu Aug 28, 2014 7:45 am

Hi,

You can take a look here viewtopic.php?f=4&t=6803&p=10948&hilit=contexturl#p10948

It is windows sample, but work under linux in similar way, so:

Code: Select all

/opt/cloverETL/clover -contexturl "/opt/cloverETL" graph/web.grf


But that means your workspace.prm is located at /opt/cloverETL/workspace.prm. That is not recommendable structure. I would recommend you:

* store graphs in different directory than Engine, eg. /opt/etl_graphs
* for each project create separate directory, eg. /opt/etl_graphs/hello_world
* in project directory should be files/directories you know from Designer - workspace.prm, graphs/web.grf, data-in/sample.txt, ...

Then your execute statement would be:

Code: Select all

/opt/cloverETL/clover -contexturl "/opt/etl_graphs/hello_world" graph/web.grf


I hope this helps.
Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Thu Aug 28, 2014 6:37 pm

Thank you :)

I made the changes that you suggested, and I was able to successfully execute a basic graph from the command line using the following statement:

/root/opt/cloverETL/bin/clover -contexturl "/root/opt/etl_graphs/web_graph" graph.web.grf

Now I can try to build on my success from here. Once again, thank you for your assistance!

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Fri Oct 10, 2014 12:08 am

Hi Jaroslav, et al

I have a follow-up question.

I am now executing a graph that includes a "Class", but when I execute the graph, I am getting an error, "Cannot find class", with the name of the class cited.

I have tried to replicate the location of the file on my local machine on the machine with the Clover Engine, but I am still getting the error. Can you advise where I should place the file(s)?

I have attached screen prints of the error that I am receiving, along with screen prints to illustrate the location of the files on my local machine and on the machine with the Clover Engine.

Thank you.

Marlon
Attachments
Clover Engine - Class Not Found Error - 2.jpg
Clover Engine - Class Not Found Error - 2.jpg (139.83 KiB) Viewed 6496 times
Clover Engine - Class Not Found Error - 1.jpg
Clover Engine - Class Not Found Error - 1.jpg (33.15 KiB) Viewed 6496 times

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Fri Oct 10, 2014 12:13 am

Hi Jaroslav,

Attached are three additional screen prints to illustrate the location of the files on the machine with the Clover Engine; this is the machine that is giving me the error. Since I am limited to three attachments to an individual post (and cannot load a .doc or .pdf file with all the screen prints in a single file), I have added these in a subsequent post here.

Thank you.

Marlon
Attachments
Clover Engine - Class Not Found Error - 6.jpg
Clover Engine - Class Not Found Error - 6.jpg (156.47 KiB) Viewed 6496 times
Clover Engine - Class Not Found Error - 5.jpg
Clover Engine - Class Not Found Error - 5.jpg (165 KiB) Viewed 6496 times
Clover Engine - Class Not Found Error - 4.jpg
Clover Engine - Class Not Found Error - 4.jpg (96.24 KiB) Viewed 6496 times

marlon4x
Posts: 6
Joined: Sun Aug 24, 2014 6:04 pm

Re: Running CloverETL from Java

Postby marlon4x » Thu Oct 16, 2014 4:12 pm

Hi Jaroslav, et al,

Just following up on my question here, I do not see a reply - can you advise?

Thank you.

Marlon