No clover.bat in 4.0.0 Download

Support/help with CloverETL implementation problems

skaszyk
Posts: 12
Joined: Mon Apr 14, 2014 9:52 pm

No clover.bat in 4.0.0 Download

Postby skaszyk » Mon Apr 14, 2014 9:57 pm

I had been using a plugin version of CloverETL withing Eclipse. However, I need the ability to run a graph from the command line (so I can schedule it via Windows Task Manager). So, today, I downloaded the standalone version:

Version: 4.0.0.026M1

I searched for a clover.bat file which is apparently needed to run from a command line and no such file was found. Did I get a corrupted download or has something major changed in this version (I'm assuming a major changes since you jumped to v4.0.0...).

Here's is the forum topic I was following:

http://forum.cloveretl.com/viewtopic.php?f=4&t=4986#p7850

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: No clover.bat in 4.0.0 Download

Postby kubosj » Fri Apr 18, 2014 11:31 am

Hi skaszyk,

The scripts have been indeed removed in the release 4.0.0-M1. They were by-product of our build process and not needed by the Designer application. Additionally we are in the process of redesigning our embedded runtime in Designer, so the scripts stop working with the next release 4.0.0-M2.

If you came across the scripts and used them, please also be aware they do not initialize plugin system correctly and graphs containing components that are not part of open source Engine do not run. In case you want to keep using CloverETL from command-line, consider switching to the open source Engine instead, where clover.bat originates from. The open source Engine however does not contain all components and does not support Jobflow or Subgraphs; those are only supported by our commercial products.
Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

skaszyk
Posts: 12
Joined: Mon Apr 14, 2014 9:52 pm

Re: No clover.bat in 4.0.0 Download

Postby skaszyk » Tue Apr 22, 2014 4:51 pm

I downloaded the open source Engine as you suggested but I just can't seem to get the command correct. No matter how I format the command, it doesn't get parsed correctly and I either get the usage printout or I get an error that it can't find the .grf (because the path isn't parsed correctly).

Can you provide some guidance for a full command that works on Windows?

Here's the usage result with my command in bold:

C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPL File Loaders>clover.bat "-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
CLOVER_HOME=C:\cloverETL\bin\.\..
CLOVER_CMD_LINE_ARGS="-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
JAVA_CMD_LINE_ARGS=
USER_CLASSPATH=
"C:\Program Files\Java\jdk1.7.0_45\bin\java.exe" -classpath ";;C:\cloverETL\bin\;C:\Program Files\Java\jdk1.7.0_45\lib\tools.jar;;C:\cloverETL\bin\.\../lib/annotations.jar;C:\cloverETL\bin\.\../lib/cloveretl.engine.jar;C:\cloverETL\bin\.\../lib/commons-cli-1.2.jar;C:\cloverETL\bin\.\../lib/commons-codec-1.6.jar;C:\cloverETL\bin\.\../lib/commons-collections-3.2.1.jar;C:\cloverETL\bin\.\../lib/commons-compress-1.7.jar;C:\cloverETL\bin\.\../lib/commons-io-2.2.jar;C:\cloverETL\bin\.\../lib/commons
-io-LICENSE.txt;C:\cloverETL\bin\.\../lib/commons-logging-1.1.3.jar;C:\cloverETL\bin\.\../lib/commons-net-3.3.jar;C:\cloverETL\bin\.\../lib/commons-pool-1.6.jar;C:\cloverETL\bin\.\../lib/dom4j-1.6.1.jar;C:\cloverETL\bin\.\../lib/ftp4j-1.7.2.jar;C:\cloverETL\bin\.\../lib/ftp4j-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpclient-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpclient-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpcore-4.3.1.jar;C:\cloverETL\bin\.\../lib/httpcore-LICENSE.txt;C:\cloverETL\bin\.\../l
ib/httpmime-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpmime-LICENSE.txt;C:\cloverETL\bin\.\../lib/icu4j-52_1.jar;C:\cloverETL\bin\.\../lib/icu4j-LICENSE.html;C:\cloverETL\bin\.\../lib/icu4j-unicode-LICENSE.txt;C:\cloverETL\bin\.\../lib/jackson-core-asl-1.9.3.jar;C:\cloverETL\bin\.\../lib/javolution.jar;C:\cloverETL\bin\.\../lib/jaxen-1.1.6.jar;C:\cloverETL\bin\.\../lib/jcifs-1.3.17.jar;C:\cloverETL\bin\.\../lib/jets3t-0.9.0.jar;C:\cloverETL\bin\.\../lib/jms.jar;C:\cloverETL\bin\.\../lib/joda-time-2
.1.jar;C:\cloverETL\bin\.\../lib/jsch-0.1.50.jar;C:\cloverETL\bin\.\../lib/jxl.jar;C:\cloverETL\bin\.\../lib/LICENSE.txt;C:\cloverETL\bin\.\../lib/log4j-1.2.17.jar;C:\cloverETL\bin\.\../lib/poi-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-schemas-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/sardine-5.0.2.jar;C:\cloverETL\bin\.\../lib/signpost-commonshttp4-1.2.1.2.jar;C:\cloverETL\bin\.\../lib/signpost-core-1.2.1.2.jar;C:\cloverETL\bin\.\
../lib/slf4j-api-1.7.2.jar;C:\cloverETL\bin\.\../lib/slf4j-log4j12-1.7.2.jar;C:\cloverETL\bin\.\../lib/truezip-driver-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-driver-zip-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-kernel-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-path-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-swing-7.7.5.jar;C:\cloverETL\bin\.\../lib/xmlbeans-2.5.0-modified-for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-1.4.7-modified-
for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-LICENSE.txt" "-Dclover.home=C:\cloverETL\bin\.\.." org.jetel.main.runGraph -plugins "C:\cloverETL\bin\.\..\plugins" "-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
Usage: runGraph [-(v|cfg|logcfg|loglevel|P:|tracking|info|plugins|pass)] <graph definition file>
Options:
-v be verbose - print even graph layout
-P:<key>=<value> add definition of property to global graph's property list
-cfg <filename> load definitions of properties from specified file
-logcfg <filename> load log4j configuration from specified file;
if not specified, "log4j.properties" should be in classpath
-loglevel <level> overrides log4j configuration and sets specified logging level for rootLogger;
possible levels are: ALL | TRACE | DEBUG | INFO | WARN | ERROR | FATAL | OFF
-tracking <seconds> how frequently output the graph processing status
-info print info about Clover library version
-plugins directory where to look for plugins/components
-pass password for decrypting of hidden connections passwords
-stdin load graph definition from STDIN
-loghost define host and port number for socket appender of log4j (log4j library is required); i.e. localhost:4445
-checkconfig only check graph configuration
-skipcheckconfig skip checking of graph configuration
-noJMX turns off sending graph tracking information
-config <filename> load default engine properties from specified file
-nodebug turns off all runtime debugging e.g edge debugging
-debugdirectory <foldername> directory dedicated to store temporary debug data; default is java's temporary folder

Note: <graph definition file> can be either local filename or URL of local/remote file

Here's where the graph can't be found when I try to use full paths in double quotes:

C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPL File Loaders>clover.bat "-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
CLOVER_HOME=C:\cloverETL\bin\.\..
CLOVER_CMD_LINE_ARGS="-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
JAVA_CMD_LINE_ARGS=
USER_CLASSPATH=
"C:\Program Files\Java\jdk1.7.0_45\bin\java.exe" -classpath ";;C:\cloverETL\bin\;C:\Program Files\Java\jdk1.7.0_45\lib\tools.jar;;C:\cloverETL\bin\.\../lib/annotations.jar;C:\cloverETL\bin\.\../lib/cloveretl.engine.jar;C:\cloverETL\bin\.\../lib/commons-cli-1.2.jar;C:\cloverETL\bin\.\../lib/commons-codec-1.6.jar;C:\cloverETL\bin\.\../lib/commons-collections-3.2.1.jar;C:\cloverETL\bin\.\../lib/commons-compress-1.7.jar;C:\cloverETL\bin\.\../lib/commons-io-2.2.jar;C:\cloverETL\bin\.\../lib/commons
-io-LICENSE.txt;C:\cloverETL\bin\.\../lib/commons-logging-1.1.3.jar;C:\cloverETL\bin\.\../lib/commons-net-3.3.jar;C:\cloverETL\bin\.\../lib/commons-pool-1.6.jar;C:\cloverETL\bin\.\../lib/dom4j-1.6.1.jar;C:\cloverETL\bin\.\../lib/ftp4j-1.7.2.jar;C:\cloverETL\bin\.\../lib/ftp4j-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpclient-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpclient-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpcore-4.3.1.jar;C:\cloverETL\bin\.\../lib/httpcore-LICENSE.txt;C:\cloverETL\bin\.\../l
ib/httpmime-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpmime-LICENSE.txt;C:\cloverETL\bin\.\../lib/icu4j-52_1.jar;C:\cloverETL\bin\.\../lib/icu4j-LICENSE.html;C:\cloverETL\bin\.\../lib/icu4j-unicode-LICENSE.txt;C:\cloverETL\bin\.\../lib/jackson-core-asl-1.9.3.jar;C:\cloverETL\bin\.\../lib/javolution.jar;C:\cloverETL\bin\.\../lib/jaxen-1.1.6.jar;C:\cloverETL\bin\.\../lib/jcifs-1.3.17.jar;C:\cloverETL\bin\.\../lib/jets3t-0.9.0.jar;C:\cloverETL\bin\.\../lib/jms.jar;C:\cloverETL\bin\.\../lib/joda-time-2
.1.jar;C:\cloverETL\bin\.\../lib/jsch-0.1.50.jar;C:\cloverETL\bin\.\../lib/jxl.jar;C:\cloverETL\bin\.\../lib/LICENSE.txt;C:\cloverETL\bin\.\../lib/log4j-1.2.17.jar;C:\cloverETL\bin\.\../lib/poi-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-schemas-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/sardine-5.0.2.jar;C:\cloverETL\bin\.\../lib/signpost-commonshttp4-1.2.1.2.jar;C:\cloverETL\bin\.\../lib/signpost-core-1.2.1.2.jar;C:\cloverETL\bin\.\
../lib/slf4j-api-1.7.2.jar;C:\cloverETL\bin\.\../lib/slf4j-log4j12-1.7.2.jar;C:\cloverETL\bin\.\../lib/truezip-driver-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-driver-zip-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-kernel-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-path-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-swing-7.7.5.jar;C:\cloverETL\bin\.\../lib/xmlbeans-2.5.0-modified-for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-1.4.7-modified-
for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-LICENSE.txt" "-Dclover.home=C:\cloverETL\bin\.\.." org.jetel.main.runGraph -plugins "C:\cloverETL\bin\.\..\plugins" "-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
INFO [main] - *** CloverETL framework/transformation graph, (c) 2002-2014 Javlin a.s. ***
INFO [main] - Running with CloverETL library version 4.0.0.M1 build#026 compiled 10/04/2014 02:48:33
INFO [main] - Running on 8 CPU(s), OS Windows 7, architecture amd64, Java version 1.7.0_45, max available memory for JVM 3715584 KB
INFO [main] - Loading default properties from: defaultProperties
DEBUG [main] - Plugin org.jetel.bulkloader loaded.
id - org.jetel.bulkloader
version - 4.0.0
.... (more logging)

ERROR [main] -
----------------------------------------------------- Error details -------------------------------------------------------
Error - graph definition file can't be read
Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf (The system cannot find the path specified)
---------------------------------------------------------------------------------------------------------------------------
(notice it's not parsing the path correctly)

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: No clover.bat in 4.0.0 Download

Postby kubosj » Wed Apr 23, 2014 8:31 am

Hi skaszyk,

for project stored in "c:\test\sample project" (space in path is fine) having graph on subpath "graph\WebServicesHTTP.grf" you may execute engine by this command:

Code: Select all

c:\test\cloverETL\bin>clover.bat -contexturl "c:\test\sample project" graph\WebServicesHTTP.grf


I hope this helps.
Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

skaszyk
Posts: 12
Joined: Mon Apr 14, 2014 9:52 pm

Re: No clover.bat in 4.0.0 Download

Postby skaszyk » Thu Apr 24, 2014 8:00 pm

Worked perfectly, thanks for your help!

Fatboy40
Posts: 1
Joined: Mon Jul 28, 2014 1:00 pm

Re: No clover.bat in 4.0.0 Download

Postby Fatboy40 » Mon Jul 28, 2014 1:04 pm

kubosj wrote:If you came across the scripts and used them, please also be aware they do not initialize plugin system correctly and graphs containing components that are not part of open source Engine do not run...


Hi Jaroslav,

So if using some of the additional transformation components present in the full Designer we must use the Server product to schedule the graphs, you cannot call the Designer from the command line / a batch file and use all transformations ?.

Thanks.

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: No clover.bat in 4.0.0 Download

Postby kubosj » Mon Jul 28, 2014 3:02 pm

Hi Fatboy40,

Yes, it is forbidden by EULA to execute graphs on command line using Engine of commercial CloverETL Designer. You can either prepare graphs using OpenSource components and execute on OpenSource Engine or purchase CloverETL Server.

Please let me know if you have further questions.
Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com


cron