Clover script validation

Support/help with CloverETL implementation problems

madan_clover3
Posts: 16
Joined: Tue Jun 10, 2014 10:39 am

Clover script validation

Postby madan_clover3 » Tue Jan 31, 2017 9:59 am

Hi

I am trying to validate clover script,

For ex., $field1=$field/2; $field3=str2date($field4, "dd.MM.yyyy");

I need to validate the above script for compilations and data type mismatch problems. Is there any way I can do this before even transformation graph is executed.

Thanks in advance.

vazquezrosariop
Posts: 96
Joined: Mon Feb 29, 2016 5:33 pm

Re: Clover script validation

Postby vazquezrosariop » Thu Feb 02, 2017 2:38 pm

Hi,

CloverETL validates CTL code directly in the Transformation editor. Thus, everytime you write a little bit of code the editor will validate for syntax error. Another way CloverETL validates CTL code would be the Check Config which is triggered when you run your graph. The Check Config will validate for any syntax error resulting in a failed graph before any data is passed though the edges.

Please let me know the information I provided helped you, if not please describe your use case further.
---
Pedro Vazquez Rosario
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

madan_clover3
Posts: 16
Joined: Tue Jun 10, 2014 10:39 am

Re: Clover script validation

Postby madan_clover3 » Mon Feb 06, 2017 5:11 am

Hi,

I am executing transformation in java, not using transform editor. I need to validate clover script as and when user submits to some xyz api so that I can warn user to correct the script then and there. Validating in transformation process is too late to intimate user about script errors.

I am looking at CTLExpressionEvaluator, not successful yet. If possible can you provide me working java example for CTLExpressionEvaluator and describe when it is useful.

Thank you.

madan_clover3
Posts: 16
Joined: Tue Jun 10, 2014 10:39 am

Re: Clover script validation

Postby madan_clover3 » Mon Feb 06, 2017 7:40 am

Tried below options :

Option 1 :

String expression = "$field1=$field/2; $field3=str2date($field4, "dd.MM.yyyy");";

Code: Select all

ITLCompilerFactory compilerFactory = new TransformFactory.DefaultCompilerFactory();
final ITLCompiler compiler = compilerFactory.createCompiler(getGraph(), recordMetadatas, recordMetadatas, DEFAULT_SOURCE_CODE_CHARSET);
List<ErrorMessage> errorMessages = compiler.validate(expression);


Error: Line 1 column 1 - Line 1 column 23: Unable to access record field in global scope!


Option 2 :

String expression = "if($pf < 0){$pf = 0;} if($basic < 0){$basic = 0;} $salary=$pf+$basic;";

Code: Select all

ITLCompilerFactory compilerFactory = new TransformFactory.DefaultCompilerFactory();
final ITLCompiler compiler = compilerFactory.createCompiler(getGraph(), recordMetadatas, recordMetadatas, Defaults.DEFAULT_SOURCE_CODE_CHARSET);
List<ErrorMessage> errorMessages = compiler.validateExpression(expression);


Error: Line 1 column 1 - Line 1 column 2: Syntax error on token 'if'! [Delete this token.]
Error: Line 1 column 1 - Line 1 column 2: Syntax error!


any inputs here ?

Thanks in advance,
Madan.

madan_clover3
Posts: 16
Joined: Tue Jun 10, 2014 10:39 am

Re: Clover script validation

Postby madan_clover3 » Mon Feb 06, 2017 7:58 am

Found some solution.

Code: Select all

String prefix = "//#CTL2 \n function integer transform() {";
String postfix = " return 0; }";
String expressionValidateFunction = prefix + expression + postfix;

ITLCompilerFactory compilerFactory = new TransformFactory.DefaultCompilerFactory();

final ITLCompiler compiler = compilerFactory.createCompiler(getGraph(), recordMetadatas, recordMetadatas, Defaults.DEFAULT_SOURCE_CODE_CHARSET);

List<ErrorMessage> errorMessages = compiler.validate(expressionValidateFunction);


I am not sure if it is right way of doing, but atleast found workable solution.

Please have a look and provide your suggestions.

Thank you,
Madan

vazquezrosariop
Posts: 96
Joined: Mon Feb 29, 2016 5:33 pm

Re: Clover script validation

Postby vazquezrosariop » Tue Feb 07, 2017 6:02 pm

Hi,

I have spoken to our developers and they agree that your last post is the approach they would take to solve this problem. However, please be aware of the fact that the returned Error Message will have shifted positions accordingly. They also recommend that you add a line break at the end of the prefix:

Code: Select all

"//#CTL2 \n function integer transform() {\n"

And at the beginning of the suffix:

Code: Select all

"\n return 0; }"
---
Pedro Vazquez Rosario
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support