Page 1 of 1

Global static variables

Posted: Fri Feb 03, 2012 8:31 am
by yabel
Hi!
Is there any way to use global static variables of any type and share them between nodes?
For example, in Reformat node1 declare static int VAR1, set the value of VAR1 in this node and then use the value of VAR1 in reformat node2?

I tried to declare class Global { public static int VAR1;} in node1 for this purpose, set Global.VAR1 = 1234567 in transform(...) in node1, but in the node2 the value of Global.VAR1 is always 0.

Any suggestions?

Thanks

Re: Global static variables

Posted: Fri Feb 03, 2012 1:53 pm
by kubosj
Hi yabel,

I would suggest usage of dictionary - see http://www.cloveretl.com/documentation/ ... onary.html

If you need both read and write it, it make sense to divide reading a writing components into 2 phases. Otherwise behavior won't be deterministic.

Your CloverETL Support team
Jarek

Re: Global static variables

Posted: Wed Mar 14, 2012 1:08 am
by jus
Hi Jarek,

How can I use dictionary in the FileURL field of XLSDataWriter? I've read the docs here and here, but couldn't seem to get it to work.
Also, I tried to use the dictionary in the command field of SystemExecute, and it didn't work. Please help.

Thanks!
Jus

Re: Global static variables

Posted: Wed Mar 14, 2012 5:14 am
by jus
Hi Jarek,

How do I use dictionary entries in the FileURL field of XLSDataWriter? I tried the examples from here and here but it didn't work.
Also, how can I use the dictionary in a commnd field of SystemExecute?

Thanks!
Jus

Re: Global static variables

Posted: Thu Mar 15, 2012 11:15 am
by kubosj
Hi jus,

SystemExecute: You can prepare batch script executed by SystemExecute. Then pass arguments through input port to it - so for bash script it will be stdin. Then you need to parse stdin and launch commands you need. Here are some hints: viewtopic.php?f=4&t=5241&p=8231&sid=78de9d6fa4b335ca1a17520c79df67e1#p8231

XLSDataWriter: The only way I know to pass dynamic file name is via "-P:name=value" to graph. There you can use ${name} substitution.

Re: Global static variables

Posted: Thu Mar 15, 2012 5:18 pm
by jus
Hi Jaroslav, thanks for your help.
I can continue working with the SystemExecute example.

Though I'm not sure what to do with the XLSDataWriter, since the filename that I am writing to will be dependent on the output from the ComplexDataReader I'm using. Is there any way at all to do this besides passing a parameter to the graph? Because I don't have the value of the parameter before running it. Also, I can't use a higher level graph that calls RunGraphs because of the bug that was fixed in 3.3.0-M1, because we need this to be in production by the end of the month.

I was thinking of writing to a generic filename, and then run SystemExecute to rename it. But can you think of another way?

Thank you so much!
Jus

Re: Global static variables

Posted: Fri Mar 16, 2012 9:20 am
by kubosj
Hi jus,

generic file name is feasible if there is certainty that graph will not run in more parallel instances. So I would recommend to pass unique file name as graph parameter. Unfortunately there is currently no way how you can generate it by "randomUUID()" and use in both Writer and SystemExec on graph side.

I can't think out better solution. In near, but not yet estimated future, there will be workflows support, which will solve issues you have.

Re: Global static variables

Posted: Thu Mar 22, 2012 10:08 pm
by jus
Hi Jaroslav, thanks for your answer.
My next question is, how should we handle those workflow things meanwhile?
Should we use external scripts? Do it in the server? How do people do it usually? and what's your suggestion?
Thanks!
Jus

Re: Global static variables

Posted: Fri Mar 23, 2012 9:02 am
by kubosj
Hi jus,

usually solution is based on:
* SystemExecute (scripting extra logic)
* RunGraph (run other graph somehow parametrized, possibly in loop)
* HttpConnector (can invoke tasks on Server via simple HTTP API)

All of them have currently some limitation, so it is necessary to use workarounds. That is the why we are preparing workflows support.

Re: Global static variables

Posted: Fri Mar 23, 2012 8:28 pm
by jus
Thanks Jaroslav for your prompt reply! I really appreciate it.

I now have a question about closing file handles.
I am using ComplexDataReader to read in a flat file. After processing is finished, I'm using SystemExecute on the next phase to move the file into an archive or reject folder. But I get this error message:

Code: Select all

ERROR [SYS_EXECUTE1_0] - The process cannot access the file because it is being used by another process.

I get the same problem when using XLSDataWriter and then trying to rename it using SystemExecute on the next phase (in the same graph).

How can I close those file handles from within the same graph? Or is there any other solution you can think of?

Thanks a bunch!
Jus

Re: Global static variables

Posted: Tue Mar 27, 2012 9:56 am
by kubosj
Hi jus,

ComplexDataReader closes opened file when graph is finished. That is not correct behavior and it will be fixed by: https://bug.javlin.eu/browse/CL-2229

For now, only workaround is to use 2 graphs - first, which will run second and then move/delete file. In second will be complex data reader.

Re: Global static variables

Posted: Tue Mar 27, 2012 6:01 pm
by jus
I guess it's the same problem with the XlsDataWriter? Will it also be fixed in 3.3.0-M2?
Thanks again Jaroslav!
Jus

Re: Global static variables

Posted: Wed Mar 28, 2012 8:25 am
by kubosj
Hi jus,

sorry I missed that. There is new issue https://bug.javlin.eu/browse/CL-2235

I would not expect it will be solved in 3.3.0-M2. The same may happen to CL-2229, although it is currently planned for M2.