Using log ports

Discussion on developing CloverETL engine, transformation components etc.

fschnack
Posts: 10
Joined: Mon Nov 08, 2010 7:27 pm

Using log ports

Postby fschnack » Fri Sep 16, 2011 8:46 pm

Hello!

I am embedding Clover 3.0.2 into a Java application. In my custom node, I wanted to use the logport that is defined in the Node class to write rejected data. The flow execution finishes successfully, but if I try to enable JMX monitoring I start getting NPE on PhaseTrackingDetail line 207.

After some digging, I've found that the problem is related to this class' nodesDetails attribute. This array is initialized with the the same size as the list of graph nodes (see line 63). When it is being populated, the TransformationGraphAnalyzer.nodesTopologicalSorting method is called to sort the nodes list. The problem is that when this method crawls through the nodes, it gets all output ports for all nodes, but it's not getting the log port. The result is that the PhaseTrackingDetail.nodesDetails attribute ends with some null values.

Would it be possible to fix it in a future release? It's a quick fix, it's only a matter of adding the log port to TransformationGraphAnalyzer.findSuccessiveNodes() method.

Another (related) question: shouldn't Node.closeAllOutputPorts() method call logPort.eof()?

Thanks!!

avackova
Posts: 841
Joined: Fri Jul 20, 2007 9:28 am

Re: Using log ports

Postby avackova » Tue Sep 20, 2011 10:26 am

Hello,
log port is not used in CloverETL. It was designed with the Node class in the very beggining of CloverETL, but has been never used and will be removed in 3.2 version. Please use just another output port like it is used in DataReader component.
Agata Vackova
Javlin a.s.
agata.vackova@javlin.eu

fschnack
Posts: 10
Joined: Mon Nov 08, 2010 7:27 pm

Re: Using log ports

Postby fschnack » Tue Sep 20, 2011 12:28 pm

Oh, really, it was discontinued? I think the documentation still refers to it on the wiki...

The problem with components such as data reader is that in order to implement an optional log port this port is fixed as the second (index = 1) and you can't add extra ports to it. Is there a pattern on how to implement this feature in Clover?

Thanks!

avackova
Posts: 841
Joined: Fri Jul 20, 2007 9:28 am

Re: Using log ports

Postby avackova » Tue Sep 20, 2011 1:22 pm

Hello,
we usually do it with another attribute such as useLogPort=true/false and if this attribute is set to true, the last connected output port is treated as logging port.
Agata Vackova

Javlin a.s.

agata.vackova@javlin.eu

fschnack
Posts: 10
Joined: Mon Nov 08, 2010 7:27 pm

Re: Using log ports

Postby fschnack » Thu Sep 22, 2011 1:23 pm

Seems much more complicated... but thanks.


cron