Memory leak in running Graph multiple times (???)

Discussion on developing CloverETL engine, transformation components etc.

esegal
Posts: 6
Joined: Wed Jun 18, 2014 9:58 am

Memory leak in running Graph multiple times (???)

Postby esegal » Wed Jun 25, 2014 12:17 pm

Hi,
my application run Clover Graph multiple times - mainly for aggregation purpose.
As I see by JConsole the application consume more and more memory until I get:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at org.jetel.graph.DirectEdge.init(DirectEdge.java:107)
at org.jetel.graph.Edge.init(Edge.java:265)
at org.jetel.graph.Phase.init(Phase.java:147)
at org.jetel.graph.TransformationGraph.init(TransformationGraph.java:463)
at org.jetel.graph.runtime.EngineInitializer.initGraph(EngineInitializer.java:176)

When I set the JVM for enlarge the Direct buffer memory I get the same result - but after longer time.
I took dump file of the application after running a while and I see 41,040 instances of org.jetel.component.DataReader (!!!)
- Well something is malfunction here - I think I use Clover as needed - mybe I'm wrong ?
my code call EngineInitializer.initEngine() only once, and after each graph running call also:
Map<String,Node> nodes= graph.getNodes();
for (Node node : nodes.values())
node.free();
graph.free();

Is any one know about memory-leak in CloverETL?? or am I doing something wrong ??

Thanks in advance !
Segal

kubosj
Posts: 372
Joined: Thu Jan 12, 2012 9:10 am

Re: Memory leak in running Graph multiple times (???)

Postby kubosj » Thu Jun 26, 2014 10:45 am

Jaroslav Kubos
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com


cron