Closing DB connection

Support/help with CloverETL implementation problems

scl84
Posts: 11
Joined: Mon Jun 30, 2014 9:25 am

Closing DB connection

Postby scl84 » Mon Oct 27, 2014 2:57 pm

Hi,

I am using DBInputTable with Generic connection to Hybris vjdbc. But for some reason we are getting the following exception:
"Can't connect to DB: java.sql.Exception:java.io.IOException: Server Returned HTTP Response code 500 for url: http://.../virtualjdbc/service/flexMode=true"

The connection is working fine and after some time, we are getting this above exception. After restarting the Hybris server, the connection is happening again properly.

Could you please let us know how we can close the connection in Clover after triggering the queries in DBInputTable?

Thanks and regards,
SCL

imriskal
Posts: 386
Joined: Wed Aug 15, 2012 8:18 am

Re: Closing DB connection

Postby imriskal » Thu Oct 30, 2014 2:54 pm

Hello,

Your DB server should log the error with it's stack trace into some error log file. Could you please try to find it and share with me? And could you please externalize your db connection and share it as well? Without your password, of course.

Thanks.
---
Lubos Imriska
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

scl84
Posts: 11
Joined: Mon Jun 30, 2014 9:25 am

Re: Closing DB connection

Postby scl84 » Fri Oct 31, 2014 3:45 pm

Hi Lubos,

Unfortunately, the DB server is at client side and we don't have access to that logs.
Also due to confidentiality issue, we can not share the client's IP address.

user=cloveruser
dbDriver=de.hybris.vjdbc.VirtualDriver
jdbcSpecific=GENERIC
name=HybrisConnection
dbURL=jdbc\:hybris\:flexiblesearch\:http\://....\:9001/virtualjdbc/service

Regards,
SCL

imriskal
Posts: 386
Joined: Wed Aug 15, 2012 8:18 am

Re: Closing DB connection

Postby imriskal » Wed Nov 05, 2014 3:27 pm

Thanks for the connection details. I understand the confidentiality issue, of course. However, I miss information about threadSafeConnection flag being set to either true or false. It is typically a part of this cfg file and it is essential piece of information because it directly influences the overall behavior.

If the connection is thread safe (threadSafeConnection=true), each component in a graph uses it's own db connection instance which is closed after the component is finished. There is a theoretical chance that if you use too many components working with DB in one graph phase, they all get their own connection and connection limit set on DB server's side prevents you from creating more connections. If another component attempts to create one more, an error is thrown.

If the connection is thread unsafe (threadSafeConnection=false), all components share exactly one instance of connection. This type of connection is closed at the very end of the graph run. This on the other hand brings a potential risk of error due to timeout. If for example DBInputTable reads something right after the graph starts, then you process the data for two hours and then you try to save the data, a timeout error may appear.

Regarding the error code 500, we are unfortunately unable to solve it without the log. Hopefully something from my explanation guides you to the issue location even without the log file.

Regards,
---
Lubos Imriska
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

scl84
Posts: 11
Joined: Mon Jun 30, 2014 9:25 am

Re: Closing DB connection

Postby scl84 » Wed Nov 26, 2014 9:28 am

Thanks Lubos for your reply.

Could you please let me know what will be the default value of threadSafeConnection if we are not setting it?

Thanks and regards,
SCL

imriskal
Posts: 386
Joined: Wed Aug 15, 2012 8:18 am

Re: Closing DB connection

Postby imriskal » Wed Nov 26, 2014 10:17 am

I think that threadSafeConnection option is by default set to true.

Regards,
---
Lubos Imriska
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support


cron