Java Forum / Databases / April 2007
DBCP + Weblogic = Runaway Leaks. Please help
kush.desai@gmail.com - 16 Apr 2007 01:09 GMT Hi everyone,
I'm facing a serious issue with connection pooling and I'd appreciate the help/advice of minds way more experienced/smarter than mine in tacking the problem. I was asked to work on a web project recently, that I found used no connection pooling. It was opening 4-5 connections per servlet call / JSP. So I decided to use DBCP to implement pooling. So far so good. My dev environment uses Tomcat whereas my UAT/Prod is Weblogic.
The app. typically connect to various datasources and the list can change at runtime. So instead of defining my datasources in an XML file, I dynamically bind them into JNDI upon start-up and rebind them whenever the datasource definition is changed (I have webscreens that allow authorized users to add/remove/edit datasources).
This works v nicely in Tomcat. I stress tested with JMeter to ensure I didn't have connection leaks and that the pool wasn't growing out of size. I move this into the Weblogic UAT environment. Its a clustered env. with 2 servers. Right off the bat, I notice its creating 18-odd INACTIVE connections to my main Oracle DB in 1 node and about 3 in the other. I have set maxActive=20, maxIdle=7, maxWait=15000, logAbandoned=true, removeAbandonedTimeout=600, removeAbandoned=true. So its strange that's creating so many connections.
As the application runs, the number of connections goes up to 60 in 1 node and 30 in the other. Then, a few minutes later, all connections get released and both nodes are back to holding 2-3 connections. I never saw anything like this in my dev Tomcat env and am stumped as to what's causing this.
Does DBCP not play well with Weblogic? Why are my parameters being respected in Tomcat and yet tossed aside in Weblogic? At one point, I had 400 inactive connections from one of the UAT servers to the database, which locked everybody else out. I've checked everybit of my code and I'm not leaking connections.
How do I fix this problem? Any ideas/suggestions folks?
Thanks, Kush
joeNOSPAM@BEA.com - 16 Apr 2007 02:20 GMT On Apr 15, 5:09 pm, kush.de...@gmail.com wrote:
> Hi everyone, > [quoted text clipped - 37 lines] > Thanks, > Kush Hi. What version of WebLogic? Here is an example of best-practices JDBC style in a WebLogic environment. You can also show me your pool definition... What parameters do you see thrown away? Joe Weinstein at BEA Systems
Here is the ideal standard for safe WebLogic JDBC. If you can adopt this, many such problems will go away. Pooling is fast, and is best used in a quick per-invoke fashion:
/* This is how you should make any of your top-level methods * that will do JDBC work for any of your user invokes. */ public void myTopLevelJDBCMethod() { Connection c = null; // ALL JDBC objects should be *method* // level objects to ensure thread-safety // and prevent connection leaking. // Define the connection object *in this // method*, before the JDBC 'try' block. try { // This is *the* JDBC try block for this method. Do // *all* the JDBC for this method in this block.
// Get the connection *directly from our DataSource* // in the try block. Do *not* get it from any method // that has kept a connection and is sharing it for // repeated use.
c = myDataSource.getConnection();
... do *all* the JDBC for this method in the scope of this try block... ... you *can* pass the connection or sub-objects to sub-methods ... but none of these methods must expect to keep or use the ... objects they receive after their method call completes...
(eg)
DoSomethingFancyWith(c);
// Use Prepared/Callable Statements. They are faster usually, // Especially because we cache them transparently with the pool. PreparedStatement p = c.prepareStatement(...); ResultSet rs = p.executeQuery(); ProcessResult(myrs);
// Close JDBC objects in the proper order: resultset, then statement, then connection
rs.close(); // always close result sets ASAP at the level they were created p.close(); // always close statements ASAP at the level they were created
...
// When the JDBC is finished in the try-block, close the con: c.close(); // always close connection ASAP in the same method // and block that created/obtained it. c = null; // set the con to null so the finally block below // knows it's been taken care of. } catch (Exception e ) { ... do whatever, according to your needs... you do not have to ... have a catch block if you don't want it... } finally { // Always have this finally block. A finally block is *crucial* // to ensure the connection is closed and returned to the pool, // (not leaked). // failsafe: Do every individual thing you want to do in the // finally block in it's own try block-catch-ignore so everything // is attempted.
try {if (c != null) c.close();} catch (Exception ignore){} } }
kush.desai@gmail.com - 16 Apr 2007 03:19 GMT > }- Hide quoted text - > > - Show quoted text -- Hide quoted text - > > - Show quoted text - Hi Joe,
Here's how I bind the datasource in JNDI
Reference ref = new Reference("javax.sql.DataSource","org.apache.commons.dbcp.BasicDataSourceFactory", null); ref.add(new StringRefAddr("maxIdle", "" + maxIdle)); ref.add(new StringRefAddr("maxActive", "" + maxActive)); ref.add(new StringRefAddr("maxWait", "" + maxWait)); ref.add(new StringRefAddr("driverClassName","oracle.jdbc.OracleDriver")); ref.add(new StringRefAddr("removeAbandoned", "Y".equals(removeAbandoned) ? "true" : "false")); ref.add(new StringRefAddr("logAbandoned", "Y".equals(logAbandoned) ? "true" : "false")); ref.add(new StringRefAddr("username", user)); ref.add(new StringRefAddr("password", pass)); ref.add(new StringRefAddr("url", connString));
try { ic.rebind("<myapp>:comp/env/jdbc/" + jndiName, ref); } catch (NamingException e) { MoleLogger.error(e.getMessage(), this.getClass().getName()); e.printStackTrace(); }
System.out.println("Done creating/updating binding for :" + jndiName);
And I use it the way you mentioned. The DB connection in question that's having problem is configured as follows
try{ bindDataSource(ic, "oracle", //database type "jdbc:oracle:thin:@<hostname>:1631:<SID>",//url "<USER>", //user "<PASS>", //pass "<jndi>", //jndiName 5, //maxIdle 30, //maxActive 15000, //maxWait //600, // removeAbandonedTimeout "false", //removeAbandoned "false" // logAbandoned ); }catch(NamingException e){ // ABORT Logger.error("Fatal Error: Could not bind", this.getClass().getName()); }
As you can see maxActive = 30, I don't see why then I have 40-odd inactive connections open...all showing as INACTIVE. Any help you can provide is appreciated.
Oddly enough, I dont think I'm having this problem with sybase datasources.
joeNOSPAM@BEA.com - 16 Apr 2007 15:33 GMT On Apr 15, 7:19 pm, kush.de...@gmail.com wrote:
> > }- Hide quoted text - > [quoted text clipped - 61 lines] > Oddly enough, I dont think I'm having this problem with sybase > datasources. Ok, so in fact you are not using WebLogc pooling at all. I highly recommend it. So far, you haven't shown anything that will affect or be affected by WebLogic, except that the JNDI tree will not constitute one pool if the WebLogic server is a cluster of more than one JVM. Each server JVM will probably have a local duplicate of your non-weblogic pool. I will be happy to help you with configuring and using WebLogic pools, if the console isn't easy enough... Joe
kush.desai@gmail.com - 16 Apr 2007 16:57 GMT On Apr 16, 10:33 am, "joeNOS...@BEA.com" <joe.weinst...@gmail.com> wrote:
> On Apr 15, 7:19 pm, kush.de...@gmail.com wrote: > [quoted text clipped - 75 lines] > > - Show quoted text - Hi Joe,
You are correct. I'm not using Weblogic's functionality at all. It is simply a Servlet container as far as the app goes, which ofcourse is a shame. However, I've been parachuted to manage this app so I dont know the rationale behind a lot of decisions.
Now the interesting bit I have confirmed is that the leak is only happening against the Oracle DB, which I use for administrative tasks like user/group management. What I did notice is that the code closes the recordset, connection etc in the finally block and there is a return in the try block. Debugging in tomcat, I saw that it was executing the finally block before doing the return in the try. I don't know if that could be a reason? It just surprises me that leak doesnt happen on Tomcat, only on Weblogic.
Also, you are correct about duplication of the JNDI tree and the pool. Whenever I change the JNDI entry through a servlet, changes aren't being propogated to the JNDI tree on the other node, which is a bit problematic.
I wanted to be able to add my datasources dynamnically instead of defining them in an admin console and I needed to do this in a vendor agnostic way. I know Weblogic has an API for manipulating JNDI and connection pools but using that would tie me to weblogic which is a no- no since since my dev env. is Tomcat. I'm ofcourse happy to recast this decision, if you can tell me its do-able differently. Thanks for your inputs.
joeNOSPAM@BEA.com - 16 Apr 2007 18:12 GMT On Apr 16, 8:57 am, kush.de...@gmail.com wrote:
> On Apr 16, 10:33 am, "joeNOS...@BEA.com" <joe.weinst...@gmail.com> > wrote: [quoted text clipped - 107 lines] > this decision, if you can tell me its do-able differently. Thanks for > your inputs. Ok, so I would now refer you to weblogic support, or our newsgroups. I'm a JDBC wonk, and it's now not a JDBC issue really. Support will get you better help about general weblogic stuff like the JNDI questions. Joe
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|