> I am currently wondering what is the default behaviour of jdbc
> database drivers with regard to if they return a ResultSet that is
[quoted text clipped - 16 lines]
> is ignored and everything is fetched, which makes Hypersonic not
> usable in situations like that.
Sorry. Most drivers do a pre-read of 1-N rows, and repeat when needed,
but won't discard rows already read. The setFetchSize() is just a hint at
how many rows to pre-fetch at a time. It has nothing to do with how many
rows in total to fetch or keep. Some drivers may discard row data once
that row has been passed, for result sets that are forward-only. Some don't.
You are suffering the fundamental problem of using the DBMS as a dumb
file system. The number one performance tenet of DBMS application architecture
is to never ship vast quantities of raw data from the DBMS to a client for
processing. The idea is to use stored procedures to do the bulk processing
where the data is.
If you can't do this, for whatever reason, you might consider making your
SQL fancier, to fetch the data in batches you define (one batch per result set),
so you can control whether a result set stays in memory etc.
Good luck,
Joe Weinstein at BEA
Halldór Ísak Gylfason - 28 Aug 2003 10:41 GMT
One of the reasons I was asking this is because I have been
experimenting with the JMS Message Queue implementation of JBoss. I
have found out that that implementation is not good at all, and
unusable with high message loads. To be fair however JBoss people
acknowledge that (at least some of them) and they are rewriting it for
4.0
One of the things that the JBoss messaging subsystem does when
booting up, is the following query to the database (when using a
PersitanceManager that stores messages in the database):
"SELECT MESSAGEID, MESSAGEBLOB FROM JMS_MESSAGES WHERE DESTINATION=?"
In other words it does a query for all messages on a single queue. If
that queue is big, there will obviously be memory problems like I was
pointing out in my message. The code that does the queue restoring in
JBoss in case you're interested is in the
org.jboss.mq.pm.jdbc2.PersistenceManager and the function is
restoreQueue:
/////////////////////////////////////////////////////////////////////////////////
//
// Message Recovery
//
/////////////////////////////////////////////////////////////////////////////////
synchronized public void restoreQueue(JMSDestination jmsDest,
SpyDestination dest) throws javax.jms.JMSException
{
if (jmsDest == null)
throw new IllegalArgumentException("Must supply non null
JMSDestination to restoreQueue");
if (dest == null)
throw new IllegalArgumentException("Must supply non null
SpyDestination to restoreQueue");
TransactionManagerStrategy tms = new
TransactionManagerStrategy();
tms.startTX();
Connection c = null;
PreparedStatement stmt = null;
ResultSet rs = null;
boolean threadWasInterrupted =
Thread.currentThread().interrupted();
try
{
c = this.getConnection();
stmt = c.prepareStatement(SELECT_MESSAGES_IN_DEST);
stmt.setString(1, dest.toString());
rs = stmt.executeQuery();
int counter=0;
while (rs.next())
{
SpyMessage message = extractMessage(rs);
// The durable subscription is not serialized
if (dest instanceof SpyTopic)
message.header.durableSubscriberID =
((SpyTopic)dest).getDurableSubscriptionID();
jmsDest.restoreMessage(message);
counter++;
}
log.debug("Restored "+counter+" message(s) to: "+dest);
}
catch (IOException e)
{
tms.setRollbackOnly();
throw new SpyJMSException("Could not restore messages to
destination : " + dest.toString(), e);
}
catch (SQLException e)
{
tms.setRollbackOnly();
throw new SpyJMSException("Could not restore messages to
destination : " + dest.toString(), e);
}
finally
{
try
{
rs.close();
}
catch (Throwable ignore)
{
}
try
{
stmt.close();
}
catch (Throwable ignore)
{
}
try
{
c.close();
}
catch (Throwable ignore)
{
}
tms.endTX();
// Restore the interrupted state of the thread
if( threadWasInterrupted )
Thread.currentThread().interrupt();
}
}