Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / October 2007

Tip: Looking for answers? Try searching our database.

Oracle Prepared statement, taking too long for complex calculations.

Thread view: 
Ankit - 04 Oct 2007 03:05 GMT
Hi All,

I have two tables A30DayUsage and A90DayUsage.
Based on these source table i have to populate another table called
resCalculationTable.

Now for the values in resCalculation table are very complex.

for example i have one value called quarter, where i will be doing
SysDate - a field from A30DayUsage table and then convert it to
quarter string. all the fields have very complex date and String
manipulation.

My java program to do this works fine but it takes a lot of time,
around 8 hours to load 57000 records.

Is there a better way of doing this ??
Thanks
Ankit.
Daniel Pitts - 04 Oct 2007 03:50 GMT
> Hi All,
>
[quoted text clipped - 15 lines]
> Thanks
> Ankit.

You might try doing it in batches. It could be that you have a lot of
overhead if you call update or insert for every row and you're not in
a batch.

Alternatively, you might be able to do the work in all SQL.  This
might be a much better approach.
Chris - 04 Oct 2007 04:06 GMT
> Hi All,
>
[quoted text clipped - 13 lines]
>
> Is there a better way of doing this ??

You haven't provided enough detail to suggest a better approach. My only
thought is that 57,000 records might be able to fit into memory easily,
in which case you could read the whole thing into memory quickly, do
your calcs there, and then dump it all back to the database.

My guess is that if this process is currently taking 8 hours with such a
small amount of data, you're probably doing very inefficient database
access. So get it out of the database altogether.

Whether this will work or not depends on your app, of course.

My only other thought: temporary tables are your friend.
Greg R. Broderick - 04 Oct 2007 08:47 GMT
Ankit <shahankit@gmail.com> wrote in news:1191463557.035295.92260@
50g2000hsm.googlegroups.com:

> Hi All,
>
[quoted text clipped - 13 lines]
>
> Is there a better way of doing this ??

Have you analyzed the performance of the query?  EXPLAIN PLAN is your friend,
as are the more sophisticated analysis tools available in Oracle.

How long does the same query, or set of queries take when run outside of
Java, in SQL*Plus?  You need to determine if it is database performance that
is your bottleneck or if it is java performance that is your bottleneck.

If you are doing calculation or processing (in Java)  of the rows returned by
your queries on these two tables, then you should perhaps also look at
splitting up the result sets from the two queries into chunks and processing
multiple chunks in multiple threads in your program.

Without knowing the structure of your database schema (tables and indexes)
and the specifics of your queries, it would be very difficult to advise you
further, though.

Cheers!

Signature

---------------------------------------------------------------------
Greg R. Broderick                  usenet200705@blackholio.dyndns.org

A. Top posters.
Q. What is the most annoying thing on Usenet?
---------------------------------------------------------------------

GArlington - 04 Oct 2007 16:38 GMT
On 4 Oct, 08:47, "Greg R. Broderick"
<usenet200...@blackholio.dyndns.org> wrote:
> Ankit <shahan...@gmail.com> wrote in news:1191463557.035295.92260@
> 50g2000hsm.googlegroups.com:
[quoted text clipped - 42 lines]
> Q. What is the most annoying thing on Usenet?
> ---------------------------------------------------------------------

8 hours = 28800 seconds
That means that you are processing just under 2 records per second, I
would say this is NOT just inefficient DB access, there is something
wrong with your calculations too.
GArlington - 04 Oct 2007 16:39 GMT
> Hi All,
>
[quoted text clipped - 15 lines]
> Thanks
> Ankit.

BTW: did you know that most DBs will let you do a lot of different
calculations (including date/time) in SQL?


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.