Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / Databases / June 2004

Tip: Looking for answers? Try searching our database.

Achieving very smooth scrolling of large data sets?

Thread view: 
John Doe - 24 Jun 2004 19:51 GMT
Hello all,

I'm looking to smoothly scroll large data sets. I'm already using
an AbstractTableModel that caches blocks of a few hundred records
at a time (plenty to fit a few displays full of data).

My getValueAt looks as follows:
--
   public Object getValueAt(int rowIndex, int columnIndex) {
       if (rowIndex>this.maxrow) {
           load_block(rowIndex);  // causes 0.3 second delay
       } else {
           if (rowIndex<this.minrow) {
               load_block(rowIndex); // 0.3 second delay here too
           }
       }
       int relrow=rowIndex-this.minrow;
       Vector record=(Vector)this.blockdata.get(relrow);
       return record.get(columnIndex);
   }
--

The load_block does decent caching; it only loads records that
are not currently in memory and keeps memory usage modest.

For line by line scrolling this works smooth enough (scrolling 10
lines per second) however when dragging the scrollbar or using PgUp
and PgDn, the system won't keep up because load_block is called more
often than the database can return the data. As a result things feel
slightly unresponsive- the scrollbar thumb lags behind the mouse.

How would I be able to optimize this so that the system will respond
faster (or simply feel more responsive)?

I'm thinking of:

- Only doing a database lookup when the user stops dragging the mouse,
 however this does not protect from PgUp/PgDn;

- skipping database lookups altogether if more scroll actions are
still
 pending, is it possible to see this somehow? This would also save
 bandwidth when the database is on a network;

- Doing the lookups in the background, any thoughts on this?

Opinions and suggestions will be much appreciated. Extra bonus points
for snippets of helpful code!

Regards,

John

(Disclaimer: I'm a quite experienced programmer but relatively
new to Java.)
Roedy Green - 24 Jun 2004 21:51 GMT
>How would I be able to optimize this so that the system will respond
>faster (or simply feel more responsive)?

You could do some preemptive caching on a low priority thread, loading
your cache with look ahead in both directions when you have nothing
better to do, and pruning your cache.

See http://mindprod.com/projects/htmlglossarypresenter.html

Another thought is you might want a custom layout manager that uses
arithmetic to do the mapping from a co-ordinate to a corresponding
component rather than doing an iterative inside test on each component
if mouse interaction is slow too.

you also could go very low level. See
http://mindprod.com/projects/javapresenter.html and look at the source
for PrettyCanvas, to handle the rendering side, regenerating just the
clipregion from raw data each time painting on a giant virtual
scrolling canvas.

Oddly a very similar problem is one the group I am working with is
also tackling.

Perhaps we need a generic solution to how to scroll though giant
lists. JTable's TableModels come closest to a general solution.

Signature

Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.

John Doe - 25 Jun 2004 17:37 GMT
Roedy Green <look-on@mindprod.com.invalid> wrote in message news:

> You could do some preemptive caching on a low priority thread, loading
> your cache with look ahead in both directions when you have nothing
> better to do, and pruning your cache.

That's definitely a possibility--probably one of the easier ones to
implement. However when the database is stored on a network server,
this increases network load substantially, especially if the database
is accessed by several client applications. In such a case, preventing
unneeded lookups would be a bit cleaner situation.

I'm OK with things taking a bit after releasing the mouse button or
PgUp/PgDn  key, the 0.3 seconds is perfectly acceptible then (showing
an hourglass cursor for that bit of time is not going to offend many
people).

> Another thought is you might want a custom layout manager that uses
> arithmetic to do the mapping from a co-ordinate to a corresponding
> component rather than doing an iterative inside test on each component
> if mouse interaction is slow too.

Below is what I perceived as the meaning of what you said above. Are
you saying to exploit the relative low sensitivity of scrolling by
scrollbar (a screen rarely being more than 1024 pixels high)?

-----------------------
- For instance the screen fits R records at any time, and
- the scrollbar is P pixels high, then we need to cache at
- most P blocks of R records to smoothly scroll through
- the entire database by mouse, regardless of the size of
- the database.
------------------------

That could in fact be an interesting way to go about it- most
computers nowadays will be able to handle, say, holding 60000 records
in RAM (60000 being 60 records per screen*the hight of the scrollbar
in pixels)

However, resizing the window would require repopulating the cache,
unless the added lack of precision is acceptible (then again if we
want precision,
we shouldn't be using scrollbars).

[Reality check: If we display text, it is difficult showing more than
256 chars horizontally on a screen, 16 mb of RAM should do the trick
(plus data structure overhead, of course. But unless we want to do
fancy stuff such as displaying full motion video streams in our table
while scrolling, we should be all right).]

I guess 60000 records can be considered 'modest' memory requirements,
however it can still be quite a load on the network. Even for a
gazillion record database, I wouldn't want to load many megabytes of
data before being able to scroll- that's where the "showing only 1
record while scrolling" would come in.

But then again, maybe I completely misunderstood what you meant.

This still woudn't fully solve the scrolling by PgUp/PgDn but I
suppose it is possible to consult the keyboard buffer and prevent a
number of lookups. In addition, navigation by PgUp/PgDn is a bit more
predictable than mouse movements- preemptive caching on a low priority
thread will work wonderfully there.

Best Regards,

John
Roedy Green - 25 Jun 2004 17:50 GMT
>- For instance the screen fits R records at any time, and
>- the scrollbar is P pixels high, then we need to cache at
>- most P blocks of R records to smoothly scroll through
>- the entire database by mouse, regardless of the size of
>- the database.

No, that's not what I was saying, but what you suggest is an
interesting idea.  When people jump around with the scroll bar, you
need only get them approximately to where they asked to go, then let
them fine scroll from there. So you only need cache a selection of
jumping in points, e.g. the start of each letter of the alphabet, and
take them to the nearest one in your cache, and then start frantically
premptively back/forward caching from there.

What I was talking about is the slow response to mousing that can
happen if you get a very large number of components in a complex
layout.  The layout manager on receiving a mouse event has to decide
which component it belongs to. The generic way of solving this is to
ask each component in turn, "is this yours?"  (speaking very loosely).

I was suggesting speeding this process up my taking advantage of any
regularity in the layout.  E.g. if it were a grid of 100 pixel squares
10 across you could compute the row as y/100 and column as x/100. Then
you could compute the component index as row*10 + col which can be
computed in o(0) time.

Signature

Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.