Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / Databases / October 2007

Tip: Looking for answers? Try searching our database.

(Mis)use of transactions

Thread view: 
RedGrittyBrick - 09 Aug 2007 21:08 GMT
It's a bit quiet in comp.lang.java.databases so I hope no-one minds a
database question that isn't particularly related to Java or JDBC
(though that's what I am actually using).

I have an app, several bits of which allow a user to maintain simple
tabular reference data that is stored in database tables.

When the user elects to maintain one of these tables, I pop up a new
window that contains a JPanel containing a JTable, an "OK" button and a
"Cancel" button.

In the JPanel's constructor I get a java.sql.Connection instance and
thus open a connection to the DBMS.

Whilst the user edits the data in the JTable I am firing off the
appropriate executeUpdate methods to send updates to the DBMS.

When the user clicks OK I invoke commit() and close() on the connection.

If the user clicks Cancel I invoke rollback() and close() on the connection.

 -- * --

However if the user does a few changes and then wanders off for lunch,
the database table is left in a locked state that obviously is likely to
cause problems for other users (or other threads in the user's app).

The only improvements I can think of are to track the changes inside the
app and only do getConnection(), executeUpdate(), close() in the "OK"
button's ActionListener.

Keeping a log of changes or comparing before and after versions of a
List<foo> in a table model seems a bit tedious.

Maybe I should do the getConnection(), executeUpdate(), close() for each
little change to a table row and remove the "Cancel" button?

What do you do?
Doug Morse - 09 Aug 2007 22:18 GMT
Hi RedGrittyBrick,

I never, ever leave control of a DBMS transaction in the hands of user, for
just the reasons you point out.  Indeed, I do everything possible to
minimize the time between starting a transaction and then either committing
it or rolling it back.

Thus, I suggest exactly what you proposed: Cache locally all the edits the
user is making and then only start the transaction and commit it once the
user says "OK".  Of course, there's always the possibility that another user
might be editing the data at the same time.  So, if it's reasonable enough
(e.g., the entire reference table isn't too large), after the user has
pressed OK but before I start the transaction, I'll requery the table and
compare it to what the user is seeing (i.e., the local cache).  If there has
been a change, I notify the user that the database has changed since they
started editing and prompt them if they'd like to review what's changed
before committing their work.  If the reference table does not have a small
number of rows, then I'll at least notify them of rows that others have
changed that the local user is also trying to change.  This user feedback
becomes especially important if another user has changed the primary key
(P.K.) of a row that the local user also edited or if another user has
deleted one of the rows the local user has edited.

Tracking the changes that the user has made should be pretty easy, if it's a
straightforward reference / lookup table.  If it's just a "Code" and
"Description" table, then I think a Nx4 array/matrix should suffice.  Column
1: Type (I,D,U for insert, delete, update, respectively), column 2: old
primary key value, column 3: new primary key value (just copy old p.k. value
if user didn't change), column 4: new description.  The N rows are the
actions taken by the user, in order.  Here's an example of what such an
in-memory change matrix might look like.  There are just two columns, Code
and Description, and Code is the P.K.  The example is taken from a reference
table of psychological "presenting problems" occurring at a crisis
intervention:

    Old   New   New                            My
Type Code  Code  Description                    Comments
---- ----- ----- ------------------------------ ------------------------------
 I  ATOT  ATOT  Atypical Behavior - Other      User added new code/desc.
 I  ATUB  ATUB  Unusual / Bizarre Behavior     ""
 I  ATVO  ATVO  Violent / Out of Control       ""
 D  ME00  ME00  Medical / Somatic              User deleted code/desc.
 D  MR00  MR00  Mental Retardation             ""
 U  PSDE  PSDE  Depression                     User fixed misspell: Depresion
 U  PSDA  PSDL  Delusional                     User changed code (P.K.)
 U  PSPA  PSPN  Paranoria, NOS                 User changed both code/desc.

A simple in-memory table like this also makes it very easy to implement an
Undo function for the user: To undo, just pop off (the bottom) of the stack.
Actually, you should probably add a fifth column: Old Description.  This
would allow you to check and see if another user has changed the Description
column while the local user was editing, as well as allow your Undo function
to undo edits to the Description column.

All pretty easy, I'd think, and shouldn't require too many LOC at all.

A variant on all this would be to copy of the reference table into a
temporary table and let the user make all of her edits on the temporary
table.  This variant thus would allow you to leave the transaction open as
long as you like w/o affecting other users.  Unlike the in-memory option,
though, it consumes server-side resources; plus, transactions usually
timeout eventually.  The upside is that if your local client crashes, and
you're using the in-memory approach, the edits are lost, but temp tables can
often be configured to persist until explicitly deleted, so the local
client, once re-started, could reconnect to the temp table and recover the
lost edits.  That said, though, I'd still recommend against allocating and
holding open server-side resources when it's not essential.  If you wanted
this level of crash recovery, you could just as well periodically write out
your in-memory changes table (example above) to a local disk file for
recovering after a local client crash.

Finally, you already mentioned the third option I can think of, namely
committing each user-specified change as you go.  I would NOT close and
re-open the connection, though, as you offered.  It's not necessary, and
will likely be incredibly slow, because creating and tearing down DB
connections is generally resource-intensive and latency-bound due to the
lock-step messages that have to go back and forth over the wire.  Just leave
the connection open as long as your UI dialog is open, but start, execute,
and commit the transaction all at once.

Hope this helps.

Doug

>  It's a bit quiet in comp.lang.java.databases so I hope no-one minds a
>  database question that isn't particularly related to Java or JDBC
[quoted text clipped - 34 lines]
>
>  What do you do?
Lew - 10 Aug 2007 02:18 GMT
> I never, ever leave control of a DBMS transaction in the hands of user, for
> just the reasons you point out.  Indeed, I do everything possible to
[quoted text clipped - 4 lines]
> user is making and then only start the transaction and commit it once the
> user says "OK".

Yes, that's the way.
> This user feedback
> becomes especially important if another user has changed the primary key
> (P.K.) of a row that the local user also edited or if another user has
> deleted one of the rows the local user has edited.

I don't suggest that you permit an "update" of PK information - really, that's
a delete of the old row and an insert of a new one.  The whole idea of a PK is
that it identifies the row, and in the object model, the object under
consideration.  By that interpretation, a different PK represents a different
object, not the same one with a new key.

There are CASCADE implications also - how do you revise all foreign-key (FK)
references to the object if the PK changes?

It's better to fix the PK as uneditable when in "edit" mode on an existing record.

You also mention "one of the rows the user has edited".  One way to minimize
conflict is to have a user edit only one logical item (e.g., one customer, one
order, one event) at a time.  This reduces the likelihood in most applications
that more than one user will need to edit (delete, ...) the same rows at the
same time.

There is a tension between reserving resources for the entire time one needs
them (e.g., locking the affected rows) and the need to release resources ASAP.

Signature

Lew

Thomas Hawtin - 09 Aug 2007 23:25 GMT
> In the JPanel's constructor I get a java.sql.Connection instance and
> thus open a connection to the DBMS.

Sounds like very confused code to me. Is it a UI class or a database
class? In any case, I'd avoid subclassing unnecessarily.

> However if the user does a few changes and then wanders off for lunch,
> the database table is left in a locked state that obviously is likely to
> cause problems for other users (or other threads in the user's app).

And quite possibly the connection times out causing all of the users
data to be discarded. I'm guessing most people don't like it when that
happens.

> Maybe I should do the getConnection(), executeUpdate(), close() for each
> little change to a table row and remove the "Cancel" button?

No particular need to close each time, although it is worth being able
to reopen then connection if it fails.

From a usability perspective I like applications that are as immediate
as possible. That means no OK/Cancel buttons. However, if you want
locking you can still implement your own locking system on top of the
database.

There is a problem in that updates that other actors make while the
window is open wont be automatically shown. You can do that relatively
efficiently by putting triggers on the main table that copy updates into
shadow tables.

Tom Hawtin
RedGrittyBrick - 10 Aug 2007 10:52 GMT
>> In the JPanel's constructor I get a java.sql.Connection instance and
>> thus open a connection to the DBMS.
>
> Sounds like very confused code to me. Is it a UI class or a database
> class?

I simplified my description somewhat. In reality the JDBC methods are in
a "model" class. I instantiate the model in the constructor of the "view".

I do find deciding what functionality should go in what classes to be
confusing. I think its something people get better at with practice.

Maybe I should do
  FooModel model = new FooModel();
  FooView view = new FooView(model);
instead of
  FooView view = new FooView();
  ...
  class FooView {
    FooModel model = new FooModel();
    ...
  }
Though I can't see a significant benefit. I probably did it the latter
way because I associate the model with the JTable more than with the
container. So my instantiation of the model is close to the
instantiation of the JTable. More to ponder.

> In any case, I'd avoid subclassing unnecessarily.

I find extending JPanel is a convenient approach. I'm aware of the
"composition vs inheritance" debate, so maybe I'll ponder this further.

<further good points snipped for brevity>

> There is a problem in that updates that other actors make while the
> window is open wont be automatically shown. You can do that relatively
> efficiently by putting triggers on the main table that copy updates into
> shadow tables.

I'm not really familiar with that approach, anyone have pointers to
further reading on this (preferably online) or relevant examples?

P.S. Thanks to Doug & Lew too, your good advice is much appreciated.
Doug Morse - 10 Aug 2007 13:41 GMT
I'm not actually sure what he means by "shadow tables", but I presume that
he means either (a) an identical reference table with recently inserted or
deleted rows (he views updates as delete's followed up inserts, which seems
sensible to me -- so by his reasoning "no updates"), and perhaps an added
timestamp of the (prior) transaction time(s), or (b) a table with just prior
transaction time, old P.K. for the affected row in the Reference table, new
P.K. for the affected row, and a char(1) column set to either 'I' or 'D' for
insert or delete, respectively.  With either kind of shadow table, your app
could periodically query it for all committed transactions since the
datetime you originally cached the Reference table contents locally.  Then,
just see if any of the P.K.'s returned (old or new) match your cached
P.K.'s.  If so, then you know that another user has affected a row you've
been caching locally and thus can notify the user and/or take action
accordingly.  Again, actually pretty simple stuff -- the LOCs to do this is
probably not terribly more text than this paragraph or so.

As far as triggers, you use them to manage the contents of the shadow table.
With every committed transaction to the Reference table, your triggers will
add the appropriate entries to the shadow table.  Your app and triggers will
also need to "checkpoint" the shadow table in some way -- either with
special, reserved values in the shadow table or with a separate shadow
checkpoint table.  Basically, you just need your app to record somewhere in
the DBMS that it cached the reference table at a certain time, and then
delete that entry when it's done locally caching.  Your triggers then use
this information to clean out the shadow table so that it doesn't grow
endlessly.  Basically, the triggers just query for the oldest app-cache
datetime, and delete any shadow table entries OLDER than that date, since in
principle no app instance needs to know about changes to the Reference table
that are older than that date (i.e., when the oldest obtained its cache).
Finally, your triggers should be coded to periodically delete all entries in
the shadow table after some maximum timeout period of, say, 24 hours. This
is needed to handle the case when an app instance crashes and in so doing
"strands" its checkpoint in the shadow table or the separate shadow
checkpoint table.

As far as managing the checkpoints, using the shadow table itself is the
most straightforward, provided there's some reserved column value you can
use to indicate a check point entry (or just add a bool column type named
"checkpoint").  The you could use, say, the Description column to indicate
some sort of app-instance ID, if you wanted to.  As long as you store the
*precise* datetime in the shadow table and keep up with that same *precise*
datetime in your app, then an app-instance ID isn't need, as each app (or
app dialog box, if you allow multiple dialogs for editing in the same app
instance) is identified by its unique datetime that it cached the Reference
Table.  Finally, if you want to use a separate shadow checkpoint table, you
need to accomplish the same tasks but against that table instead -- the
logic of it all is identical either way you go.

If you let me know what DBMS you're using, I can give some hints / pointers
as to what triggers you would need to create.

Finally, I "lied" a bit to keep things simple, just to get the concepts
across more easily.  In "real life", I wouldn't use the triggers to manage
the clearing of the checkpoints, provided your DBMS let's you schedule
certain stored procedures to run at a certain time (or, even if not, you're
comfortable with using some scheduler, such as cron under unix, to run a
stored procedure periodically, say, every 3 or 6 hours).  Moving this
"clearing the checkpoints" functionality out of the triggers and into
periodic stored procedures will keep the triggers from further slowing down
committing transactions that involve the Reference table.  The stored
procedures, then, periodically look at the oldest checkpoint and clear out
any rows OLDER than the oldest checkpoint, and also clear out any rows older
than, say 24 hours, to make sure that an app crash somewhere didn't orphan
a checkpoint (and cause the shadow table to start growing endlessly).

Hope this makes sense and doesn't sound too complicated -- it really isn't.

Hope this helps.

Doug

> > ...
> >
[quoted text clipped - 5 lines]
>  I'm not really familiar with that approach, anyone have pointers to
>  further reading on this (preferably online) or relevant examples?
Doug Morse - 10 Aug 2007 13:52 GMT
Hi,

Oops, just be sure that I'm 100% clear re: the last paragraph in my previous
post:

You DO still of course to need to use triggers.  The "lie" I told had
to do with using them to manage the checkpoints.  THAT functionality is much
better handled with periodic stored procedures.  Triggers, however, are
still need to manage the contents of the shadow table.  Specifically, you
will need "ON INSERT", "ON UPDATE", and "ON DELETE" triggers for the
Reference table.

Doug

>  ...
>
[quoted text clipped - 3 lines]
>  certain stored procedures to run at a certain time (or, even if not, you're
>  ...
Ian Wilson - 10 Aug 2007 15:26 GMT
> If you let me know what DBMS you're using, I can give some hints / pointers
> as to what triggers you would need to create.

I'm using Apache Derby (AKA IBM Cloudscape AKA JavaDB).

In network mode (would the same apply in embedded mode?)
Arne Vajhøj - 19 Aug 2007 03:26 GMT
> It's a bit quiet in comp.lang.java.databases so I hope no-one minds a
> database question that isn't particularly related to Java or JDBC
[quoted text clipped - 35 lines]
>
> What do you do?

I think you should drop database builtin locking and go to
application based locking.

Search for optimistic locking and pessimistic locking.

There are well known techniques to handle database operations
in human time (minutes) instead of computer time (milliseconds).

Arne
RedGrittyBrick - 20 Aug 2007 17:17 GMT
> I think you should drop database builtin locking and go to
> application based locking.
[quoted text clipped - 3 lines]
> There are well known techniques to handle database operations
> in human time (minutes) instead of computer time (milliseconds).

My Google-fu appears to be weak today. I just get lots of refs to EJB
locking, Apache Object-relational-bridging etc. These don't appear to be
what you are referring to :-(
Arne Vajhøj - 01 Oct 2007 03:44 GMT
>> I think you should drop database builtin locking and go to
>> application based locking.
[quoted text clipped - 7 lines]
> locking, Apache Object-relational-bridging etc. These don't appear to be
> what you are referring to :-(

I may have been a bit too optimistic.

:-(

http://c2.com/cgi/wiki?OptimisticLocking
http://c2.com/cgi/wiki?PessimisticLocking
http://martinfowler.com/eaaCatalog/optimisticOfflineLock.html
http://martinfowler.com/eaaCatalog/pessimisticOfflineLock.html
http://www.javaworld.com/javaworld/jw-01-2006/jw-0130-pojo.html?page=6

say something, but I must admit that it is far from clear
what they really mean.

If you can get your hands on a copy of Fowlers book, then
it has examples in Java/JDBC.

Arne
RedGrittyBrick - 01 Oct 2007 14:20 GMT
>>> I think you should drop database builtin locking and go to
>>> application based locking.
[quoted text clipped - 23 lines]
> If you can get your hands on a copy of Fowlers book, then
> it has examples in Java/JDBC.

Many thanks :-)


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.