Java Forum / Databases / October 2007
(Mis)use of transactions
RedGrittyBrick - 09 Aug 2007 21:08 GMT It's a bit quiet in comp.lang.java.databases so I hope no-one minds a database question that isn't particularly related to Java or JDBC (though that's what I am actually using).
I have an app, several bits of which allow a user to maintain simple tabular reference data that is stored in database tables.
When the user elects to maintain one of these tables, I pop up a new window that contains a JPanel containing a JTable, an "OK" button and a "Cancel" button.
In the JPanel's constructor I get a java.sql.Connection instance and thus open a connection to the DBMS.
Whilst the user edits the data in the JTable I am firing off the appropriate executeUpdate methods to send updates to the DBMS.
When the user clicks OK I invoke commit() and close() on the connection.
If the user clicks Cancel I invoke rollback() and close() on the connection.
-- * --
However if the user does a few changes and then wanders off for lunch, the database table is left in a locked state that obviously is likely to cause problems for other users (or other threads in the user's app).
The only improvements I can think of are to track the changes inside the app and only do getConnection(), executeUpdate(), close() in the "OK" button's ActionListener.
Keeping a log of changes or comparing before and after versions of a List<foo> in a table model seems a bit tedious.
Maybe I should do the getConnection(), executeUpdate(), close() for each little change to a table row and remove the "Cancel" button?
What do you do?
Doug Morse - 09 Aug 2007 22:18 GMT Hi RedGrittyBrick,
I never, ever leave control of a DBMS transaction in the hands of user, for just the reasons you point out. Indeed, I do everything possible to minimize the time between starting a transaction and then either committing it or rolling it back.
Thus, I suggest exactly what you proposed: Cache locally all the edits the user is making and then only start the transaction and commit it once the user says "OK". Of course, there's always the possibility that another user might be editing the data at the same time. So, if it's reasonable enough (e.g., the entire reference table isn't too large), after the user has pressed OK but before I start the transaction, I'll requery the table and compare it to what the user is seeing (i.e., the local cache). If there has been a change, I notify the user that the database has changed since they started editing and prompt them if they'd like to review what's changed before committing their work. If the reference table does not have a small number of rows, then I'll at least notify them of rows that others have changed that the local user is also trying to change. This user feedback becomes especially important if another user has changed the primary key (P.K.) of a row that the local user also edited or if another user has deleted one of the rows the local user has edited.
Tracking the changes that the user has made should be pretty easy, if it's a straightforward reference / lookup table. If it's just a "Code" and "Description" table, then I think a Nx4 array/matrix should suffice. Column 1: Type (I,D,U for insert, delete, update, respectively), column 2: old primary key value, column 3: new primary key value (just copy old p.k. value if user didn't change), column 4: new description. The N rows are the actions taken by the user, in order. Here's an example of what such an in-memory change matrix might look like. There are just two columns, Code and Description, and Code is the P.K. The example is taken from a reference table of psychological "presenting problems" occurring at a crisis intervention:
Old New New My Type Code Code Description Comments ---- ----- ----- ------------------------------ ------------------------------ I ATOT ATOT Atypical Behavior - Other User added new code/desc. I ATUB ATUB Unusual / Bizarre Behavior "" I ATVO ATVO Violent / Out of Control "" D ME00 ME00 Medical / Somatic User deleted code/desc. D MR00 MR00 Mental Retardation "" U PSDE PSDE Depression User fixed misspell: Depresion U PSDA PSDL Delusional User changed code (P.K.) U PSPA PSPN Paranoria, NOS User changed both code/desc.
A simple in-memory table like this also makes it very easy to implement an Undo function for the user: To undo, just pop off (the bottom) of the stack. Actually, you should probably add a fifth column: Old Description. This would allow you to check and see if another user has changed the Description column while the local user was editing, as well as allow your Undo function to undo edits to the Description column.
All pretty easy, I'd think, and shouldn't require too many LOC at all.
A variant on all this would be to copy of the reference table into a temporary table and let the user make all of her edits on the temporary table. This variant thus would allow you to leave the transaction open as long as you like w/o affecting other users. Unlike the in-memory option, though, it consumes server-side resources; plus, transactions usually timeout eventually. The upside is that if your local client crashes, and you're using the in-memory approach, the edits are lost, but temp tables can often be configured to persist until explicitly deleted, so the local client, once re-started, could reconnect to the temp table and recover the lost edits. That said, though, I'd still recommend against allocating and holding open server-side resources when it's not essential. If you wanted this level of crash recovery, you could just as well periodically write out your in-memory changes table (example above) to a local disk file for recovering after a local client crash.
Finally, you already mentioned the third option I can think of, namely committing each user-specified change as you go. I would NOT close and re-open the connection, though, as you offered. It's not necessary, and will likely be incredibly slow, because creating and tearing down DB connections is generally resource-intensive and latency-bound due to the lock-step messages that have to go back and forth over the wire. Just leave the connection open as long as your UI dialog is open, but start, execute, and commit the transaction all at once.
Hope this helps.
Doug
> It's a bit quiet in comp.lang.java.databases so I hope no-one minds a > database question that isn't particularly related to Java or JDBC [quoted text clipped - 34 lines] > > What do you do? Lew - 10 Aug 2007 02:18 GMT > I never, ever leave control of a DBMS transaction in the hands of user, for > just the reasons you point out. Indeed, I do everything possible to [quoted text clipped - 4 lines] > user is making and then only start the transaction and commit it once the > user says "OK". Yes, that's the way.
> This user feedback > becomes especially important if another user has changed the primary key > (P.K.) of a row that the local user also edited or if another user has > deleted one of the rows the local user has edited. I don't suggest that you permit an "update" of PK information - really, that's a delete of the old row and an insert of a new one. The whole idea of a PK is that it identifies the row, and in the object model, the object under consideration. By that interpretation, a different PK represents a different object, not the same one with a new key.
There are CASCADE implications also - how do you revise all foreign-key (FK) references to the object if the PK changes?
It's better to fix the PK as uneditable when in "edit" mode on an existing record.
You also mention "one of the rows the user has edited". One way to minimize conflict is to have a user edit only one logical item (e.g., one customer, one order, one event) at a time. This reduces the likelihood in most applications that more than one user will need to edit (delete, ...) the same rows at the same time.
There is a tension between reserving resources for the entire time one needs them (e.g., locking the affected rows) and the need to release resources ASAP.
 Signature Lew
Thomas Hawtin - 09 Aug 2007 23:25 GMT > In the JPanel's constructor I get a java.sql.Connection instance and > thus open a connection to the DBMS. Sounds like very confused code to me. Is it a UI class or a database class? In any case, I'd avoid subclassing unnecessarily.
> However if the user does a few changes and then wanders off for lunch, > the database table is left in a locked state that obviously is likely to > cause problems for other users (or other threads in the user's app). And quite possibly the connection times out causing all of the users data to be discarded. I'm guessing most people don't like it when that happens.
> Maybe I should do the getConnection(), executeUpdate(), close() for each > little change to a table row and remove the "Cancel" button? No particular need to close each time, although it is worth being able to reopen then connection if it fails.
From a usability perspective I like applications that are as immediate as possible. That means no OK/Cancel buttons. However, if you want locking you can still implement your own locking system on top of the database.
There is a problem in that updates that other actors make while the window is open wont be automatically shown. You can do that relatively efficiently by putting triggers on the main table that copy updates into shadow tables.
Tom Hawtin
RedGrittyBrick - 10 Aug 2007 10:52 GMT >> In the JPanel's constructor I get a java.sql.Connection instance and >> thus open a connection to the DBMS. > > Sounds like very confused code to me. Is it a UI class or a database > class? I simplified my description somewhat. In reality the JDBC methods are in a "model" class. I instantiate the model in the constructor of the "view".
I do find deciding what functionality should go in what classes to be confusing. I think its something people get better at with practice.
Maybe I should do FooModel model = new FooModel(); FooView view = new FooView(model); instead of FooView view = new FooView(); ... class FooView { FooModel model = new FooModel(); ... } Though I can't see a significant benefit. I probably did it the latter way because I associate the model with the JTable more than with the container. So my instantiation of the model is close to the instantiation of the JTable. More to ponder.
> In any case, I'd avoid subclassing unnecessarily. I find extending JPanel is a convenient approach. I'm aware of the "composition vs inheritance" debate, so maybe I'll ponder this further.
<further good points snipped for brevity>
> There is a problem in that updates that other actors make while the > window is open wont be automatically shown. You can do that relatively > efficiently by putting triggers on the main table that copy updates into > shadow tables. I'm not really familiar with that approach, anyone have pointers to further reading on this (preferably online) or relevant examples?
P.S. Thanks to Doug & Lew too, your good advice is much appreciated.
Doug Morse - 10 Aug 2007 13:41 GMT I'm not actually sure what he means by "shadow tables", but I presume that he means either (a) an identical reference table with recently inserted or deleted rows (he views updates as delete's followed up inserts, which seems sensible to me -- so by his reasoning "no updates"), and perhaps an added timestamp of the (prior) transaction time(s), or (b) a table with just prior transaction time, old P.K. for the affected row in the Reference table, new P.K. for the affected row, and a char(1) column set to either 'I' or 'D' for insert or delete, respectively. With either kind of shadow table, your app could periodically query it for all committed transactions since the datetime you originally cached the Reference table contents locally. Then, just see if any of the P.K.'s returned (old or new) match your cached P.K.'s. If so, then you know that another user has affected a row you've been caching locally and thus can notify the user and/or take action accordingly. Again, actually pretty simple stuff -- the LOCs to do this is probably not terribly more text than this paragraph or so.
As far as triggers, you use them to manage the contents of the shadow table. With every committed transaction to the Reference table, your triggers will add the appropriate entries to the shadow table. Your app and triggers will also need to "checkpoint" the shadow table in some way -- either with special, reserved values in the shadow table or with a separate shadow checkpoint table. Basically, you just need your app to record somewhere in the DBMS that it cached the reference table at a certain time, and then delete that entry when it's done locally caching. Your triggers then use this information to clean out the shadow table so that it doesn't grow endlessly. Basically, the triggers just query for the oldest app-cache datetime, and delete any shadow table entries OLDER than that date, since in principle no app instance needs to know about changes to the Reference table that are older than that date (i.e., when the oldest obtained its cache). Finally, your triggers should be coded to periodically delete all entries in the shadow table after some maximum timeout period of, say, 24 hours. This is needed to handle the case when an app instance crashes and in so doing "strands" its checkpoint in the shadow table or the separate shadow checkpoint table.
As far as managing the checkpoints, using the shadow table itself is the most straightforward, provided there's some reserved column value you can use to indicate a check point entry (or just add a bool column type named "checkpoint"). The you could use, say, the Description column to indicate some sort of app-instance ID, if you wanted to. As long as you store the *precise* datetime in the shadow table and keep up with that same *precise* datetime in your app, then an app-instance ID isn't need, as each app (or app dialog box, if you allow multiple dialogs for editing in the same app instance) is identified by its unique datetime that it cached the Reference Table. Finally, if you want to use a separate shadow checkpoint table, you need to accomplish the same tasks but against that table instead -- the logic of it all is identical either way you go.
If you let me know what DBMS you're using, I can give some hints / pointers as to what triggers you would need to create.
Finally, I "lied" a bit to keep things simple, just to get the concepts across more easily. In "real life", I wouldn't use the triggers to manage the clearing of the checkpoints, provided your DBMS let's you schedule certain stored procedures to run at a certain time (or, even if not, you're comfortable with using some scheduler, such as cron under unix, to run a stored procedure periodically, say, every 3 or 6 hours). Moving this "clearing the checkpoints" functionality out of the triggers and into periodic stored procedures will keep the triggers from further slowing down committing transactions that involve the Reference table. The stored procedures, then, periodically look at the oldest checkpoint and clear out any rows OLDER than the oldest checkpoint, and also clear out any rows older than, say 24 hours, to make sure that an app crash somewhere didn't orphan a checkpoint (and cause the shadow table to start growing endlessly).
Hope this makes sense and doesn't sound too complicated -- it really isn't.
Hope this helps.
Doug
> > ... > > [quoted text clipped - 5 lines] > I'm not really familiar with that approach, anyone have pointers to > further reading on this (preferably online) or relevant examples? Doug Morse - 10 Aug 2007 13:52 GMT Hi,
Oops, just be sure that I'm 100% clear re: the last paragraph in my previous post:
You DO still of course to need to use triggers. The "lie" I told had to do with using them to manage the checkpoints. THAT functionality is much better handled with periodic stored procedures. Triggers, however, are still need to manage the contents of the shadow table. Specifically, you will need "ON INSERT", "ON UPDATE", and "ON DELETE" triggers for the Reference table.
Doug
> ... > [quoted text clipped - 3 lines] > certain stored procedures to run at a certain time (or, even if not, you're > ... Ian Wilson - 10 Aug 2007 15:26 GMT > If you let me know what DBMS you're using, I can give some hints / pointers > as to what triggers you would need to create. I'm using Apache Derby (AKA IBM Cloudscape AKA JavaDB).
In network mode (would the same apply in embedded mode?)
Arne Vajhøj - 19 Aug 2007 03:26 GMT > It's a bit quiet in comp.lang.java.databases so I hope no-one minds a > database question that isn't particularly related to Java or JDBC [quoted text clipped - 35 lines] > > What do you do? I think you should drop database builtin locking and go to application based locking.
Search for optimistic locking and pessimistic locking.
There are well known techniques to handle database operations in human time (minutes) instead of computer time (milliseconds).
Arne
RedGrittyBrick - 20 Aug 2007 17:17 GMT > I think you should drop database builtin locking and go to > application based locking. [quoted text clipped - 3 lines] > There are well known techniques to handle database operations > in human time (minutes) instead of computer time (milliseconds). My Google-fu appears to be weak today. I just get lots of refs to EJB locking, Apache Object-relational-bridging etc. These don't appear to be what you are referring to :-(
Arne Vajhøj - 01 Oct 2007 03:44 GMT >> I think you should drop database builtin locking and go to >> application based locking. [quoted text clipped - 7 lines] > locking, Apache Object-relational-bridging etc. These don't appear to be > what you are referring to :-( I may have been a bit too optimistic.
:-( http://c2.com/cgi/wiki?OptimisticLocking http://c2.com/cgi/wiki?PessimisticLocking http://martinfowler.com/eaaCatalog/optimisticOfflineLock.html http://martinfowler.com/eaaCatalog/pessimisticOfflineLock.html http://www.javaworld.com/javaworld/jw-01-2006/jw-0130-pojo.html?page=6
say something, but I must admit that it is far from clear what they really mean.
If you can get your hands on a copy of Fowlers book, then it has examples in Java/JDBC.
Arne
RedGrittyBrick - 01 Oct 2007 14:20 GMT >>> I think you should drop database builtin locking and go to >>> application based locking. [quoted text clipped - 23 lines] > If you can get your hands on a copy of Fowlers book, then > it has examples in Java/JDBC. Many thanks :-)
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|