Java Forum / General / July 2007
Problem with FileLock
alejandrina - 13 Jul 2007 22:34 GMT Hi experts,
We need to update a file (on a file server) from many different machines. To synchromize the updates I am using FileLock.
Everything works as advertised (ie if a machine gets the lock, it writes to the file while the other machines wait, everything is written in the proper order). The same test, using Linux machines, fails: the file is clobbered (meaning one update gets on top of another). No Exceptions are thrown; in fact the debug statements indicate that all the machines are acquiring the exclusive lock)
Anyone can shed light? Here is the critical method with some debug statements:
public void write (String s) throws Exception { ByteBuffer bb = stringToByteBuffer(s);
//lock the file and wait till we can FileChannel channel = fos.getChannel(); FileLock lock = null; try { while ((lock = channel.tryLock()) == null) { System.out.println (Utils.getHostname() + " Failed lock...wait"); Thread.sleep(100); }
System.out.println (Utils.getHostname() + " Locked:" + lock); System.out.println (Utils.getHostname() + " Lock type is "+ ((lock.isShared())?"shared":"exclusive"));
System.out.println (Utils.getHostname() + " Is lock valid: " + lock.isValid());
//write the title first if noone's done it //and they asked for one if (channel.size() <= 0 && bbTitle != null) channel.write (bbTitle); channel.write(bb);
} catch (Exception e) { throw (e); } finally { if (lock != null) { System.out.println (Utils.getHostname() + " Releasing lock"); lock.release(); } }
}
Oliver Wong - 13 Jul 2007 23:55 GMT > Hi experts, > [quoted text clipped - 7 lines] > another). No Exceptions are thrown; in fact the debug statements > indicate that all the machines are acquiring the exclusive lock) http://java.sun.com/javase/6/docs/api/java/nio/channels/FileLock.html <quote> Whether or not a lock actually prevents another program from accessing the content of the locked region is system-dependent and therefore unspecified. The native file-locking facilities of some systems are merely advisory, meaning that programs must cooperatively observe a known locking protocol in order to guarantee data integrity. On other systems native file locks are mandatory, meaning that if one program locks a region of a file then other programs are actually prevented from accessing that region in a way that would violate the lock. On yet other systems, whether native file locks are advisory or mandatory is configurable on a per-file basis. To ensure consistent and correct behavior across platforms, it is strongly recommended that the locks provided by this API be used as if they were advisory locks. [...] On some systems, closing a channel releases all locks held by the Java virtual machine on the underlying file regardless of whether the locks were acquired via that channel or via another channel open on the same file. It is strongly recommended that, within a program, a unique channel be used to acquire all locks on any given file. [...] In general, great care should be taken when locking files that reside on network filesystems. </quote>
- Oliver
alejandrina - 14 Jul 2007 01:17 GMT > > Hi experts, > [quoted text clipped - 34 lines] > > - Oliver Yes, I have read all this. So, what is the real meaning of "advisory"? Can you or can't you issue a lock on a file on Linux?
Which systems are "advisory" and why doesn't anyone spell this out?
Patricia Shanahan - 14 Jul 2007 14:51 GMT ...
> Yes, I have read all this. So, what is the real meaning of "advisory"? ...
"Advisory", in conjunction with locking, usually means that threads that do not choose to play by the rules can go ahead and access the protected resource regardless of the state of the locks.
If that is the intended interpretation, I don't think the quoted passage helps with understanding your problem, because you have already checked that your threads only access the resource while in possession of an exclusive lock on it.
Have you tried calling force() after the write? Maybe there is some buffering in the FileChannel that is delaying the effect of the write past the release of the lock.
Patricia
alejandrina - 16 Jul 2007 15:33 GMT > ... > > Yes, I have read all this. So, what is the real meaning of "advisory"? [quoted text clipped - 15 lines] > > Patricia That would completely negate the effect of the lock, wouldn't it?
Patricia Shanahan - 16 Jul 2007 16:54 GMT >> ... >>> Yes, I have read all this. So, what is the real meaning of "advisory"? [quoted text clipped - 16 lines] > > That would completely negate the effect of the lock, wouldn't it? No, the lock would still serve to serialize the periods during which a thread is entitled to write to the file. The force is related to making sure the write really happens between during that period.
However, given other posts I'm not sure this is worth trying until after you have checked that the lock demon is running on the NFS server.
Patricia
Lew - 14 Jul 2007 15:12 GMT > Yes, I have read all this. So, what is the real meaning of "advisory"? "Advisory" means that any program can find out if another program has a lock on a file, but need not respect it.
> Can you or can't you issue a lock on a file on Linux? Yes, but you can't enforce it.
> Which systems are "advisory" You have to check the docs for each system. Perhaps Google is your friend (GIYF) on this question - have you tried it?
> and why doesn't anyone spell this out? Why should they?
Besides, they do. It's in the docs for each system.
 Signature Lew
Gordon Beaton - 14 Jul 2007 15:23 GMT > We need to update a file (on a file server) from many different > machines. To synchromize the updates I am using FileLock. [quoted text clipped - 5 lines] > another). No Exceptions are thrown; in fact the debug statements > indicate that all the machines are acquiring the exclusive lock) If you are accessing the files over NFS, then I believe (but am not sure) that
- the locks may be unknown to the server, and consequently to other NFS clients.
- each client might be caching its updates, which are not guaranteed to be visible to other clients without a close/open sequence in between (if even then). This could result in corruption even if the locking is in fact working as you expect it to.
What happens when you run two instances of your application on the *same* host?
I would recommend that you update the file from a single (server) process that does so on behalf of the other (client) processes, so the updates are atomic and strictly serialized, without any need for locking.
/gordon
--
Lew - 14 Jul 2007 15:35 GMT > I would recommend that you update the file from a single (server) > process that does so on behalf of the other (client) processes, so the > updates are atomic and strictly serialized, without any need for > locking. This is an excellent pattern (it was called "resource manager" in QNX, I don't know of the GoF name for it if there is one) to protect a single, serialized resource from trouble with multiple clients. The resource manager listens for service requests for the resource, and mediates all such requests.
 Signature Lew
bencoe@gmail.com - 14 Jul 2007 18:54 GMT > > I would recommend that you update the file from a single (server) > > process that does so on behalf of the other (client) processes, so the [quoted text clipped - 8 lines] > -- > Lew I concur, I have had a lot of trouble trying to get file locks working in a server environment (you can run into very nasty problems like a file that doesn't unlock). Using a shared object to take care of IO works, much, much better.
---- Ben http://www.plink-search.com
Nigel Wade - 16 Jul 2007 10:31 GMT >> We need to update a file (on a file server) from many different >> machines. To synchromize the updates I am using FileLock. [quoted text clipped - 24 lines] > updates are atomic and strictly serialized, without any need for > locking. For NFS rpc.lockd must be running on the server, it must be contactable using RPC (i.e. the dynamic RPC port it is registered at must not be blocked by a firewall), each client needs to co-operates by sending the lock request, and waiting for the lock responses. Even then it might not work.
NFS was designed to be stateless, principally so that it could survive a server re-boot or network drop-out. File locking is inherently stateful (hence the need for the additional service rpc.lockd) and the two don't mix well. Basically, don't attempt file locking over NFS unless you have plenty of time to spare to debug it thoroughly. There are many failure modes and you should not rely on it without rigorous testing of how you code reacts to each failure mode.
 Signature Nigel Wade, System Administrator, Space Plasma Physics Group, University of Leicester, Leicester, LE1 7RH, UK E-mail : nmw@ion.le.ac.uk Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
alejandrina - 16 Jul 2007 15:31 GMT > >> We need to update a file (on a file server) from many different > >> machines. To synchromize the updates I am using FileLock. [quoted text clipped - 43 lines] > E-mail : n...@ion.le.ac.uk > Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555 AHHHHHHHHHH. That's the kind of info we need! Thanks so much...I'll continue this discussion with the sys admin.
alejandrina - 30 Jul 2007 18:14 GMT > > >> We need to update a file (on a file server) from many different > > >> machines. To synchromize the updates I am using FileLock. [quoted text clipped - 46 lines] > AHHHHHHHHHH. That's the kind of info we need! Thanks so much...I'll > continue this discussion with the sys admin. We were running Samba...When that went away, everything worked perfectly.
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|