Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / July 2007

Tip: Looking for answers? Try searching our database.

Problem with FileLock

Thread view: 
alejandrina - 13 Jul 2007 22:34 GMT
Hi experts,

We need to update a file (on a file server) from many different
machines. To synchromize the updates I am using FileLock.

Everything works as advertised (ie if a machine gets the lock,
it writes to the file while the other machines wait, everything is
written in the proper order). The same test, using Linux machines,
fails: the file is clobbered (meaning one update gets on top of
another). No Exceptions are thrown; in fact the debug statements
indicate that all the machines are acquiring the exclusive lock)

Anyone can shed light? Here is the critical method with some
debug statements:

    public void write (String s) throws Exception {
        ByteBuffer bb = stringToByteBuffer(s);

        //lock the file and wait till we can
        FileChannel channel = fos.getChannel();
        FileLock lock = null;
        try {
            while ((lock = channel.tryLock()) == null)  {
                System.out.println (Utils.getHostname() + " Failed lock...wait");
                Thread.sleep(100);
            }

            System.out.println (Utils.getHostname() + " Locked:" + lock);
            System.out.println (Utils.getHostname() + " Lock type is "+
((lock.isShared())?"shared":"exclusive"));

            System.out.println (Utils.getHostname() + " Is lock valid: " +
lock.isValid());

           //write the title first if noone's done it
           //and they asked for one
           if (channel.size() <= 0 && bbTitle != null)
               channel.write (bbTitle);
           channel.write(bb);

        } catch (Exception e) {
           throw (e);
        } finally {
           if (lock != null) {
               System.out.println (Utils.getHostname() + " Releasing lock");
               lock.release();
           }
        }

    }
Oliver Wong - 13 Jul 2007 23:55 GMT
> Hi experts,
>
[quoted text clipped - 7 lines]
> another). No Exceptions are thrown; in fact the debug statements
> indicate that all the machines are acquiring the exclusive lock)

http://java.sun.com/javase/6/docs/api/java/nio/channels/FileLock.html
<quote>
Whether or not a lock actually prevents another program from accessing the
content of the locked region is system-dependent and therefore
unspecified. The native file-locking facilities of some systems are merely
advisory, meaning that programs must cooperatively observe a known locking
protocol in order to guarantee data integrity. On other systems native
file locks are mandatory, meaning that if one program locks a region of a
file then other programs are actually prevented from accessing that region
in a way that would violate the lock. On yet other systems, whether native
file locks are advisory or mandatory is configurable on a per-file basis.
To ensure consistent and correct behavior across platforms, it is strongly
recommended that the locks provided by this API be used as if they were
advisory locks.
[...]
On some systems, closing a channel releases all locks held by the Java
virtual machine on the underlying file regardless of whether the locks
were acquired via that channel or via another channel open on the same
file. It is strongly recommended that, within a program, a unique channel
be used to acquire all locks on any given file.
[...]
In general, great care should be taken when locking files that reside on
network filesystems.
</quote>

   - Oliver
alejandrina - 14 Jul 2007 01:17 GMT
> > Hi experts,
>
[quoted text clipped - 34 lines]
>
>     - Oliver

Yes, I have read all this. So, what is the real meaning of "advisory"?
Can you or can't you issue a lock on a file on Linux?

Which systems are "advisory" and why doesn't anyone spell this out?
Patricia Shanahan - 14 Jul 2007 14:51 GMT
...
> Yes, I have read all this. So, what is the real meaning of "advisory"?
...

"Advisory", in conjunction with locking, usually means that threads that
do not choose to play by the rules can go ahead and access the protected
resource regardless of the state of the locks.

If that is the intended interpretation, I don't think the quoted passage
helps with understanding your problem, because you have already checked
that your threads only access the resource while in possession of an
exclusive lock on it.

Have you tried calling force() after the write? Maybe there is some
buffering in the FileChannel that is delaying the effect of the write
past the release of the lock.

Patricia
alejandrina - 16 Jul 2007 15:33 GMT
> ...
> > Yes, I have read all this. So, what is the real meaning of "advisory"?
[quoted text clipped - 15 lines]
>
> Patricia

That would completely negate the effect of the lock, wouldn't it?
Patricia Shanahan - 16 Jul 2007 16:54 GMT
>> ...
>>> Yes, I have read all this. So, what is the real meaning of "advisory"?
[quoted text clipped - 16 lines]
>
> That would completely negate the effect of the lock, wouldn't it?

No, the lock would still serve to serialize the periods during which a
thread is entitled to write to the file. The force is related to making
sure the write really happens between during that period.

However, given other posts I'm not sure this is worth trying until after
you have checked that the lock demon is running on the NFS server.

Patricia
Lew - 14 Jul 2007 15:12 GMT
> Yes, I have read all this. So, what is the real meaning of "advisory"?

"Advisory" means that any program can find out if another program has a lock
on a file, but need not respect it.

> Can you or can't you issue a lock on a file on Linux?

Yes, but you can't enforce it.

> Which systems are "advisory"

You have to check the docs for each system.  Perhaps Google is your friend
(GIYF) on this question - have you tried it?

>  and why doesn't anyone spell this out?

Why should they?

Besides, they do.  It's in the docs for each system.

Signature

Lew

Gordon Beaton - 14 Jul 2007 15:23 GMT
> We need to update a file (on a file server) from many different
> machines. To synchromize the updates I am using FileLock.
[quoted text clipped - 5 lines]
> another). No Exceptions are thrown; in fact the debug statements
> indicate that all the machines are acquiring the exclusive lock)

If you are accessing the files over NFS, then I believe (but am not
sure) that

- the locks may be unknown to the server, and consequently to other
 NFS clients.

- each client might be caching its updates, which are not guaranteed
 to be visible to other clients without a close/open sequence in
 between (if even then). This could result in corruption even if the
 locking is in fact working as you expect it to.

What happens when you run two instances of your application on the
*same* host?

I would recommend that you update the file from a single (server)
process that does so on behalf of the other (client) processes, so the
updates are atomic and strictly serialized, without any need for
locking.

/gordon

--
Lew - 14 Jul 2007 15:35 GMT
> I would recommend that you update the file from a single (server)
> process that does so on behalf of the other (client) processes, so the
> updates are atomic and strictly serialized, without any need for
> locking.

This is an excellent pattern (it was called "resource manager" in QNX, I don't
know of the GoF name for it if there is one) to protect a single, serialized
resource from trouble with multiple clients.  The resource manager listens for
service requests for the resource, and mediates all such requests.

Signature

Lew

bencoe@gmail.com - 14 Jul 2007 18:54 GMT
> > I would recommend that you update the file from a single (server)
> > process that does so on behalf of the other (client) processes, so the
[quoted text clipped - 8 lines]
> --
> Lew

I concur, I have had a lot of trouble trying to get file locks working
in a server environment (you can run into very nasty problems like a
file that doesn't unlock). Using a shared object to take care of IO
works, much, much better.

----
Ben
http://www.plink-search.com
Nigel Wade - 16 Jul 2007 10:31 GMT
>> We need to update a file (on a file server) from many different
>> machines. To synchromize the updates I am using FileLock.
[quoted text clipped - 24 lines]
> updates are atomic and strictly serialized, without any need for
> locking.

For NFS rpc.lockd must be running on the server, it must be contactable using
RPC (i.e. the dynamic RPC port it is registered at must not be blocked by a
firewall), each client needs to co-operates by sending the lock request, and
waiting for the lock responses. Even then it might not work.

NFS was designed to be stateless, principally so that it could survive a server
re-boot or network drop-out. File locking is inherently stateful (hence the
need for the additional service rpc.lockd) and the two don't mix well.
Basically, don't attempt file locking over NFS unless you have plenty of time
to spare to debug it thoroughly. There are many failure modes and you should
not rely on it without rigorous testing of how you code reacts to each failure
mode.

Signature

Nigel Wade, System Administrator, Space Plasma Physics Group,
           University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw@ion.le.ac.uk
Phone :     +44 (0)116 2523548, Fax : +44 (0)116 2523555

alejandrina - 16 Jul 2007 15:31 GMT
> >> We need to update a file (on a file server) from many different
> >> machines. To synchromize the updates I am using FileLock.
[quoted text clipped - 43 lines]
> E-mail :    n...@ion.le.ac.uk
> Phone :     +44 (0)116 2523548, Fax : +44 (0)116 2523555

AHHHHHHHHHH. That's the kind of info we need! Thanks so much...I'll
continue this discussion with the sys admin.
alejandrina - 30 Jul 2007 18:14 GMT
> > >> We need to update a file (on a file server) from many different
> > >> machines. To synchromize the updates I am using FileLock.
[quoted text clipped - 46 lines]
> AHHHHHHHHHH. That's the kind of info we need! Thanks so much...I'll
> continue this discussion with the sys admin.

We were running Samba...When that went away, everything worked
perfectly.


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.