Any sure fire way to check file existence on Linux NFS?

Question

I am working on a Java program that requires to check the existence of files.

Well, simple enough, the code make use calls to File.exists() for checking file existence. And the problem I have is, it reports false positive. That means the file does not actually exist but exists() method returns true. No exception was captured (at least no exception like "Stale NFS handle"). The program even managed to read the file through InputStream, getting 0 bytes as expected and yet no exception. The target directory is a Linux NFS. And I am 100% sure that the file being looked for never exists.

I know there are known bugs (kind of API limitation) exist for java.io.File.exists(). So I've then added another way round by checking file existence using Linux command ls. Instead of making call to File.exists() the Java code now runs a Linux command to ls the target file. If exit code is 0, file exists. Otherwise, file does not exist.

The number of times the issue is hit seems to be reduced with the introduction of the trick, but still pops. Again, no error was captured anywhere (stdout this time). That means the problem is so serious that even native Linux command won't fix for 100% of the time.

So there are couple of questions around:

1. I believe Java's well known issue on File.exists() is about reporting false negative. Where file was reported to not exist but in fact does exist. As the API does not throws IOException for File.exists(), it choose to swallow the Exception in the case calls to OS's underlying native functions failed e.g. NFS timeout. But then this does not explain the false positive case I am having, given that the file never exist. Any throw on this one?
2. My understanding on Linux ls exit code is, 0 means okay, equivalent to file exists. Is this understanding wrong? The man page of ls is not so clear on explaining the meaning of exit code: Exit status is 0 if OK, 1 if minor problems, 2 if serious trouble.
3. All right, back to subject. Any surefire way to check File existence with Java on Linux? Before we see JDK7 with NIO2 officially released.

Answer 1

Here is a JUnit test that shows the problem and some Java Code that actually tries to read the file.

The problem happens e.g. using Samba on OSX Mavericks. A possible reason is explaned by the statement in: http://appleinsider.com/articles/13/06/11/apple-shifts-from-afp-file-sharing-to-smb2-in-os-x-109-mavericks

It aggressively caches file and folder properties and uses opportunistic locking to enable better caching of data.

Please find below a checkFile that will actually attempt to read a few bytes and forcing a true file access to avoid the caching misbehaviour ...

JUnit test:

/**
 * test file exists function on Network drive replace the testfile name and ssh computer
     * with your actual environment
 * @throws Exception
 */
@Test
public void testFileExistsOnNetworkDrive() throws Exception {
    String testFileName="/Volumes/bitplan/tmp/testFileExists.txt";
    File testFile=new File(testFileName);
    testFile.delete();
    for (int i=0;i<10;i++) {
        Thread.sleep(50);
        System.out.println(""+i+":"+OCRJob.checkExists(testFile));
        switch (i) {
        case 3:
            // FileUtils.writeStringToFile(testFile, "here we go");
            Runtime.getRuntime().exec("/usr/bin/ssh phobos /usr/bin/touch "+testFileName);
            break;
        }
    }
}

checkExists source code:

/**
 * check if the given file exists
 * @param f
 * @return true if file exists
 */
public static boolean checkExists(File f)  {
    try {
        byte[] buffer = new byte[4];
        InputStream is = new FileInputStream(f);
        if (is.read(buffer) != buffer.length) { 
            // do something 
        }
        is.close();
        return true;
    } catch (java.io.IOException fnfe) {

    }
    return false;
}

Answer 2

If you need to be robust, try to read the file - and fail gracefully if the file is not there (or there is a permission or other problem). This applies to any other language than Java as well.

The only safe way to tell if the file exist and you can read from it is to actually read a data from the file. Regardless of a file system - local, or remote. The reason is a race condition which can occur right after you get success from checkAccess(path): check, then open file, and you find it suddenly does not exist. Some other thread (or another remote client) may have removed it, or has acquired an exclusive lock. So don't bother checking access, but rather try to read the file. Spending time in running ls just makes race condition window easier to fit.

Answer 3

JDK7 was released a few months ago. There are exists and notExists methods in the Files class but they return a boolean rather than throwing an exception. If you really want an exception then use FileSystems.getDefault().provider().checkAccess(path) and it will throw an exception if the file does not exist.