public abstract class

FSDirectory

extends Directory
java.lang.Object
   ↳ org.apache.lucene.store.Directory
     ↳ org.apache.lucene.store.FSDirectory
Known Direct Subclasses

Class Overview

Base class for Directory implementations that store index files in the file system. There are currently three core subclasses:

  • SimpleFSDirectory is a straightforward implementation using java.io.RandomAccessFile. However, it has poor concurrent performance (multiple threads will bottleneck) as it synchronizes when multiple threads read from the same file.
  • NIOFSDirectory uses java.nio's FileChannel's positional io when reading to avoid synchronization when reading from the same file. Unfortunately, due to a Windows-only Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using interrupt() or cancel(boolean) should use SimpleFSDirectory instead. See NIOFSDirectory java doc for details.
  • MMapDirectory uses memory-mapped IO when reading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's close() is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using interrupt() or cancel(boolean) should use SimpleFSDirectory instead. See MMapDirectory java doc for details.
Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the open(File) method, to allow Lucene to choose the best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use open(File). For all others, you should instantiate the desired implementation directly.

The locking implementation is by default NativeFSLockFactory, but can be changed by passing in a custom LockFactory instance.

See Also

Summary

Fields
public static final int DEFAULT_READ_CHUNK_SIZE Default read chunk size.
protected File directory The underlying filesystem directory
[Expand]
Inherited Fields
From class org.apache.lucene.store.Directory
Protected Constructors
FSDirectory(File path, LockFactory lockFactory)
Create a new FSDirectory for the named location (ctor for subclasses).
Public Methods
synchronized void close()
Closes the store to future operations.
void deleteFile(String name)
Removes an existing file in the directory.
boolean fileExists(String name)
Returns true iff a file with the given name exists.
long fileLength(String name)
Returns the length in bytes of a file in the directory.
long fileModified(String name)
Returns the time the named file was last modified.
static long fileModified(File directory, String name)
Returns the time the named file was last modified.
File getFile()
String getLockID()
Return a string identifier that uniquely differentiates this Directory instance from other Directory instances.
final int getReadChunkSize()
The maximum number of bytes to read at once from the underlying file during readBytes(byte[], int, int).
String[] listAll()
Lists all files (not subdirectories) in the directory.
static String[] listAll(File dir)
Lists all files (not subdirectories) in the directory.
static FSDirectory open(File path, LockFactory lockFactory)
Just like open(File), but allows you to also specify a custom LockFactory.
static FSDirectory open(File path)
Creates an FSDirectory instance, trying to pick the best implementation given the current environment.
IndexInput openInput(String name)
Returns a stream reading an existing file.
final void setReadChunkSize(int chunkSize)
Sets the maximum number of bytes read at once from the underlying file during readBytes(byte[], int, int).
void sync(String name)
Ensure that any writes to this file are moved to stable storage.
String toString()
For debug output.
void touchFile(String name)
Set the modified time of an existing file to now.
Protected Methods
final void initOutput(String name)
Initializes the directory to create a new file with the given name.
[Expand]
Inherited Methods
From class org.apache.lucene.store.Directory
From class java.lang.Object
From interface java.io.Closeable

Fields

public static final int DEFAULT_READ_CHUNK_SIZE

Default read chunk size. This is a conditional default: on 32bit JVMs, it defaults to 100 MB. On 64bit JVMs, it's Integer.MAX_VALUE.

See Also

protected File directory

The underlying filesystem directory

Protected Constructors

protected FSDirectory (File path, LockFactory lockFactory)

Create a new FSDirectory for the named location (ctor for subclasses).

Parameters
path the path of the directory
lockFactory the lock factory to use, or null for the default (NativeFSLockFactory);
Throws
IOException

Public Methods

public synchronized void close ()

Closes the store to future operations.

public void deleteFile (String name)

Removes an existing file in the directory.

Throws
IOException

public boolean fileExists (String name)

Returns true iff a file with the given name exists.

public long fileLength (String name)

Returns the length in bytes of a file in the directory.

public long fileModified (String name)

Returns the time the named file was last modified.

public static long fileModified (File directory, String name)

Returns the time the named file was last modified.

public File getFile ()

public String getLockID ()

Return a string identifier that uniquely differentiates this Directory instance from other Directory instances. This ID should be the same if two Directory instances (even in different JVMs and/or on different machines) are considered "the same index". This is how locking "scopes" to the right index.

public final int getReadChunkSize ()

The maximum number of bytes to read at once from the underlying file during readBytes(byte[], int, int).

See Also

public String[] listAll ()

Lists all files (not subdirectories) in the directory.

Throws
IOException
See Also

public static String[] listAll (File dir)

Lists all files (not subdirectories) in the directory. This method never returns null (throws IOException instead).

Throws
NoSuchDirectoryException if the directory does not exist, or does exist but is not a directory.
IOException if list() returns null

public static FSDirectory open (File path, LockFactory lockFactory)

Just like open(File), but allows you to also specify a custom LockFactory.

Throws
IOException

public static FSDirectory open (File path)

Creates an FSDirectory instance, trying to pick the best implementation given the current environment. The directory returned uses the NativeFSLockFactory.

Currently this returns NIOFSDirectory on non-Windows JREs and SimpleFSDirectory on Windows. It is highly recommended that you consult the implementation's documentation for your platform before using this method.

NOTE: this method may suddenly change which implementation is returned from release to release, in the event that higher performance defaults become possible; if the precise implementation is important to your application, please instantiate it directly, instead. On 64 bit systems, it may also good to return MMapDirectory, but this is disabled because of officially missing unmap support in Java. For optimal performance you should consider using this implementation on 64 bit JVMs.

See above

Throws
IOException

public IndexInput openInput (String name)

Returns a stream reading an existing file.

Throws
IOException

public final void setReadChunkSize (int chunkSize)

Sets the maximum number of bytes read at once from the underlying file during readBytes(byte[], int, int). The default value is DEFAULT_READ_CHUNK_SIZE;

This was introduced due to Sun JVM Bug 6478546, which throws an incorrect OutOfMemoryError when attempting to read too many bytes at once. It only happens on 32bit JVMs with a large maximum heap size.

Changes to this value will not impact any already-opened IndexInputs. You should call this before attempting to open an index on the directory.

NOTE: This value should be as large as possible to reduce any possible performance impact. If you still encounter an incorrect OutOfMemoryError, trying lowering the chunk size.

public void sync (String name)

Ensure that any writes to this file are moved to stable storage. Lucene uses this to properly commit changes to the index, to prevent a machine/OS crash from corrupting the index.

Throws
IOException

public String toString ()

For debug output.

public void touchFile (String name)

Set the modified time of an existing file to now.

Protected Methods

protected final void initOutput (String name)

Initializes the directory to create a new file with the given name. This method should be used in createOutput(String).

Throws
IOException