public class IgniteHadoopFileSystem extends AbstractFileSystem implements Closeable
IGFS Hadoop 2.x file system driver over file system API. To use
IGFS as Hadoop file system, you should configure this class
in Hadoop's core-site.xml as follows:
<property>
<name>fs.default.name</name>
<value>igfs://ipc</value>
</property>
<property>
<name>fs.igfs.impl</name>
<value>org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem</value>
</property>
You should also add Ignite JAR and all libraries to Hadoop classpath. To
do this, add following lines to conf/hadoop-env.sh script in Hadoop
distribution:
export IGNITE_HOME=/path/to/Ignite/distribution export HADOOP_CLASSPATH=$IGNITE_HOME/ignite*.jar for f in $IGNITE_HOME/libs/*.jar; do export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f; done
job-submitter or job-scheduler processes, while data nodes are usually
started together with Hadoop task-tracker processes.
For sample client and data node configuration refer to config/hadoop/default-config-client.xml
and config/hadoop/default-config.xml configuration files in Ignite installation.
statistics| Constructor and Description |
|---|
IgniteHadoopFileSystem(URI name,
Configuration cfg) |
| Modifier and Type | Method and Description |
|---|---|
void |
checkPath(Path path) |
void |
close() |
void |
colocateFileWrites(boolean colocateFileWrites)
Public setter that can be used by direct users of FS or Visor.
|
FSDataOutputStream |
createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission perm,
int bufSize,
short replication,
long blockSize,
Progressable progress,
Options.ChecksumOpt checksumOpt,
boolean createParent) |
boolean |
delete(Path f,
boolean recursive) |
BlockLocation[] |
getFileBlockLocations(Path path,
long start,
long len) |
FileChecksum |
getFileChecksum(Path f) |
FileStatus |
getFileStatus(Path f) |
FsStatus |
getFsStatus() |
FsServerDefaults |
getServerDefaults() |
URI |
getUri() |
int |
getUriDefaultPort() |
FileStatus[] |
listStatus(Path f) |
void |
mkdir(Path f,
FsPermission perm,
boolean createParent) |
IgfsMode |
mode(Path path)
Resolve path mode.
|
FSDataInputStream |
open(Path f,
int bufSize) |
void |
renameInternal(Path src,
Path dst) |
void |
setOwner(Path p,
String usr,
String grp) |
void |
setPermission(Path p,
FsPermission perm) |
boolean |
setReplication(Path f,
short replication) |
void |
setTimes(Path f,
long mtime,
long atime) |
void |
setVerifyChecksum(boolean verifyChecksum) |
boolean |
supportsSymlinks() |
String |
toString() |
String |
user()
Returns the user name this File System is created on behalf of.
|
access, checkScheme, clearStatistics, create, createFileSystem, createSymlink, equals, get, getAclStatus, getAllStatistics, getCanonicalServiceName, getDelegationTokens, getFileLinkStatus, getFsStatus, getHomeDirectory, getInitialWorkingDirectory, getLinkTarget, getStatistics, getStatistics, getUriPath, getXAttr, getXAttrs, getXAttrs, hashCode, isValidName, listCorruptFileBlocks, listLocatedStatus, listStatusIterator, listXAttrs, makeQualified, modifyAclEntries, open, printStatistics, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameInternal, resolvePath, setAcl, setXAttr, setXAttr, truncatepublic IgniteHadoopFileSystem(URI name, Configuration cfg) throws URISyntaxException, IOException
name - URI for file system.cfg - Configuration.URISyntaxException - if name has invalid syntax.IOException - If initialization failed.public void checkPath(Path path)
checkPath in class AbstractFileSystempublic void colocateFileWrites(boolean colocateFileWrites)
colocateFileWrites - Whether all ongoing file writes should be colocated.public void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableIOExceptionpublic URI getUri()
getUri in class AbstractFileSystempublic int getUriDefaultPort()
getUriDefaultPort in class AbstractFileSystempublic FsServerDefaults getServerDefaults() throws IOException
getServerDefaults in class AbstractFileSystemIOExceptionpublic boolean setReplication(Path f, short replication) throws IOException
setReplication in class AbstractFileSystemIOExceptionpublic void setTimes(Path f, long mtime, long atime) throws IOException
setTimes in class AbstractFileSystemIOExceptionpublic FsStatus getFsStatus() throws IOException
getFsStatus in class AbstractFileSystemIOExceptionpublic void setPermission(Path p, FsPermission perm) throws IOException
setPermission in class AbstractFileSystemIOExceptionpublic void setOwner(Path p, String usr, String grp) throws IOException
setOwner in class AbstractFileSystemIOExceptionpublic FSDataInputStream open(Path f, int bufSize) throws IOException
open in class AbstractFileSystemIOExceptionpublic FSDataOutputStream createInternal(Path f, EnumSet<CreateFlag> flag, FsPermission perm, int bufSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt, boolean createParent) throws IOException
createInternal in class AbstractFileSystemIOExceptionpublic boolean supportsSymlinks()
supportsSymlinks in class AbstractFileSystempublic void renameInternal(Path src, Path dst) throws IOException
renameInternal in class AbstractFileSystemIOExceptionpublic boolean delete(Path f, boolean recursive) throws IOException
delete in class AbstractFileSystemIOExceptionpublic void setVerifyChecksum(boolean verifyChecksum)
throws IOException
setVerifyChecksum in class AbstractFileSystemIOExceptionpublic FileChecksum getFileChecksum(Path f) throws IOException
getFileChecksum in class AbstractFileSystemIOExceptionpublic FileStatus[] listStatus(Path f) throws IOException
listStatus in class AbstractFileSystemIOExceptionpublic void mkdir(Path f, FsPermission perm, boolean createParent) throws IOException
mkdir in class AbstractFileSystemIOExceptionpublic FileStatus getFileStatus(Path f) throws IOException
getFileStatus in class AbstractFileSystemIOExceptionpublic BlockLocation[] getFileBlockLocations(Path path, long start, long len) throws IOException
getFileBlockLocations in class AbstractFileSystemIOExceptionpublic IgfsMode mode(Path path)
path - HDFS path.public String user()
Follow @ApacheIgnite
Ignite Fabric : ver. 1.5.11 Release Date : April 8 2016