Hadoop Distributed File System (HDFS)
HDFS holds very large amount of data and provides easier access. To store such huge data, the files are stored across multiple machines. These files are stored in redundant fashion to rescue the system from possible data losses in case of failure. HDFS also makes applications available to parallel processing.
List Command
Lists the contents of the directory specified by path, showing the names, permissions, owner, size and modification date for each entry. -lsr behaves like -ls, but recursively displays entries in all subdirectories of path.
Disk Usage Command
Shows disk usage, in bytes, for all the files which match path; filenames are reported with the full HDFS protocol prefix. -dus is like -du, but prints a summary of disk usage of all files/directories in the path.
Move Command
Moves the file or directory indicated by src to dest, within HDFS.
Copy Command
Copies the file or directory identified by src to dest, within HDFS.
Remove Command
Removes the file or empty directory identified by path. Removes the file or directory identified by path. Recursively deletes any child entries (i.e., files or subdirectories of path).
Put Command
Copies the file or directory from the local file system identified by localSrc to dest within the DFS.
Cat Command
Displays the contents of filename on stdout.
Get Command
Copies the file or directory in HDFS identified by src to the local file system path identified by localDest.
Copy and Move From Local file system Command
-copyFromLocal is dentical to -get command above. Works like -get, but deletes the HDFS copy on success.
Set Replication Command
Sets the target replication factor for files identified by path to rep. (The actual replication factor will move toward the target over time)
Tail Command
Shows the last 1KB of file on stdout.
Make Directory Command
Creates a directory named path in HDFS.
Test Command
Returns 1 if path exists; has zero length; or is a directory or 0 otherwise.
Change Command
chmod changes the file permissions associated with one or more objects identified by path. Performs changes recursively with -R. mode is a 3-digit octal mode, or {augo}+/-{rwxX}. Assumes if no scope is specified and does not apply an umask.
chown sets the owning user and/or group for files or directories identified by path. Sets owner recursively if -R is specified.
chgrp Sets the owning group for files or directories identified by path.... Sets group recursively if -R is specified.
Help Command
Returns usage information for one of the commands listed above. You must omit the leading '-' character in cmd.
HDFS holds very large amount of data and provides easier access. To store such huge data, the files are stored across multiple machines. These files are stored in redundant fashion to rescue the system from possible data losses in case of failure. HDFS also makes applications available to parallel processing.
List Command
Lists the contents of the directory specified by path, showing the names, permissions, owner, size and modification date for each entry. -lsr behaves like -ls, but recursively displays entries in all subdirectories of path.
$ hdfs dfs -ls <args> $ hdfs dfs -lsr <path>
Shows disk usage, in bytes, for all the files which match path; filenames are reported with the full HDFS protocol prefix. -dus is like -du, but prints a summary of disk usage of all files/directories in the path.
$ hdfs dfs -du <path> $ hdfs dfs -dus <path>
Moves the file or directory indicated by src to dest, within HDFS.
$ hdfs dfs -mv <src> <dest>
Copies the file or directory identified by src to dest, within HDFS.
$ hdfs dfs -cp <src> <dest>
Removes the file or empty directory identified by path. Removes the file or directory identified by path. Recursively deletes any child entries (i.e., files or subdirectories of path).
$ hdfs dfs -rm <path> $ hdfs dfs -rmr <path>
Copies the file or directory from the local file system identified by localSrc to dest within the DFS.
$ hdfs dfs -put <localSrc> <dest>
Displays the contents of filename on stdout.
$ hdfs dfs -cat <file-name>
Copies the file or directory in HDFS identified by src to the local file system path identified by localDest.
$ hdfs dfs -get /user/output/ /home/hadoop_tp/
-copyFromLocal is dentical to -get command above. Works like -get, but deletes the HDFS copy on success.
$ hdfs dfs -copyFromLocal<locSrc> <dest> $ hdfs dfs -moveFromLocal<localSrc> <dest>
Sets the target replication factor for files identified by path to rep. (The actual replication factor will move toward the target over time)
$ hdfs dfs -setrep [-R] [-w] rep <path>
Shows the last 1KB of file on stdout.
$ hdfs dfs -tail [-f] <file2name>
Creates a directory named path in HDFS.
$ hdfs dfs -mkdir dirName
Returns 1 if path exists; has zero length; or is a directory or 0 otherwise.
$ hdfs dfs -test -[ezd] <path>
chmod changes the file permissions associated with one or more objects identified by path. Performs changes recursively with -R. mode is a 3-digit octal mode, or {augo}+/-{rwxX}. Assumes if no scope is specified and does not apply an umask.
chown sets the owning user and/or group for files or directories identified by path. Sets owner recursively if -R is specified.
chgrp Sets the owning group for files or directories identified by path.... Sets group recursively if -R is specified.
$ hdfs dfs -chmod [-R] mode,mode,...<path> $ hdfs dfs -chown [-R] [owner][:[group]] <path> $ hdfs dfs -chgrp [-R] group <path>
Returns usage information for one of the commands listed above. You must omit the leading '-' character in cmd.
$ hdfs dfs -help <cmd-name>
Comments
Post a Comment