HDFS Commands Updated
HDFS Commands Updated
HDFS Commands Updated
Commands
Types of Bigdata under Big Data:
Bigdata are categorized into three different types
Data
Semi
Structured Unstructured
Structured
Structured Data
• The Hadoop fs shell command ls displays a list of the contents of a directory specified in
the path provided by the user. It shows the name, permissions, owner, size, and
modification date for each file or directories in the specified directory.
• hadoop fs -ls /
• The Hadoop fs shell command ls displays a list of the contents of a directory specified in the path
provided by the user. It shows the name, permissions, owner, size, and modification date for each
file or directories in the specified directory.
• hadoop fs -ls /
• This Hadoop fs command behaves like -ls, but recursively displays entries in all subdirectories of a
path.
Hadoop HDFS put Command
Usage:
• The Hadoop fs shell command put is similar to
the copyFromLocal, which copies files from the local
filesystem to the destination in the Hadoop filesystem.
• hadoop fs -put <localsrc> <dest>
Hadoop HDFS copyFromLocal
Command Usage:
• This command copies the file/Directory from the local file system to
HDFS. which copies files or directory from the local filesystem to the
destination in the Hadoop filesystem
• hadoop fs -copyFromLocal <localsrc> <hdfs destination>
• Hadoop HDFS copyFromLocal Command Example:
• Here in the below example, we are trying to copy the ‘test1’ file present in
the local file system to the newDataFlair directory of Hadoop.
Hadoop HDFS get Command Usage:
• Here using the tail command, we are trying to display the 1KB of
file ‘test’ present in the newDataFlair directory on the HDFS
filesystem.
• hadoop fs -tail [-f] <file>
(Key1, value1) (Key k, value k) (Key1, value1) (Key k, value k) (Key1, value1) (Key k, value k)
Reduce Reduce
Car, 1
Deer Bear River Car, 1 Car, 1 Car, 3 Bear, 2
Car Car River Car Car River Car ,1 Car, 1 Car, 3
Deer Car River River, 1 Deer, 2
River, 2
Deer, 1 Deer, 2
Deer, 1
Deer, 1
Deer Car River
Car, 1
Bear, 1
River, 1
River, 2
River, 1
Input Final
Splitting Mapping Suffling Result
Reducing