Hadoop常用操作_常用的hadoop操作

启动，关闭

D:\hadoop-2.8.3\hadoop-2.8.3\sbin>start-all.cmd

D:\hadoop-2.8.3\hadoop-2.8.3\sbin>stop-all.cmd

start-all.sh等价于start-dfs.sh + start-yarn.sh

单进程启动。

sbin/start-dfs.sh

---------------

    sbin/hadoop-daemons.sh --config .. --hostname .. start namenode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start datanode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start sescondarynamenode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start zkfc ...         //

sbin/start-yarn.sh
--------------  
    libexec/yarn-config.sh
    sbin/yarn-daemon.sh --config $YARN_CONF_DIR  start resourcemanager
    sbin/yarn-daemons.sh  --config $YARN_CONF_DIR  start nodemanager

web地址

http://localhost:50070/explorer.html#/

新增目录

创建新目录

hdfs dfs -mkdir -p /cmd_create_dir

将本地文件存储至hadoop

hdfs dfs -put ./demo_mvn.jar /test/

新建空文件

hdfs dfs -touchz /cmd_create_dir/empty.txt

某个文件重命名

hdfs dfs -mv /cmd_create_dir/empty.txt /cmd_create_dir/rename_empty.txt

修改目录

下载文件

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hadoop dfs -get /test/test.xml C:\Users\chang\Desktop\fsdownload\security

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hdfs dfs -get  /test/test.xml C:\Users\chang\Desktop\fsdownload\security

get: `C:/Users/chang/Desktop/fsdownload/security/test.xml': File exists

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hdfs dfs -get  /test/test.xml C:\Users\chang\Desktop\fsdownload\

hdfs dfs -get /cmd_create_dir C:\Users\chang\Desktop\fsdownload\

将hadoop指定目录下所有内容保存为一个文件，同时down至本地

   hdfs dfs –getmerge /user /home/t

删除目录

hadoop fs -rm -r -skipTrash /path_to_file/file_name

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hadoop fs -rm -r -skipTrash /test/word_output_json

Deleted /test/word_output_json

hadoop fs -rm -r /test/new2

将正在运行的hadoop作业kill掉

   hadoop job –kill  [job-id]

查看目录

hadoop fs -cat /test/word_output_json/part-r-00000

查看指定目录下内容

hdfs dfs -ls /

打开某个已存在文件

hdfs dfs -cat /test/test.xml

hdfs dfs -lsr /

查看文件或目录的权限：

hdfs dfs -ls -d /path/to/directory

或者使用更详细的格式：

hdfs dfs -getfacl /path/to/file_or_directory

更改文件或目录的拥有者：

hdfs dfs -chown -R new_owner:new_group /path/to/file_or_directory
这里，-R选项表示递归地更改指定路径下所有文件和目录的拥有者。

更改文件或目录的权限：

hdfs dfs -chmod -R 755 /path/to/file_or_directory

HBase常用操作

D:\hbase-2.0.2-bin\hbase-2.0.2\bin>start-hbase.cmd

D:\hbase-2.0.2-bin\hbase-2.0.2\bin>hbase shell

status

create 'testtable','coln1'

list_namespace_tables 'default'

hbase(main):011:0> create_namespace 'testnamespace'

Took 0.2850 seconds

hbase(main):012:0> create 'testnamespace:testtable', 'coln1'

Created table testnamespace:testtable

Took 0.7770 seconds

=> Hbase::Table - testnamespace:testtable

hbase(main):013:0> list

新增数据

hbase(main):023:0> put "testtable", "row1", "coln1",'zhangsan'

Took 0.1300 seconds

hbase(main):024:0> put "testtable", "row2", "coln1",'lisi'

Took 0.0020 seconds

查询数据

hbase(main):027:0> get "testtable", "row1"

COLUMN                CELL

 coln1:               timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0180 seconds

hbase(main):028:0> get "testtable", "coln1"

COLUMN                CELL

0 row(s)

Took 0.0040 seconds

hbase(main):029:0> get "testtable", "row1", "coln1"

COLUMN                CELL

 coln1:               timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0040 seconds

删除数据

delete "testtable", "row1", "coln1"

deleteall "testtable", "row1"

hbase(main):071:0> disable "testtable"

Took 0.7690 seconds

hbase(main):072:0> drop "testtable"

Took 0.2570 seconds

hbase(main):073:0> drop "testtable1"