hadoop+zookeeper+hbase伪分布式安装

基本安装步骤

安装包下载

从大数据组件下载地址下载以下组件安装包

1. hadoop-2.6.0-cdh5.6.0.tar.gz
   
2. hbase-1.0.0-cdh5.6.0.tar.gz
   
3. zookeeper-3.4.5-cdh5.6.0.tar.gz
   从java下载地址下载以下组件安装包
   

4. jdk-8u144-linux-x64.tar.gz

   java环境

5. 解压 tar zxvf jdk-8u144-linux-x64.tar.gz
6. 配置环境变量
   1. vim /etc/profile
      

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144
   export JRE_HOME=$JAVA_HOME/jre
   export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
   export PATH=.:$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
   

   1. source /etc/profile

SSH免密登陆

• 教程链接

hadoop安装

1. 解压 tar zxvf hadoop-2.6.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hadoop-2.6.0-cdh5.6.0/etc/hadoop/下
   • 修改core-site.xml
     

   <configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/cdh/hadoop/tmp</value>
    </property>
    <property>
        <name>dfs.name.dir</name>
        <value>/opt/cdh/hadoop/name</value>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://ubuntu:9000</value>
    </property>
   </configuration>
   

   • 修改mapred-site.xml
     

   <configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>ubuntu:9001</value>
    </property>
   </configuration>
   

   • 修改hdfs-site.xml

   <configuration>
    <property> 
        <name>dfs.data.dir</name>
        <value>/opt/cdh/hadoop/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
   </configuration>
   

   • 修改hadoop-env.sh
     

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144
   

3. 格式化namenode
   

hadoop-2.6.0-cdh5.6.0/bin/hadoop namenode -format 

1. 启动hdfs
   

hadoop-2.6.0-cdh5.6.0/sbin/start-all.sh

1. 验证
   • jps
   • 登录网址: http://ip:50070

zookeeper安装

1. 解压 tar zxvf zookeeper-3.4.5-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录zookeeper-3.4.5-cdh5.6.0/conf/下
   • zoo.cfg
     

   tickTime=2000
   initLimit=10
   syncLimit=5
   dataDir=/opt/cdh/zookeeper
   clientPort=2181
   

3. 启动zookeeper
   

zookeeper-3.4.5-cdh5.6.0/bin/zkServer.sh start

hbase安装

1. 解压 tar zxvf hbase-1.0.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hbase-1.0.0-cdh5.6.0/conf/下
   • hbase-env.sh
     

   export JAVA_HOME=/opt/cdh/jdk1.8.0_144
   export HBASE_MANAGES_ZK=false
   

   • regionservers
     

   ubuntu
   

   • hbase-site.xml

   <configuration>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>    
    </property>
    <property> 
        <name>hbase.rootdir</name> 
        <value>hdfs://ubuntu:9000/hbase</value> 
    </property> 
    <property> 
        <name>hbase.master</name> 
        <value>hdfs://ubuntu:60000</value> 
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>ubuntu:2181</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/opt/cdh/zookeeper</value>
    </property>
   </configuration>
   

3. 启动hbase
   hbase-1.0.0-cdh5.6.0/bin/start-hbase.sh
4. 验证
   1. jps
   2. hbase-1.0.0-cdh5.6.0/bin/hbase shell
   3. 登陆网址： http://ip:60010

IDEA连接hbase登陆

1. 新建maven项目
2. 修改pom.xml文件
   

    <repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
        </repository>
    </repositories>
 
    <dependencies>
 
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase</artifactId>
            <version>1.0.0-cdh5.6.0</version>
        </dependency>
 
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>1.0.0-cdh5.6.0</version>
        </dependency>
    </dependencies>

1. 按照百度搜索结果编写demo程序并执行

ERROR解决

1. 先确认以下地址一致
   linux下的/etc/hosts里的ip与windows下C:\Windows\System32\drivers\etc\hosts里的ip映射一致
2. 禁用Hbase自带的zookeeper
   • 修改hbase-env.sh里的export HBASE_MANAGES_ZK=false