基于ambari 2.6.2.0的docker镜像安装大数据服务组件_ambari agent 2.6.2.0 百度云

之前有使用docker搭建了一套Apache Hadoop版本的大数据平台，参考我的码云地址：https://gitee.com/elbertmalone/HadoopHACluster，整个编写脚本和搭建过程花了很多时间，且灵活性不好。Ambari在大数据集群部署方面有得天独厚的优势，但是集群操作系统安装准备工作以及基础包的安装还是需要花费很多的时间。为了节省大数据集群的部署时间接下来我们用Docker容器化的方案部署Ambari。

费话少说，放码出来。让我们开始吧！

Ambari的架构

从Ambari的架构主要有两个组件：Ambari Server和Ambari Agent。

编写CentOS7支持Systemd镜像Dockfile

FROM centos:7
 
MAINTAINER elbert.lau
 
# 2, Setting up systemd
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
 
VOLUME [ "/sys/fs/cgroup" ]
ENTRYPOINT ["/usr/sbin/init"]
 
RUN yum install -y systemd* && yum clean all

构建ambari-systemd镜像

#!/bin/bash
 
docker rmi local/ambari-systemd:2.6.2.0-155
docker build --rm -t local/ambari-systemd:2.6.2.0-155 .

编写Ambari Repos镜像Dockerfile

FROM local/ambari-systemd:2.6.2.0-155
 
MAINTAINER elbert.lau
 
RUN yum -y install httpd; yum clean all; systemctl enable httpd.service
 
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari-2.6.2.0-centos7.tar.gz /var/www/html
 
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz /var/www/html
 
ADD http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz /var/www/html/HDP-UTILS
 
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz /var/www/html
 
EXPOSE 80

构建ambari-repos镜像

#!/bin/bash
 
docker rmi local/ambari-repos:2.6.2.0-155
docker build --rm -t local/ambari-repos:2.6.2.0-155 .

编写Ambari Base镜像Dockerfile

FROM local/ambari-systemd:2.6.2.0-155
 
MAINTAINER elbert.lau
 
# 1, epel-release install
RUN yum install epel-release -y && yum clean all
RUN yum update -y && yum clean all
 
RUN yum install -y yum-utils yum-plugin-ovl tar git curl bind-utils unzip wget && yum clean all
 
# 2, Setup sshd
RUN yum install -y openssh-server openssh-clients && yum clean all
RUN systemctl enable sshd
 
# 3, kerberos client
RUN yum install -y krb5-workstation && yum clean all
 
# 4, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum install -y initscripts && yum clean all
 
RUN curl -o /usr/bin/jq http://stedolan.github.io/jq/download/linux64/jq && chmod +x /usr/bin/jq
 
ENV JDK_ARTIFACT jdk-8u112-linux-x64.tar.gz
ENV JDK_VERSION jdk1.8.0_112
 
# 5, install Ambari specified 1.8 jdk
RUN mkdir /usr/jdk64 && cd /usr/jdk64 && wget http://public-repo-1.hortonworks.com/ARTIFACTS/$JDK_ARTIFACT \
&& tar -xf $JDK_ARTIFACT && rm -f $JDK_ARTIFACT
 
ENV JAVA_HOME /usr/jdk64/$JDK_VERSION
ENV PATH $PATH:$JAVA_HOME/bin
 
# 6, jce
ADD http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip $JAVA_HOME/jre/lib/security/
RUN cd $JAVA_HOME/jre/lib/security && unzip jce_policy-8.zip \
&& rm -f jce_policy-8.zip && mv UnlimitedJCEPolicyJDK8/*jar . && rm -rf UnlimitedJCEPolicyJDK8
 
ADD etc/yum.conf /etc/yum.conf
 
# 7, for ERROR: SSLError: Failed to connect. Please check openssl library versions.
RUN sed -i 's/verify=platform_default/verify=disable/' /etc/python/cert-verification.cfg

构建ambari-base镜像

#!/bin/bash
 
docker rmi local/ambari-base:2.6.2.0-155
docker build --rm -t local/ambari-base:2.6.2.0-155 .

编写Ambari Server镜像Dockerfile

FROM local/ambari-base:2.6.2.0-155
 
MAINTAINER elbert.lau
 
# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo
 
# 2, initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start postgresql
RUN yum -y install ambari-server; yum clean all
 
# 3, add psql connectors to ambari-server so it can be downloaded by services (e.g.: Ranger)
ADD https://jdbc.postgresql.org/download/postgresql-42.2.5.jar /var/lib/ambari-server/resources/postgres-jdbc-driver.jar
 
RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf
 
# 4, ambari server setup
RUN ambari-server setup --silent --java-home $JAVA_HOME; systemctl enable ambari-server.service
 
EXPOSE 8080

构建ambari-server镜像

#!/bin/bash
 
docker rmi local/ambari-server:2.6.2.0-155
docker build --rm -t local/ambari-server:2.6.2.0-155 .

编写Ambari Agent镜像Dockerfile

FROM local/ambari-base:2.6.2.0-155
 
MAINTAINER elbert.lau
 
# 1, add ambari repo
ADD http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari.repo /etc/yum.repos.d/ambari.repo
ADD http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo /etc/yum.repos.d/hdp.repo
ADD http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/hdp.gpl.repo /etc/yum.repos.d/hdp.gpl.repo
 
# 2, install ambari agent
RUN yum -y install ambari-agent; yum clean all
 
RUN echo DefaultEnvironment=\"JAVA_HOME=$JAVA_HOME\" >> /etc/systemd/system.conf

构建ambari-agent镜像

#!/bin/bash
 
docker rmi local/ambari-agent:2.6.2.0-155
docker build --rm -t local/ambari-agent:2.6.2.0-155 .

参考资料

【1】https://github.com/sequenceiq/docker-ambari