人工智能uu

这个屌丝很懒，什么也没留下！

热门标签

ambari容器化离线部署

作者：人工智能uu | 2024-06-21 14:20:15

踩

ambari容器

离线yum源构建

离线centos7 yum源构建

1、下载镜像

    1）镜像地址：http://mirrors.aliyun.com/centos/7/isos/x86_64/
        下载文件：CentOS-7-x86_64-Everything-2009.iso
        http://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-2009.iso
    2）镜像说明
        打开centos官网，里面主要是以下几块内容
        1.DVD ISO ： 最新版本基本版下载，带有常用的软件，无需自己下载安装（通常情况下），一般安装的是这个版本
        2.Everything ISO ：最新版本完整版，带有大部分软件，无需自己下载安装（通常情况下），体积最大
        3.Minimal ISO：精简版，带有必备软件，其他软件需要自己下载安装，但是体积最小
        4.Older Versions：过往版本系统下载
        5.Need the Source：源码下载
1
2
3
4
5
6
7
8
9
10

2、挂载镜像文件

    mkdir -p /mnt/centos7/
    mount -o loop CentOS-7-x86_64-Everything-2009.iso /mnt/centos7/
1
2

3、拉取官网Tomcat(也可以使用httpd)

    docker pull tomcat:jdk8-adoptopenjdk-openj9
1

4、启动容器centos7

    docker run --name centos7 -itd -p 18080:8080 tomcat:jdk8-adoptopenjdk-openj9
1

5、复制yum源到容器

    docker cp /mnt/centos7 centos7:/usr/local/tomcat/webapps
1

6、构建centos7镜像

    docker commit -m="centos yum repos" -a="pony" centos7 pony/centos7-yumrepos:v1
1

7、导出centos7镜像

    docker save -o centos7.tar pony/centos7-yumrepos:v1
1

离线ambari 和 hdp yum源构建与centos7 类似

参考：
    离线ambari 和hdp yum 源下载
        https://blog.csdn.net/ZYC88888/article/details/103009119
    离线方式部署Ambari2.6.0.0
        https://www.cnblogs.com/yinzhengjie/p/9631920.html
1
2
3
4
5

ambari 镜像构建

依赖的资源

mysql-connector-java-5.1.49.jar

配置文件

nginx.conf

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /data/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
  worker_connections 1024;
}

http {
  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  '$status $body_bytes_sent "$http_referer" '
  '"$http_user_agent" "$http_x_forwarded_for"';

  access_log  /data/nginx/access.log  main;

  sendfile            on;
  tcp_nopush          on;
  tcp_nodelay         on;
  keepalive_timeout   65;
  types_hash_max_size 2048;

  include             /etc/nginx/mime.types;
  default_type        application/octet-stream;

  include /etc/nginx/conf.d/*.conf;

  client_max_body_size          2048m;
  server_names_hash_bucket_size   128;
  client_header_buffer_size       32k;
  large_client_header_buffers   4 32k;

  #server {
  #    listen       80 default_server;
  #    listen       [::]:80 default_server;
  #    server_name  _;
  #    root         /usr/share/nginx/html;
  #
  #    # Load configuration files for the default server block.
  #    include /etc/nginx/default.d/*.conf;
  #
  #    location / {
  #    }
  #
  #    error_page 404 /404.html;
  #    location = /404.html {
  #    }
  #
  #    error_page 500 502 503 504 /50x.html;
  #    location = /50x.html {
  #    }
  #}

  # Settings for a TLS enabled server.
  #
  #    server {
  #        listen       443 ssl http2 default_server;
  #        listen       [::]:443 ssl http2 default_server;
  #        server_name  _;
  #        root         /usr/share/nginx/html;
  #
  #        ssl_certificate "/etc/pki/nginx/server.crt";
  #        ssl_certificate_key "/etc/pki/nginx/private/server.key";
  #        ssl_session_cache shared:SSL:1m;
  #        ssl_session_timeout  10m;
  #        ssl_ciphers HIGH:!aNULL:!MD5;
  #        ssl_prefer_server_ciphers on;
  #
  #        # Load configuration files for the default server block.
  #        include /etc/nginx/default.d/*.conf;
  #
  #        location / {
  #        }
  #
  #        error_page 404 /404.html;
  #        location = /404.html {
  #        }
  #
  #        error_page 500 502 503 504 /50x.html;
  #        location = /50x.html {
  #        }
  #    }

}


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

init.repo

#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
name=ambari Version - ambari-2.7.3.0
baseurl=http://192.168.0.1:18080/ambari2.7.3/centos7/2.7.3.0-139/
gpgcheck=1
gpgkey=http://192.168.0.1:18080/ambari2.7.3/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

#centos7
[CentOS-7-x86_64-Everything-2009]
name=CentOS-7-x86_64-Everything-2009
baseurl=http://192.168.0.1:18080/centos7
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled=1
priority=1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

yum.conf

[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
override_install_langs=en_US.UTF-8
tsflags=nodocs

exclude=systemd* 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

Dockerfile

FROM centos:7
MAINTAINER pony

# install nginx
RUN yum install -y epel-release && yum -y install nginx && yum clean all
ADD ../nginx.conf /etc/nginx/nginx.conf

# yum repos config && install ambari-server & ambari-agent
COPY ../mysql-connector-java-5.1.49.jar /opt/
COPY ../init.repo /etc/yum.repos.d/init.repo
RUN yum -y install ambari-server; \
    yum -y install ambari-agent; \
    yum clean all; \
    sed -i  "s#ambari.root.dir=.*#ambari.root.dir=/data/ambari-server#g" /etc/ambari-server/conf/log4j.properties; \
    sed -i  "s#hostname=.*#hostname=hdp01#g" /etc/ambari-agent/conf/ambari-agent.ini; \
    sed -i  "s#logdir=.*#logdir=/data/ambari-agent/log#g" /etc/ambari-agent/conf/ambari-agent.ini

# Configure the offline YUM source
RUN mv /etc/yum.repos.d /etc/yum.repos.d.bak && mkdir /etc/yum.repos.d
COPY ../yum.conf /etc/yum.conf
COPY ../init.repo /etc/yum.repos.d/init.repo

#Setting up systemd
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME ["/sys/fs/cgroup"]
ENTRYPOINT ["/usr/sbin/init"]

RUN yum install -y systemd* && yum clean all

#RUN yum install -y yum-utils yum-plugin-ovl tar git curl bind-utils unzip wget sudo net-tools && yum clean all
RUN yum install -y curl unzip wget sudo net-tools nc && yum clean all

# Setup sshd
RUN yum install -y openssh-server openssh-clients && yum clean all
RUN systemctl enable sshd

# install mysql client
RUN yum install -y mariadb.x86_64 mariadb-libs.x86_64 && yum clean all

ENV JDK_ARTIFACT jdk-8u112-linux-x64.tar.gz
ENV JDK_VERSION jdk1.8.0_112

# install Ambari specified 1.8 jdk
RUN mkdir /usr/jdk64 && cd /usr/jdk64 && wget http://public-repo-1.hortonworks.com/ARTIFACTS/$JDK_ARTIFACT && \
    tar -xf $JDK_ARTIFACT && rm -f $JDK_ARTIFACT
ENV JAVA_HOME /usr/jdk64/$JDK_VERSION
ENV PATH $PATH:$JAVA_HOME/bin

# jce
ADD http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip $JAVA_HOME/jre/lib/security/
RUN cd $JAVA_HOME/jre/lib/security && unzip jce_policy-8.zip && rm -f jce_policy-8.zip && mv UnlimitedJCEPolicyJDK8/*jar . && rm -rf UnlimitedJCEPolicyJDK8

# limits config
RUN echo "* soft memlock unlimited" >> /etc/security/limits.conf; \
    echo "* hard memlock unlimited" >> /etc/security/limits.conf; \
    echo "* soft nofile 65535"      >> /etc/security/limits.conf; \
    echo "* hard nofile 65535"      >> /etc/security/limits.conf; \
    echo "* soft nproc unlimited"   >> /etc/security/limits.conf; \
    echo "* hard nproc unlimited"   >> /etc/security/limits.conf

# set the time zone
RUN rm -f /etc/localtime; ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

VOLUME [ "/data","/data/nginx" ]

EXPOSE 8080 8088 50070 19888 18081

#RUN rm -f /etc/yum.repos.d/*

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

构建镜像

docker build -t ambri:1.0 ./

编排文件

配置文件

ambari.repo

#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
name=ambari Version - ambari-2.7.3.0
baseurl=http://yumrepos:8080/ambari2.7.3/centos7/2.7.3.0-139/
gpgcheck=1
gpgkey=http://yumrepos:8080/ambari2.7.3/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

1
2
3
4
5
6
7
8
9

centos7.repo

#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
name=ambari Version - ambari-2.7.3.0
baseurl=http://yumrepos:8080/ambari2.7.3/centos7/2.7.3.0-139/
gpgcheck=1
gpgkey=http://yumrepos:8080/ambari2.7.3/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

1
2
3
4
5
6
7
8
9

proxy.conf

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /data/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /data/nginx/access.log  main;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    include /etc/nginx/conf.d/*.conf;

    #正向代理配置
    server{
        listen 80;   #监听端口 
        resolver 8.8.8.8;   #DNS
        resolver_timeout 10s;  # DNS解析超时时间
        location / {
            proxy_pass http://$http_host$request_uri;
            proxy_set_header Host $http_host;
            proxy_buffers 256 4k;
            proxy_max_temp_file_size 0;
            proxy_connect_timeout 30;
            proxy_cache_valid 200 302 10m;
            proxy_cache_valid 301 1h;
            proxy_cache_valid any 1m;
        }
    }
    
    #==========反向代理配置====================

    # ambari-server UI
    server {
        listen       8080;# 访问端口(自行修改)
        server_name  localhost;
        charset utf-8;
        #access_log  /var/log/nginx/yarn-access.log  main;
        #error_log /var/log/nginx/yarn-error.log;
        location / {
            proxy_pass http://hdp01:8080;
        }

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
    
    # yarn ResourceManager UI
    server {
        listen       8088;# 访问端口(自行修改)
        server_name  localhost;
        charset utf-8;
        #access_log  /var/log/nginx/yarn-access.log  main;
        #error_log /var/log/nginx/yarn-error.log;
        location / {
            proxy_pass http://hdp02:8088;
        }
        
        error_page 404 /404.html;
        location = /404.html {
        }
        
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
    #NameNode UI
    upstream namenodeui {
        ip_hash;
        server hdp02:50070;
        server hdp03:50070;
        
    }
    server {
        listen       50070;# 访问端口(自行修改)
        server_name  localhost;
        charset utf-8;
        location / {
            proxy_pass http://namenodeui;
        }
        
        error_page 404 /404.html;
        location = /404.html {
        }
        
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
    #JobHistory UI
    server {
        listen       19888;# 访问端口(自行修改)
        server_name  localhost;
        charset utf-8;
        location / {
            proxy_pass http://hdp03:19888;
        }
        
        error_page 404 /404.html;
        location = /404.html {
        }
        
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
     
    #Spark2 History Server UI
    server {
        listen       18081;# 访问端口(自行修改)
        server_name  localhost;
        charset utf-8;
        location / {
            proxy_pass http://hdp02:18081;
        }
        
        error_page 404 /404.html;
        location = /404.html {
        }
        
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
}


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

编排文件

docker-compose.yml

version: '3'
services:

  yumrepos:
    image: pony/all-yumrepos:v1
    container_name: yumrepos
    hostname: yumrepos
    restart: always
    deploy:
      resources:
        limits:
          cpus: '1.00'
          memory: 1G
        reservations:
          cpus: '0.25'
          memory: 100M
    ports:
      - "18080:8080"
    networks:
      bdpnet:
        ipv4_address: 192.168.1.101

  mysql:
    image: mysql:5.7
    container_name: mysql
    hostname: mysql
    restart: always
    deploy:
      resources:
        limits:
          cpus: '1.00'
          memory: 2G
        reservations:
          cpus: '0.25'
          memory: 100M
    volumes:
      #此处为映射，无空格
      - /data/bdp/mysql:/var/lib/mysql:rw
    environment:
      MYSQL_ROOT_PASSWORD: xxx
    ports:
      - "3306:3306"
    networks:
      bdpnet:
        ipv4_address: 192.168.1.102

  proxy:
    image: pony/bdp:v1
    container_name: proxy
    hostname: proxy
    privileged: false
    entrypoint: nginx
    command:
      - -g
      - 'daemon off;'
    ports:
      - 8080:8080
      - 8088:8088
      - 50070:50070
      - 19888:19888
      - 18081:18081
    volumes:
      - /data/bdp/proxy:/data/nginx
      - /data/bdp/proxy.conf:/etc/nginx/nginx.conf
    deploy:
      resources:
        limits:
          cpus: '1.00'
          memory: 1G
        reservations:
          cpus: '0.25'
          memory: 100M
    networks:
      bdpnet:
        ipv4_address: 192.168.1.103

  hdp01:
    image: pony/bdp:v1
    container_name: hdp01
    hostname: hdp01
    privileged: true
    entrypoint: /usr/sbin/init
    #ports:
    #  - 8081:8080
    volumes:
      - /data/bdp/hdp01:/data
      - /data/bdp/ambari.repo:/etc/yum.repos.d/ambari.repo
      - /data/bdp/centos7.repo:/etc/yum.repos.d/centos7.repo
    deploy:
      #mode: replicated
      #replicas: 1
      #restart_policy:
      #  condition: any
      #  max_attempts: 5
      resources:
        limits:
          cpus: '2.00'
          memory: 5G
        reservations:
          cpus: '0.25'
          memory: 100M
    depends_on:
      - mysql
      - yumrepos
    networks:
      bdpnet:
        ipv4_address: 192.168.1.11

  hdp02:
    image: pony/bdp:v1
    container_name: hdp02
    hostname: hdp02
    privileged: true
    entrypoint: /usr/sbin/init
    volumes:
      - /data/bdp/hdp02:/data
      - /data/bdp/ambari.repo:/etc/yum.repos.d/ambari.repo
      - /data/bdp/centos7.repo:/etc/yum.repos.d/centos7.repo
    deploy:
      resources:
        limits:
          cpus: '2.00'
          memory: 20G
        reservations:
          cpus: '0.25'
          memory: 200M
    depends_on:
      - mysql
      - yumrepos
    networks:
      bdpnet:
        ipv4_address: 192.168.1.12

  hdp03:
    image: pony/bdp:v1
    container_name: hdp03
    hostname: hdp03
    privileged: true
    entrypoint: /usr/sbin/init
    volumes:
      - /data/bdp/hdp03:/data
      - /data/bdp/ambari.repo:/etc/yum.repos.d/ambari.repo
      - /data/bdp/centos7.repo:/etc/yum.repos.d/centos7.repo
    deploy:
      resources:
        limits:
          cpus: '2.00'
          memory: 20G
        reservations:
          cpus: '0.25'
          memory: 200M
    depends_on:
      - mysql
      - yumrepos
    networks:
      bdpnet:
        ipv4_address: 192.168.1.13

  hdp04:
    image: pony/bdp:v1
    container_name: hdp04
    hostname: hdp04
    privileged: true
    entrypoint: /usr/sbin/init
    volumes:
      - /data/bdp/hdp04:/data
      - /data/bdp/ambari.repo:/etc/yum.repos.d/ambari.repo
      - /data/bdp/centos7.repo:/etc/yum.repos.d/centos7.repo
    deploy:
      resources:
        limits:
          cpus: '5.00'
          memory: 20G
        reservations:
          cpus: '0.25'
          memory: 200M
    depends_on:
      - mysql
      - yumrepos
    networks:
      bdpnet:
        ipv4_address: 192.168.1.14

  hdp05:
    image: pony/bdp:v1
    container_name: hdp05
    hostname: hdp05
    privileged: true
    entrypoint: /usr/sbin/init
    volumes:
      - /data/bdp/hdp05:/data
      - /data/bdp/ambari.repo:/etc/yum.repos.d/ambari.repo
      - /data/bdp/centos7.repo:/etc/yum.repos.d/centos7.repo
    deploy:
      resources:
        limits:
          cpus: '5.00'
          memory: 20G
        reservations:
          cpus: '0.25'
          memory: 200M
    depends_on:
      - mysql
      - yumrepos
    networks:
      bdpnet:
        ipv4_address: 192.168.1.15

#networks:
#  default:
#    external:
#      name: ambari

networks:
  bdpnet:
    external: true
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217

ambari容器化部署文档

创建网络

docker network create -d bridge --subnet 192.168.1.0/24 --gateway 192.168.1.1 bdpnet 
1

数据库配置

root密码为构建容器时传入的密码
mysql -hmysql -uroot -P3306 -pxxx

ambari元数据
        DROP DATABASE ambari;
        CREATE DATABASE ambari DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
        GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%' IDENTIFIED BY 'xxx' WITH GRANT OPTION;
        flush privileges;
hive元数据
        DROP DATABASE hive;
        CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
        GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%' IDENTIFIED BY 'xxx' WITH GRANT OPTION;
        flush privileges;
初始化ambari数据库：
    use ambari;
    set names utf8;
    source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

ambari-server 配置

ambari-server setup
JAVA_HOME: /usr/jdk64/jdk1.8.0_112/
jdbc driver path: /opt/mysql-connector-java-5.1.49.jar

mv /usr/share/java /usr/share/java.bak;mkdir /usr/share/java
cp /opt/mysql-connector-java-5.1.49.jar /usr/share/java
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-5.1.49.jar
ambari-server start
1
2
3
4
5
6
7
8

ambari-agent 配置

如果ambari-server 主机名有变化，这个需要配置
ambari-agent start
1
2

ambari webui 设置

1.设置一个合适的hadoop集群名称

2.选择hdp版本

选择3.0版本
1

3.yum 源配置

# ambari-hdp-1.repo
[HDP-3.0-repo-1]
name=HDP-3.0-repo-1
baseurl=http://yumrepos:8080/HDP-3.0.1.0/centos7/3.0.1.0-187

path=/
enabled=1
gpgcheck=0
[HDP-3.0-GPL-repo-1]
name=HDP-3.0-GPL-repo-1
baseurl=http://yumrepos:8080/HDP-3.0.1.0/centos7/3.0.1.0-187

path=/
enabled=1
gpgcheck=0
[HDP-UTILS-1.1.0.22-repo-1]
name=HDP-UTILS-1.1.0.22-repo-1
baseurl=http://yumrepos:8080/HDP-UTILS-1.1.0.22/centos7/1.1.0.22

path=/
enabled=1
gpgcheck=0

注意：
    yum源生成失败，需要在所有的ambari-agent节点删除产生的yum源文件，然后返回yum源配置页面重新配置
    rm -f /etc/yum.repos.d/ambari-hdp-1.repo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

4.确认hadoop集群主机

配置安装ambari-agent的所有节点
1

5.选择服务

选择所需要安装的服务
1

6.角色分配

合理的分配不同角色在合适的节点上
1

7.设置服务秘钥

为granfana hive等服务设置密码
1

8.配置hive元数据信息

配置hive元数据库的连接信息，在这里选择：Existing MySQL / MariaDB
1

9.配置数据目录

配置元数据、日志数据及业务数据目录等，这里需要注意数据目录需要配置到挂载卷上。
1

10. 重启hadoop集群之前需要做的设置

SPARK2 THRIFT SERVERS 需要修改如下配置，否则读取不到hive的元数据
metastore.catalog.default=spark 改为 metastore.catalog.default=hive

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/人工智能uu/article/detail/743303

ambari容器化离线部署

文章目录

离线yum源构建

离线centos7 yum源构建

1、下载镜像

2、挂载镜像文件

3、拉取官网Tomcat(也可以使用httpd)

4、启动容器centos7

5、复制yum源到容器

6、构建centos7镜像

7、导出centos7镜像

离线ambari 和 hdp yum源构建与centos7 类似

ambari 镜像构建

依赖的资源

配置文件

Dockerfile

构建镜像

编排文件

配置文件

编排文件

ambari容器化部署文档

创建网络

数据库配置

ambari-server 配置

ambari-agent 配置

ambari webui 设置

1.设置一个合适的hadoop集群名称

2.选择hdp版本

3.yum 源配置

4.确认hadoop集群主机

5.选择服务

6.角色分配

7.设置服务秘钥

8.配置hive元数据信息

9.配置数据目录

10. 重启hadoop集群之前需要做的设置