赞
踩
持续更新中…
本篇文章以实践做记录总结在Redhat上编译、安装 ambari-2.7.4 源码以及安装服务的全部过程,文章基本是涵盖了所有步骤,可依据此文章逐步编译、安装成功。
# Unzip
tar -zxvf jdk-8u151-linux-x64.tar.gz -C /opt
rm -rf jdk-8u151-linux-x64.tar.gz
# Configure environment variables
vim /etc/profile
# build:java
export JAVA_HOME=export JAVA_HOME=/opt/jdk1.8.0_151
export PATH=$PATH:$JAVA_HOME/bin
# View the installed version
source /etc/profile
java -version
# Download(archive.apache.org) wget https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz # Unzip tar -zxvf apache-maven-3.3.9-bin.tar.gz -C /opt rm -rf apache-maven-3.3.9-bin.tar.gz # Configure environment variables vim /etc/profile # build:maven export MAVEN_HOME=/opt/apache-maven-3.3.9 export PATH=$PATH:$MAVEN_HOME/bin # View the installed version source /etc/profile mvn -v
# Download(https://nodejs.org/dist/----v6.17.1) wget https://nodejs.org/dist/v6.17.1/node-v6.17.1-linux-x64.tar.gz # Unzip tar -zxvf node-v6.17.1-linux-x64.tar.gz -C /opt rm -rf node-v6.17.1-linux-x64.tar.gz # Configure environment variables vim /etc/profile # build:node export NODE_HOME=/opt/node-v6.17.1-linux-x64 export PATH=$PATH:$NODE_HOME/bin # View the installed version source /etc/profile node -v npm -v
# Find the node root directory
cd `npm root -g`
# Change Taobao source
npm config set registry https://registry.npm.taobao.org
npm info underscore
# Install brunch-1.7.20
npm install -g brunch@1.7.20
# View the installed version
brunch -V
yum install -y rpm-build gcc-c++ python-devel git
maven配置阿里云镜像源下载会更快。
# 打开maven配置文件
vim /opt/apache-maven-3.3.9/conf/settings.xml
<mirrors> <!-- Amabri build mirror source --> <mirror> <id>alimaven</id> <name>aliyun maven</name> <url>http://maven.aliyun.com/nexus/content/groups/public/</url> <mirrorOf>central</mirrorOf> </mirror> <mirror> <id>alimaven</id> <mirrorOf>central</mirrorOf> <name>aliyun maven</name> <url>http://maven.aliyun.com/nexus/content/repositories/central/</url> </mirror> <mirror> <id>nexus-hortonworks</id> <mirrorOf>*,!central</mirrorOf> <name>Nexus hortonworks</name> <url>https://repo.hortonworks.com/content/groups/public/</url> </mirror> <mirror> <id>HDPReleases</id> <name>HDP Releases</name> <url>http://repo.hortonworks.com/content/repositories/releases/</url> <mirrorOf>central</mirrorOf> </mirror> </mirrors>
# github
wget https://github.com/apache/ambari/archive/refs/tags/release-2.7.4.tar.gz
# archive(推荐使用)
wget https://archive.apache.org/dist/ambari/ambari-2.7.4/apache-ambari-2.7.4-src.tar.gz
有时候电脑ping不通github.com,可以配置github的hosts
vim /etc/hosts
140.82.113.4 github.com
# 如果需要使用HDP的3.0和3.1,可以将stacks/HDP/3.0和3.1拷贝到源码ambari-server中,放在此路径下
/opt/build/ambari/apache-ambari-2.7.4-src/ambari-server/src/main/resources/stacks/HDP
# 修改105
vim /opt/build/ambari/apache-ambari-2.7.4-src/ambari-admin/pom.xml
:105
# 修改前 和 修改后
<argument>${basedir}/src/main/resources/ui/admin-web/node_modules/bower/bin/bower</argument>
<argument>bower</argument>
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hadoop/hadoop-3.1.1.3.1.4.0-315.tar.gz
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/phoenix/phoenix-5.0.0.3.1.4.0-315.tar.gz
# 可下载
https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz
<!-- 这里需要将需要的依赖包放在自己对应的路径,然后写在这里 -->
<hbase.tar>file:///opt/soft/hbase-2.0.2.3.1.4.0-315-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.0-315</hbase.folder>
<hadoop.tar>file:///opt/soft/hadoop-3.1.1.3.1.4.0-315.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.0-315</hadoop.folder>
<grafana.folder>grafana-6.4.2</grafana.folder>
<grafana.tar>file:///opt/soft/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
<phoenix.tar>file:///opt/soft/phoenix-5.0.0.3.1.4.0-315.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.0-315</phoenix.folder>
https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.4
# 下面为官方文档 (Redhat)
wget http://www.apache.org/dist/ambari/ambari-2.7.4/apache-ambari-2.7.4-src.tar.gz (use the suggested mirror from above)
tar xfvz apache-ambari-2.7.4-src.tar.gz
cd apache-ambari-2.7.4-src
mvn versions:set -DnewVersion=2.7.4.0.0
pushd ambari-metrics
mvn versions:set -DnewVersion=2.7.4.0.0
popd
# 整体编译推荐
mvn -B clean install rpm:rpm -DnewVersion=2.7.4.0.0 -DbuildNumber=b730f30585dd67c10d3b841317100f17d4b2c5f1 -DskipTests -Drat.skip=true -Dpython.ver="python >= 2.6"
mvn -B -X -e install package rpm:rpm -DnewVersion=2.7.4.0.0 -DbuildNumber=b730f30585dd67c10d3b841317100f17d4b2c5f1 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -Preplaceurl
# 编译ambari-metrics使用(此命令server和agent 打不出rpm包)
mvn clean package -Dbuild-rpm -DskipTests
这里只取了部分编译出现的异常问题,可能还会有其他的问题,但是大多数都是和下载依赖包有关系,这里没有总结到的可以试试删除报错时候下载的包,然后重新下载试试。
# 无rpm包,比较快 mvn clean install -DskipTests -Drat.skip=true # 有rpm包,比较慢 mvn -B clean install rpm:rpm -DnewVersion=2.7.4.0.0 -DbuildNumber=b730f30585dd67c10d3b841317100f17d4b2c5f1 -DskipTests -Drat.skip=true -Dpython.ver="python >= 2.6" # -B参数:该参数表示让Maven使用批处理模式构建项目,能够避免一些需要人工参与交互而造成的挂起状态。 # 如果第一次编译报错之后,重新编译有问题,则在命令中加上 -Drat.skip=true # -X:如果看详细debug信息可加上该参数 mvn -B clean install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X # 因为会报错,重复执行mvn,把clean去掉可以快不少 mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X # 从某个点开始恢复,就不用把之前build好的再重新来一次了 # 如果报错末提示如:mvn <args> -rf :ambari-metrics-storm-sink-legacy mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X -rf :ambari-metrics-storm-sink-legacy
cd /opt/build/ambari/apache-ambari-2.7.4-src/ambari-admin
mvn -B clean install rpm:rpm -DnewVersion=2.7.4.0.0 -DbuildNumber=b730f30585dd67c10d3b841317100f17d4b2c5f1 -DskipTests -Drat.skip=true -Dpython.ver="python >= 2.6"
异常:
[ERROR] Failed to execute goal on project ambari-metrics-common: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-common:jar:2.7.4.0.0: Failed to collect dependencies at org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Failed to read artifact descriptor for org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Could not find artifact org.apache.ambari:ambari:pom:2.7.4.0.0 in nexus-hortonworks (https://repo.hortonworks.com/content/groups/public/) -> [Help 1]
解决:
可能和提示的不一样,查看上面发现有一个包下不下来,手动去下载,然后再编译
cd /tmp/phantomjs
rm -rf ./*
wget https://github.com/Medium/phantomjs/releases/download/v2.1.1//phantomjs-2.1.1-linux-x86_64.tar.bz2
异常:
.7.4/ambari-admin/src/main/resources/ui/admin-web/node_modules/phantomjs/lib/phantom [INFO] Install exited unexpectedly [ERROR] npm ERR! Linux 3.10.0-1160.45.1.el7.x86_64 [ERROR] npm ERR! argv "/opt/ambari/gitlab/ambari-2.7.4/ambari-admin/src/main/resources/ui/admin-web/node/node" "/opt/ambari/gitlab/ambari-2.7.4/ambari-admin/src/main/resources/ui/admin-web/node/node_modules/npm/bin/npm-cli.js" "install" "--unsafe-perm" [ERROR] npm ERR! node v4.5.0 [ERROR] npm ERR! npm v2.15.0 [ERROR] npm ERR! code ELIFECYCLE [ERROR] [ERROR] npm ERR! phantomjs@1.9.20 install: `node install.js` [ERROR] npm ERR! Exit status 1 [ERROR] npm ERR! [ERROR] npm ERR! Failed at the phantomjs@1.9.20 install script 'node install.js'. [ERROR] npm ERR! This is most likely a problem with the phantomjs package, [ERROR] npm ERR! not with npm itself. [ERROR] npm ERR! Tell the author that this fails on your system: [ERROR] npm ERR! node install.js [ERROR] npm ERR! You can get information on how to open an issue for this project with: [ERROR] npm ERR! npm bugs phantomjs [ERROR] npm ERR! Or if that isn't available, you can get their info via: [ERROR] npm ERR! [ERROR] npm ERR! npm owner ls phantomjs [ERROR] npm ERR! There is likely additional logging output above. [ERROR] [ERROR] npm ERR! Please include the following file with any support request: [ERROR] npm ERR! /opt/ambari/gitlab/ambari-2.7.4/ambari-admin/src/main/resources/ui/admin-web/npm-debug.log [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------
解决:
此类错误可能是包下载不对,或者存在等等,去对应目录下删掉包,再按照链接去下载,再去编译(多次尝试)
cd /tmp/phantomjs/
rm -rf ./*
注意编译中会出现这个问题,但是最后的报错不一定会出现这个问题:
注意:ambari-admin 中的路径不一样
异常:
[ERROR] Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] Saving to /tmp/npm_config_tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
解决:
cd /tmp/npm_config_tmp/phantomjs/
rm -rf ./*
wget https://github.com/Medium/phantomjs/releases/download/v2.1.1//phantomjs-2.1.1-linux-x86_64.tar.bz2
异常:
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not download Yarn: Could not download https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz: Connect to github.com:443 [github.com/20.205.243.166] failed: Connection refused (Connection refused) -> [Help 1]
解决:
mkdir -p /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./
cd /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./
wget https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz
异常:
[INFO] Node v4.5.0 is already installed.
[INFO] Installing Yarn version v0.23.2
[INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz to /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:yarn (yarn install) on project ambari-web: Failed to run task: 'yarn install --ignore-engines --pure-lockfile' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
解决:
手动安装yarn
npm install -g yarn
yarn安装失败
[INFO] --- frontend-maven-plugin:1.4:yarn (yarn install) @ ambari-web ---
[INFO] Running 'yarn install --ignore-engines --pure-lockfile' in /opt/ambari/build/apache-ambari-2.7.4-src/ambari-web
[INFO] yarn install v0.23.2
[ERROR] warning Ambari@2.4.0: No license field
[INFO] [1/4] Resolving packages...
[INFO] [2/4] Fetching packages...
[ERROR] warning fsevents@0.3.8: The platform "linux" is incompatible with this module.
[INFO] info "fsevents@0.3.8" is an optional dependency and failed compatibility check. Excluding it from installation.
[ERROR] warning fsevents@1.1.1: The platform "linux" is incompatible with this module.
[INFO] info "fsevents@1.1.1" is an optional dependency and failed compatibility check. Excluding it from installation.
[INFO] [3/4] Linking dependencies...
[INFO] [4/4] Building fresh packages...
解决方案
https://yarn.bootcss.com/docs/install/#centos-stable
# On CentOS, Fedora and RHEL, you can install Yarn via our RPM package repository.
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
# Then you can simply:
sudo yum install yarn
yarn --version
cd /opt/ambari/build/apache-ambari-2.7.4-src/ambari-web
yarn install --ignore-engines --pure-lockfile
异常:
package: [copy] Copying 36 files to /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/package/libs [copy] Copying 1 file to /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/package/libs [mkdir] Created dir: /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate [mkdir] Created dir: /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate/data [get] Getting: https://repo1.maven.org/maven2/org/apache/lucene/lucene-core/6.6.2/lucene-core-6.6.2.jar [get] To: /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate/lucene-core-6.6.2.jar [get] Error getting https://repo1.maven.org/maven2/org/apache/lucene/lucene-core/6.6.2/lucene-core-6.6.2.jar to /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate/lucene-core-6.6.2.jar [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] ambari-infra ....................................... SUCCESS [ 0.516 s] [INFO] Ambari Infra Solr Client ........................... FAILURE [02:09 min] [INFO] Ambari Infra Solr Plugin ........................... SKIPPED [INFO] Ambari Infra Manager ............................... SKIPPED [INFO] Ambari Infra Assembly .............................. SKIPPED [INFO] Ambari Infra Manager Integration Tests ............. SKIPPED # 摘取下面处理 [get] Error getting https://repo1.maven.org/maven2/org/apache/lucene/lucene-core/6.6.2/lucene-core-6.6.2.jar to /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate/lucene-core-6.6.2.jar
解决:
cd /opt/build/ambari/apache-ambari-2.7.4-src/ambari-infra/ambari-infra-solr-client/target/migrate/
wget https://repo1.maven.org/maven2/org/apache/lucene/lucene-core/6.6.2/lucene-core-6.6.2.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ambari-infra ....................................... SUCCESS [ 0.543 s]
[INFO] Ambari Infra Solr Client ........................... SUCCESS [ 6.404 s]
[INFO] Ambari Infra Solr Plugin ........................... SUCCESS [ 5.065 s]
[INFO] Ambari Infra Manager ............................... SUCCESS [04:44 min]
[INFO] Ambari Infra Assembly .............................. SUCCESS [ 1.885 s]
[INFO] Ambari Infra Manager Integration Tests ............. SUCCESS [01:26 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
异常:
[ERROR] Failed to execute goal on project ambari-agent: Could not resolve dependencies for project org.apache.ambari:ambari-agent:jar:2.7.4.0.0: Failed to collect dependencies at org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Failed to read artifact descriptor for org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Failure to find org.apache.ambari:ambari:pom:2.7.4.0.0 in https://repo.hortonworks.com/content/groups/public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus-hortonworks has elapsed or updates are forced -> [Help 1]
解决:
这个问题比较奇怪,更换仓库之后OK了。
异常:
[ERROR] Failed to execute goal on project ambari-metrics-common: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-common:jar:2.7.4.0.0: Failed to collect dependencies at org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Failed to read artifact descriptor for org.apache.ambari:ambari-utility:jar:1.0.0.0-SNAPSHOT: Could not find artifact org.apache.ambari:ambari:pom:2.7.4.0.0 in nexus-hortonworks (https://repo.hortonworks.com/content/groups/public/) -> [Help 1]
解决:
编译的两个pom.xml文件下载不了
# 无法找到此路径
[INFO] Downloading: https://repo.hortonworks.com/content/groups/public/org/apache/ambari/ambari/2.7.4.0.0/ambari-2.7.4.0.0.pom
[INFO] Downloading: http://maven.aliyun.com/nexus/content/groups/public/org/apache/ambari/ambari/2.7.4.0.0/ambari-2.7.4.0.0.pom
cd /tmp/phantomjs
rm -rf ./*
wget https://github.com/Medium/phantomjs/releases/download/v2.1.1//phantomjs-2.1.1-linux-x86_64.tar.bz2
异常:
[exec] compilation terminated.
[exec] error: command 'gcc' failed with exit status 1
解决:
未安装 python-devel
yum install python-devel
异常:
Download Ambari Grafana:
[mkdir] Created dir: /opt/ambari/apache-ambari-2.7.4-src/ambari-metrics/ambari-metrics-grafana/target/grafana
[get] Getting: file:/opt/soft/grafana-2.6.0.linux-x64.tar.gz
[get] To: /opt/ambari/apache-ambari-2.7.4-src/ambari-metrics/ambari-metrics-grafana/target/grafana/grafana.tgz
[get] Error getting file:/opt/soft/grafana-2.6.0.linux-x64.tar.gz to /opt/ambari/apache-ambari-2.7.4-src/ambari-metrics/ambari-metrics-grafana/target/grafana/grafana.tgz
解决:
本身是因为替换了四个依赖包,其中grafana的包名与代码中不一致,解决方案:因为grafana本身的连接可以下载,所以此包不替换。
<!-- 这里需要将需要的依赖包放在自己对应的路径,然后写在这里 -->
<hbase.tar>file:///opt/soft/hbase-2.0.2.3.1.4.0-315-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.0-315</hbase.folder>
<hadoop.tar>file:///opt/soft/hadoop-3.1.1.3.1.4.0-315.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.0-315</hadoop.folder>
<grafana.folder>grafana-6.4.2</grafana.folder>
<grafana.tar>file:///opt/soft/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
<phoenix.tar>file:///opt/soft/phoenix-5.0.0.3.1.4.0-315.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.0-315</phoenix.folder>
异常:
Failed to execute goal on project ambari-metrics-storm-sink-legacy: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-storm-sink-legacy:jar:2.7.5.0.0: Could not find artifact org.apache.storm:storm-core:jar:0.10.0.2.3.0.0-2557 in apache-hadoop (https://repo.hortonworks.com/content/groups/public/)
解决:
<解决办法>进入ambari-metrics,修改pom.xml
把第一个 repository 替换为 http://nexus-private.hortonworks.com/nexus/content/groups/public
这样也行
考虑篇幅的情况,这里先安装 ambari-server,安装成功后在 webUI 页面以安装 ZOOKEEPER 和 AMBARI_METRICS 为例,所有的安装包都是使用自己编译好的来装;
安装环境的配置可能会和之前的编译环境重合,如果在编译环境安装,则有些工具的安装可以忽略。
暂时先列出大纲,下面均已实现,有空再补上步骤。
主要包括:修改 hostname、配置 hosts文件、修改网络配置文件、关闭防火墙、免密登录、jdk 安装、
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。