【Dolphinscheduler3.2.0】部署_dolphinscheduler3.2.0安装教程

1.准备条件

安装jdk1.8+,mysql,zookeeper(zookeeper看实际场景,单机或者集群都没问题)
网上很多，安装下就行，后边要用
1
2

2.安装dolphinscheduler

创建目录:
mkdir /mydata && cd /mydata

下载:
wget https://www.apache.org/dyn/closer.lua/dolphinscheduler/3.2.0/apache-dolphinscheduler-3.2.0-bin.tar.gz

解压:
tar -zvxf apache-dolphinscheduler-3.2.0-bin.tar.gz
1
2
3
4
5
6
7
8

3.修改配置

1.修改install_env.sh

cd /mydata/apache-dolphinscheduler-3.2.0-bin/bin/env
vi install_env.sh
// 看自己需要单机还是集群搭建,下边两种二选一


# 单机搭建
ips="10.2.1.247"
# masters="ds1,ds2"
masters="10.2.1.247"
# workers="ds3:default,ds4:default,ds5:default"
workers="10.2.1.207:default"
# alertServer="ds4"
alertServer="10.2.1.247"
# apiServers="ds5"
apiServers="10.2.1.247"


# 集群搭建,逗号拼接就行,记得到时候在对应机器把服务启起来
ips="10.2.1.247,10.2.1.248,10.2.1.249"
# masters="ds1,ds2"
masters="10.2.1.247,10.2.1.248"
# workers="ds3:default,ds4:default,ds5:default"
workers="10.2.1.248:default,10.2.1.249:default"
# alertServer="ds4"
alertServer="10.2.1.247"
# apiServers="ds5"
apiServers="10.2.1.247"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

2.修改dolphinscheduler_env.sh

cd /mydata/apache-dolphinscheduler-3.2.0-bin/bin/env
vi dolphinscheduler_env.sh


# 数据库
export DATABASE=${DATABASE:-mysql}
export SPRING_PROFILES_ACTIVE=${DATABASE}
# export SPRING_DATASOURCE_URL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler"
export SPRING_DATASOURCE_URL="jdbc:mysql://10.2.1.247:3306/dolphin?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
# export SPRING_DATASOURCE_USERNAME={user}
export SPRING_DATASOURCE_USERNAME=xxx
# export SPRING_DATASOURCE_PASSWORD={password}
export SPRING_DATASOURCE_PASSWORD=xxx

# 时区信息
# DolphinScheduler server related configuration
export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}

# zookerper
# Registry center configuration, determines the type and link of the registry center
export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
# export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}
# zookerper集群可以逗号拼接
export REGISTRY_ZOOKEEPER_CONNECT_STRING="10.2.1.247:2181"

# 下边的环境变量网上很多会让在这个地方设置
# jdk,DATAX,python,hadoop等
# 不过我试了不太清楚为啥不生效，可以不用加，上边就可以了 在安全中心-环境管理设置就行

export HADOOP_CONF_DIR=/etc/hadoop/conf
export SPARK_HOME=/opt/soft/spark
export PYTHON_LAUNCHER=/usr/local/python2.7/bin/python2.7
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_141
export HIVE_HOME=/opt/soft/hive
export FLINK_HOME=/opt/soft/flink
export DATAX_LAUNCHER=/opt/soft/datax/bin/datax.py
export YARN_CONF_DIR=/etc/hadoop/conf
export PATH=$HADOOP_HOME/bin:$SPARK_HOME/bin:$PYTHON_LAUNCHER:$JAVA_HOME/bin:$HIVE_HOME/bin:$FLINK_HOME/bin:$DATAX_LAUNCHER:$PATH
export HADOOP_CLASSPATH=`hadoop classpath`
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

4.下载mysql-connector-java-8.0.16.jar

cd /mydata
wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
mv mysql-connector-java-8.0.16.jar /mydata/apache-dolphinscheduler-3.2.0-bin/tools/libs
cp /mydata/apache-dolphinscheduler-3.2.0-bin/tools/libs/mysql-connector-java-8.0.16.jar /mydata/apache-dolphinscheduler-3.2.0-bin/master-server/libs
cp /mydata/apache-dolphinscheduler-3.2.0-bin/tools/libs/mysql-connector-java-8.0.16.jar /mydata/apache-dolphinscheduler-3.2.0-bin/worker-server/libs
cp /mydata/apache-dolphinscheduler-3.2.0-bin/tools/libs/mysql-connector-java-8.0.16.jar /mydata/apache-dolphinscheduler-3.2.0-bin/alert-server/libs
cp /mydata/apache-dolphinscheduler-3.2.0-bin/tools/libs/mysql-connector-java-8.0.16.jar /mydata/apache-dolphinscheduler-3.2.0-bin/api-server/libs
1
2
3
4
5
6
7

5.多节点配置

把/mydata/apache-dolphinscheduler-3.2.0-bin整个目录复制到其他节点就行，单机不需要操作
如果要用DATAX，最好先安装完python和DATAX再复制，具体操作在另外一篇文章，操作完再复制。如果用到clickhouse，还需要一个clickhousewriter。如果不用，直接下一步就行。

6.初始化数据库

// 创建dolphinscheduler用户
mysql -uroot -p
CREATE DATABASE dolphin;
GRANT ALL PRIVILEGES ON dolphin.* TO 'dolphin'@'%' IDENTIFIED BY 'dolphin';
FLUSH PRIVILEGES;
exit;
// 执行脚本
sh /mydata/apache-dolphinscheduler-3.2.0-bin/tools/bin/upgrade-schema.sh
1
2
3
4
5
6
7
8

7.Dolphinscheduler启动停止

// 上边(1.修改install_env.sh) 这一步怎么配置，就在哪个服务器启动对应服务

// 启动
cd /mydata/apache-dolphinscheduler-3.2.0-bin/bin
./dolphinscheduler-daemon.sh stop  master-server
./dolphinscheduler-daemon.sh stop  api-server
./dolphinscheduler-daemon.sh stop  alert-server
./dolphinscheduler-daemon.sh stop  worker-server

// 停止
cd /mydata/apache-dolphinscheduler-3.2.0-bin/bin
./dolphinscheduler-daemon.sh start  master-server
./dolphinscheduler-daemon.sh start  api-server
./dolphinscheduler-daemon.sh start  alert-server
./dolphinscheduler-daemon.sh start  worker-server

// 查看日志
tail -f /mydata/apache-dolphinscheduler-3.2.0-bin/master-server/logs/dolphinscheduler-master.log
tail -f /mydata/apache-dolphinscheduler-3.2.0-bin/api-server/logs/dolphinscheduler-api.log
tail -f /mydata/apache-dolphinscheduler-3.2.0-bin/alert-server/logs/dolphinscheduler-alert.log
tail -f /mydata/apache-dolphinscheduler-3.2.0-bin/worker-server/logs/dolphinscheduler-worker.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

8.访问

alertServer服务部署在哪,用对应IP

http://XXXX:12345/dolphinscheduler/ui/login
1
2
3

默认账号，密码： admin/dolphinscheduler123