赞
踩
一、apache-dolphinscheduler介绍
Apache DolphinScheduler 是一个分布式的开源调度系统,它提供了一种可视化、可扩展、高可用的任务调度和数据处理方案。
二、apache-dolphinscheduler下载
2.1、下载apache-dolphinscheduler
https://dolphinscheduler.apache.org/en-us/download/3.2.0
三、apache-dolphinscheduler配置
3.1、配置host,配置免密
3.2、进入/usr/local/bigdata/apache-dolphinschedulerW/bin/env
3.3、修改dolphinscheduler_env.sh
- export JAVA_HOME=${JAVA_HOME:-/usr/local/jdk1.8.0_391}
- export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-node4:2181}
- export DATABASE=${DATABASE:-mysql}
- export SPRING_PROFILES_ACTIVE=${DATABASE}
- export SPRING_DATASOURCE_URL="jdbc:mysql://192.168.1.103:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true"
- export SPRING_DATASOURCE_USERNAME=root
- export SPRING_DATASOURCE_PASSWORD=123456
3.4、修改install_env.sh
- #
- # Licensed to the Apache Software Foundation (ASF) under one or more
- # contributor license agreements. See the NOTICE file distributed with
- # this work for additional information regarding copyright ownership.
- # The ASF licenses this file to You under the Apache License, Version 2.0
- # (the "License"); you may not use this file except in compliance with
- # the License. You may obtain a copy of the License at
- #
- # http://www.apache.org/licenses/LICENSE-2.0
- #
- # Unless required by applicable law or agreed to in writing, software
- # distributed under the License is distributed on an "AS IS" BASIS,
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- # See the License for the specific language governing permissions and
- # limitations under the License.
- #
-
- # ---------------------------------------------------------
- # INSTALL MACHINE
- # ---------------------------------------------------------
- # A comma separated list of machine hostname or IP would be installed DolphinScheduler,
- # including master, worker, api, alert. If you want to deploy in pseudo-distributed
- # mode, just write a pseudo-distributed hostname
- # Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5"
- ips=${ips:-"node4"}
-
- # Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine
- # modify it if you use different ssh port
- sshPort=${sshPort:-"22"}
-
- # A comma separated list of machine hostname or IP would be installed Master server, it
- # must be a subset of configuration `ips`.
- # Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2"
- masters=${masters:-"node4"}
-
- # A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a
- # subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts
- # Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default"
- workers=${workers:-"node4:default"}
-
- # A comma separated list of machine hostname or IP would be installed Alert server, it
- # must be a subset of configuration `ips`.
- # Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3"
- alertServer=${alertServer:-"node4"}
-
- # A comma separated list of machine hostname or IP would be installed API server, it
- # must be a subset of configuration `ips`.
- # Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1"
- apiServers=${apiServers:-"node4"}
-
- # The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists.
- # Do not set this configuration same as the current path (pwd). Do not add quotes to it if you using related path.
- installPath=${installPath:-"/tmp/dolphinscheduler"}
-
- # The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh`
- # script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs
- # to be created by this user
- deployUser=${deployUser:-"root"}
-
- # The root of zookeeper, for now DolphinScheduler default registry server is zookeeper.
- # It will delete ${zkRoot} in the zookeeper when you run install.sh, so please keep it same as registry.zookeeper.namespace in yml files.
- # Similarly, if you want to modify the value, please modify registry.zookeeper.namespace in yml files as well.
- zkRoot=${zkRoot:-"/dolphinscheduler"}
3.5、执行install.sh
./install.sh
3.6、启动zookeeper
3.7、启动结果
- node4:default
- node4 master server is starting
- Begin start master-server......
- starting master-server, logging to /tmp/dolphinscheduler/master-server/logs
- Overwrite master-server/conf/dolphinscheduler_env.sh using bin/env/dolphinscheduler_env.sh.
- End start master-server.
- node4 worker server is starting
- Begin start worker-server......
- starting worker-server, logging to /tmp/dolphinscheduler/worker-server/logs
- Overwrite worker-server/conf/dolphinscheduler_env.sh using bin/env/dolphinscheduler_env.sh.
- End start worker-server.
- Begin start alert-server......
- starting alert-server, logging to /tmp/dolphinscheduler/alert-server/logs
- Overwrite alert-server/conf/dolphinscheduler_env.sh using bin/env/dolphinscheduler_env.sh.
- End start alert-server.
- node4 api server is starting
- Begin start api-server......
- starting api-server, logging to /tmp/dolphinscheduler/api-server/logs
- Overwrite api-server/conf/dolphinscheduler_env.sh using bin/env/dolphinscheduler_env.sh.
- End start api-server.
- query server status
-
-
- ====================== dolphinscheduler server config =============================
- 1.dolphinscheduler server node config hosts:[ node4 ]
- 2.master server node config hosts:[ node4 ]
- 3.worker server node config hosts:[ node4:default ]
- 4.alert server node config hosts:[ node4 ]
- 5.api server node config hosts:[ node4 ]
-
-
- ====================== dolphinscheduler server status =============================
- node server state
-
-
- node4 Begin status master-server......
- master-server [ RUNNING ]
- End status master-server.
- node4 Begin status worker-server......
- worker-server [ RUNNING ]
- End status worker-server.
- node4 Begin status alert-server......
- alert-server [ RUNNING ]
- End status alert-server.
- node4 Begin status api-server......
- api-server [ RUNNING ]
- End status api-server.
- [root@node4 bin]# jps
- 61345 WorkerServer
- 61395 AlertServer
- 61305 MasterServer
- 61452 ApiApplicationServer
- 61692 Jps
- 55391 QuorumPeerMain
3.8、启动伪集群
./start-all.sh
3.9、查看启动进程
- [root@node4 bin]# jps
- 61345 WorkerServer
- 64500 MasterServer
- 61452 ApiApplicationServer
- 64557 AlertServer
- 66396 Jps
- 55391 QuorumPeerMain
3.10、关闭伪集群
./stop-all.sh
- [root@node4 bin]# ./stop-all.sh
- node4:default
- node4 master server is stopping
- Begin stop master-server......
- stopping master-server
- End stop master-server.
- node4 worker server is stopping
- Begin stop worker-server......
- stopping worker-server
- End stop worker-server.
- Begin stop alert-server......
- stopping alert-server
- End stop alert-server.
- node4 api server is stopping
- Begin stop api-server......
- stopping api-server
- End stop api-server.
四、apache-dolphinscheduler启动
http://192.168.42.142:12345/dolphinscheduler/ui
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。