Azkaban是开源的工作流任务调度工具,起初是LinkedIn用来调度大数据任务的,这里介绍单节点Azkaban的安装。
Azkaban官网 https://azkaban.github.io
Azkaban文档 https://azkaban.readthedocs.io/en/latest
Azkaban Github https://github.com/azkaban/azkaban
编译安装
wget https://github.com/azkaban/azkaban/archive/refs/tags/4.0.0.tar.gz
mv 4.0.0.tar.gz azkaban-4.0.0.tar.gz
tar -zxf azkaban-4.0.0.tar.gz -C /opt
cd azkaban-4.0.0
./gradlew build
./gradlew installDist
编译过程输出
[root@ctkf01 azkaban-4.0.0]# ./gradlew build
Downloading https://services.gradle.org/distributions/gradle-4.6-all.zip
.....................................................................................................
Unzipping /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6-all.zip to /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg
Set executable permissions for: /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6/bin/gradle
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
Download https://plugins.gradle.org/m2/com/gradle/build-scan/com.gradle.build-scan.gradle.plugin/1.9/com.gradle.build-scan.gradle.plugin-1.9.pom
......
......
> Task :azkaban-exec-server:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.6/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 11m 18s
112 actionable tasks: 112 executed
[root@ctkf01 azkaban-4.0.0]# ./gradlew installDist
Parallel execution with configuration on demand is an incubating feature.
> Task :azkaban-web-server:restliTemplateGenerator
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
> Task :azkaban-web-server:npm_install
added 39 packages in 2.376s
BUILD SUCCESSFUL in 9s
56 actionable tasks: 14 executed, 42 up-to-date
启动Solo Server
solo server是单节点Azkaban实例,可用于测试、演示、简单调度等需求,特点是:
- 安装容易:不需要额外安装数据库。
- 启动容易:一个进程就包含了web server和executor server。
- 全功能:包含所有Azkaban功能,可以安装插件。
cd azkaban-solo-server/build/install/azkaban-solo-server
bin/start-solo.sh
# 浏览器访问
http://localhost:8081/
用户名/密码:azkaban/azkaban
导入样例
可以把/opt/azkaban-4.0.0/az-examples目录下的3个examples下载到本地,然后通过浏览器上传上去。
[root@ctkf01 az-examples]# ls -l
total 12
drwxrwxr-x 2 root root 4096 Mar 18 03:01 conditional-flow
drwxrwxr-x 2 root root 4096 Mar 18 03:01 flow20-projects
drwxrwxr-x 2 root root 4096 Mar 18 03:01 kafka-event-based-trigger
新建一个Project,然后在这个Project中Upload刚才下载到本地的examples。
欢迎关注我的公众号“九万里大数据”,原创技术文章第一时间推送。
欢迎访问原创技术博客网站 jwldata.com,排版更清晰,阅读更爽快。