• 欢迎关注微信公众号:九万里大数据
  • 请使用Ctrl+D收藏本站到书签栏
  • 手机也可访问本站 jwldata.com

单节点Azkaban的安装

大数据技术 九万里大数据 2年前 (2021-04-12) 1151次浏览 0个评论 扫描二维码
文章目录[隐藏]

Azkaban是开源的工作流任务调度工具,起初是LinkedIn用来调度大数据任务的,这里介绍单节点Azkaban的安装。

Azkaban官网 https://azkaban.github.io
Azkaban文档 https://azkaban.readthedocs.io/en/latest
Azkaban Github https://github.com/azkaban/azkaban

编译安装

wget https://github.com/azkaban/azkaban/archive/refs/tags/4.0.0.tar.gz

mv 4.0.0.tar.gz azkaban-4.0.0.tar.gz 

tar -zxf azkaban-4.0.0.tar.gz -C /opt

cd azkaban-4.0.0

./gradlew build

./gradlew installDist

编译过程输出

[root@ctkf01 azkaban-4.0.0]# ./gradlew build
Downloading https://services.gradle.org/distributions/gradle-4.6-all.zip
.....................................................................................................
Unzipping /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6-all.zip to /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg
Set executable permissions for: /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6/bin/gradle
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
Download https://plugins.gradle.org/m2/com/gradle/build-scan/com.gradle.build-scan.gradle.plugin/1.9/com.gradle.build-scan.gradle.plugin-1.9.pom
......
......

> Task :azkaban-exec-server:compileTestJava 
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 11m 18s
112 actionable tasks: 112 executed

单节点Azkaban的安装

[root@ctkf01 azkaban-4.0.0]# ./gradlew installDist
Parallel execution with configuration on demand is an incubating feature.

> Task :azkaban-web-server:restliTemplateGenerator 
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

> Task :azkaban-web-server:npm_install 
added 39 packages in 2.376s

BUILD SUCCESSFUL in 9s
56 actionable tasks: 14 executed, 42 up-to-date

单节点Azkaban的安装

启动Solo Server

solo server是单节点Azkaban实例,可用于测试、演示、简单调度等需求,特点是:

  • 安装容易:不需要额外安装数据库。
  • 启动容易:一个进程就包含了web server和executor server。
  • 全功能:包含所有Azkaban功能,可以安装插件。
cd azkaban-solo-server/build/install/azkaban-solo-server
bin/start-solo.sh

# 浏览器访问
http://localhost:8081/
用户名/密码:azkaban/azkaban

单节点Azkaban的安装

导入样例

可以把/opt/azkaban-4.0.0/az-examples目录下的3个examples下载到本地,然后通过浏览器上传上去。

[root@ctkf01 az-examples]# ls -l
total 12
drwxrwxr-x 2 root root 4096 Mar 18 03:01 conditional-flow
drwxrwxr-x 2 root root 4096 Mar 18 03:01 flow20-projects
drwxrwxr-x 2 root root 4096 Mar 18 03:01 kafka-event-based-trigger

新建一个Project,然后在这个Project中Upload刚才下载到本地的examples。
单节点Azkaban的安装


欢迎关注我的公众号“九万里大数据”,原创技术文章第一时间推送。
欢迎访问原创技术博客网站 jwldata.com,排版更清晰,阅读更爽快。


单节点Azkaban的安装
 


本站文章,如未注明,均为原创 | 原创文章版权归九万里大数据所有,未经许可不得转载。
本文链接:单节点Azkaban的安装
喜欢 (2)

您必须 登录 才能发表评论!