Flink 1.12 CDH 6.3 集成

之前记录的：Flink 1.9 CDH 6.3 集成
有些下载链接可能被官方关闭了，这里介绍1.12版本集成，并把安装包下载地址换为百度网盘链接

下载安装包

链接: https://pan.baidu.com/s/112fiaaMAMMXMsyiTDh3qjg 
提取码: ar5f

安装包内容

FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha
FLINK_ON_YARN-1.12.1.jar
manifest.json

安装

第一步

将FLINK_ON_YARN-1.12.1.jar放到 /opt/cloudera/csd目录下

第二步

在我们安装httpd服务 /var/www/html目录下创建flink目录，将下面三个文件放在flink目录下

FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha
manifest.json

第三步 修改httpd服务

参考：javascript:void(0) 新增 .parcel配置

[root@localhost bigdata]# vim /etc/httpd/conf/httpd.conf

#在<IfModule mime_module>模块添加 parcel选项

<IfModule mime_module>
   #
   # TypesConfig points to the file containing the list of mappings from
   # filename extension to MIME-type.
   #
   TypesConfig /etc/mime.types

   #
   # AddType allows you to add to or override the MIME configuration
   # file specified in TypesConfig for specific file types.
   #
   #AddType application/x-gzip .tgz
   #
   # AddEncoding allows you to have certain browsers uncompress
   # information on the fly. Note: Not all browsers support this.
   #
   #AddEncoding x-compress .Z
   #AddEncoding x-gzip .gz .tgz
   #
   # If the AddEncoding directives above are commented-out, then you
   # probably should define those extensions to indicate media types:
   #
   AddType application/x-compress .Z
   AddType application/x-gzip .gz .tgz .parcel #此处添加.parcel

   #

重启http服务

[root@localhost bigdata]#systemctl restart httpd

3.配置CDH Flink Parcel

点击配置进入，新增flink链接

确认后会在parcel中出现flink，如下图

PS:这里截图是之前版本的

点击下载

点击分配

点击激活

激活成功

重启服务

systemctl restart cloudera-scm-server

添加服务

添加Flink

选择需要安装节点点击继续

注意这里安装一个节点就行，yarn模式都是在yarn中运行

启动成功

重启Cloudera Management Service，(不重启的话CDH无法监控到)

问题解决

NoClassDefFoundError: org/apache

启动flink时失败，日志报错招不到Yarn的jar包

下载jar包

https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-3-uber/3.1.1.7.2.9.0-173-9.0

上传到目录

/opt/cloudera/parcels/FLINK/lib/flink/lib

java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;

16:12:34.454 [main] ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli - Error while running the Flink session.
java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:230) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:156) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851) [flink-dist_2.12-1.12.1.jar:1.12.1]

------------------------------------------------------------
 The program finished with the following exception:

java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:230)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:156)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851)
[root@r-wb-14 ~]#

下载jar包放到目录

https://mvnrepository.com/artifact/commons-cli/commons-cli/1.3.1

/opt/cloudera/parcels/FLINK/lib/flink/lib

kerberos login configuration is invalid

原因是因为配置了kerberos，上面安装包默认的配置，这个要删除掉

相关资料文档：

1. Flink1.12.3+CDH6.3.1集成.docx
   链接: https://pan.baidu.com/s/1QaaihgJM7QDO7yRzi4xbCg
   提取码: 97n2