IntelliJ IDEA将代码打成Jar包的方式,注意,一定要确保打包Jar的时候日志输出是准确的,没有报错才行,而不是单纯看到有Jar包生成了就觉得包打好了,如果你不看打包报错日志,单纯去copy走生成的错误的包,后面hive引用该包还是会报错,你还的改然后重新打包,然后将该包上传到hdfs上,如我把它上传到hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar
;
万事具备,现在开始打开hive的客户端,创建永久的UDF函数,Hive Cli环境下操作如下;
hive> create function rowyet.picratio as 'org.example.PicRatio_UDF' using jar 'hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar';
Added [/opt/hive/log/c0017eab-0d8b-49bd-8157-09237071085c_resources/udf-1.0-SNAPSHOT-jar-with-dependencies.jar] to class path
Added resources: [hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar]
OK
Time taken: 6.03 seconds
--测试
hive> select rowyet.picratio("4:3")
> ;
OK
4:3
Time taken: 0.902 seconds, Fetched: 1 row(s)
hive> select rowyet.picratio("3:4");
OK
4:3
Time taken: 0.064 seconds, Fetched: 1 row(s)也可以临时性添加函数,具体HiveQL如下,一般不常用,都是建立永久的居多;create temporary function rowyet.picratio as 'org.example.PicRatio_UDF' using jar 'hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar';
异常分析
- 如果连加载Jar都报错,未出现以下加载Jar的日志的语句就报错了; 如报错
FAILED: SemanticException java.lang.IllegalArgumentException: java.net.UnknownHostException: hive
,说明hive客户端内没有加载到Jar包,请检查你的Jar路径对不对,是否多了少了/
;
Added [/opt/hive/log/c0017eab-0d8b-49bd-8157-09237071085c_resources/udf-1.0-SNAPSHOT-jar-with-dependencies.jar] to class path
Added resources: [hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar]
OK
- 如果加载Jar显示有日志,而是报错
Failed to register rowyet.picratio using class com.ruoyin.hiveudf.PicRatio_UDF
这种类似的,说明是Jar打包的有问题,Jar内存在问题,请回去看打包日志,是否存在错误,修复掉即可。
删除自定义UDF函数
也支持删除自定义得UDF,具体HiveQL如下;
hive> drop function rowyet.picratio;
Added [/opt/hive/log/ce6a5159-2063-4796-b4cb-5a6762b0ad15_resources/udf-1.0-SNAPSHOT-jar-with-dependencies.jar] to class path
Added resources: [hdfs:///hive/hive_udf_jar/udf-1.0-SNAPSHOT-jar-with-dependencies.jar]
OK
Time taken: 0.752 seconds
hive> select rowyet.picratio("4:3")
> ;
FAILED: SemanticException [Error 10011]: Invalid function rowyet.picratio