eclipse开发mapreduce程序,本地可以正常运行,使用eclipse打包上传到Linux上,使用hadoop jar命令执行,报错如下:
[root@single test]# hadoop jar salarysum.jar hdfs://single:9000/input/emp.txt /output
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [rsrc:org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:rsrc:slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
20/06/27 21:15:11 INFO client.RMProxy: Connecting to ResourceManager at single/192.168.128.11:8032
20/06/27 21:15:18 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy thi
s.20/06/27 21:15:18 WARN mapreduce.JobResourceUploader: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
20/06/27 21:15:18 INFO input.FileInputFormat: Total input paths to process : 1
20/06/27 21:15:23 INFO mapreduce.JobSubmitter: number of splits:1
20/06/27 21:15:23 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1592642952782_0021
20/06/27 21:15:23 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
20/06/27 21:15:27 INFO impl.YarnClientImpl: Submitted application application_1592642952782_0021
20/06/27 21:15:27 INFO mapreduce.Job: The url to track the job: http://single:8088/proxy/application_1592642952782_0021/
20/06/27 21:15:27 INFO mapreduce.Job: Running job: job_1592642952782_0021
20/06/27 21:15:41 INFO mapreduce.Job: Job job_1592642952782_0021 running in uber mode : false
20/06/27 21:15:41 INFO mapreduce.Job: map 0% reduce 0%
20/06/27 21:15:54 INFO mapreduce.Job: map 100% reduce 0%
20/06/27 21:15:54 INFO mapreduce.Job: Task Id : attempt_1592642952782_0021_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.leboop.www.totalsalary.SalarySumMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
核心:
No job jar file set. User classes may not be found. See Job or Job#setJar(String).
Job jar is not present. Not adding any jar to the list of resources.
找不到jar包。
解决方法:
Configuration conf = new Configuration();
conf.set("mapreduce.job.jar", "salarysum.jar");
Job job = Job.getInstance(conf, "Salary Sum");
主类中设置jar包名salarysum.jar。