赞
踩
java.lang.Exception: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in localfetcher#1 at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529) Caused by: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in localfetcher#1 at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376) at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: E:/bigdata/hdfs_temp/mapred/local/localRunner/Ruby%20Wang/jobcache/job_local1561914882_0001/attempt_local1561914882_0001_m_000000_0/output/file.out.index at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:200) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767) at org.apache.hadoop.io.SecureIOUtils.openFSDataInputStream(SecureIOUtils.java:156) at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:71) at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:62) at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:57) at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.copyMapOutput(LocalFetcher.java:124) at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.doCopy(LocalFetcher.java:102) at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.run(LocalFetcher.java:85)
这个问题主要原因可以看一下后边那个报错
Caused by: java.io.FileNotFoundException: E:/bigdata/hdfs_temp/mapred/local/localRunner/Ruby%20Wang/jobcache/job_local1561914882_0001/attempt_local1561914882_0001_m_000000_0/output/file.out.index
说是文件找不到异常
根据大佬的解析 一个是路径问题 一个是没有这个文件
这里发现这个文件是自动创建的 那么就是路径问题
众所周知
hadoop路径里是不能有空格的 然而这个自创的路径里有空格 于是就要更改这个路径
大佬解决办法:Hadoop本地运行出现:Caused by: java.io.FileNotFoundException:D:/tmp/hadoop-win2010/mapred/local/
具体:
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。