赞
踩
背景:hadoop服务运行在linux服务器上,windows环境下进行hadoop相关开发
报错:windows环境下进行hadoop开发时报错 Failed to locate the winutils binary in the hadoop binary path
- --------setUp---------
- [ERROR] method:org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:425)
- Failed to locate the winutils binary in the hadoop binary path
- java.io.IOException: Could not locate executable D:\software\hadoop\hadoop-2.6.0-cdh5.15.1\bin\winutils.exe in the Hadoop binaries.
- at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:407)
- at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:422)
- at org.apache.hadoop.util.Shell.<clinit>(Shell.java:415)
- at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
- at org.apache.hadoop.fs.CommonConfigurationKeysPublic.<clinit>(CommonConfigurationKeysPublic.java:311)
- at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:536)
- at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:711)
- at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:694)
- at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
- at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2816)
- at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:98)
- at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2853)
- at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2835)
- at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:387)
- at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:176)
- at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:173)
- at java.security.AccessController.doPrivileged(Native Method)
- at javax.security.auth.Subject.doAs(Subject.java:422)
- at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
- at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:173)
- at com.imooc.bigdata.hadoop.hdfs.HDFSApp.setUp(HDFSApp.java:46)
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
- at java.lang.reflect.Method.invoke(Method.java:498)
- at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
- at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
- at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
- at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
- at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
- at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
- at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
- at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
- at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
- at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
- at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
- at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
- at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
- at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
- at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
- at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
- at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
- at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
- at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
- [WARN ] method:org.apache.hadoop.util.NativeCodeLoader.<clinit>(NativeCodeLoader.java:62)
- Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
- --------tearDown---------
-
- Process finished with exit code 0
原因:在windows环境下进行hadoop开发时也需要配置hadoop相关环境变量(与在linux下配置步骤相同),另外还需将缺少的hadoop.dll、winutils.exe两个文件拷贝到hadoop安装目录下对应的bin目录下。这两个文件(也可以使用hadoop源码自行编译)的下载地址:hdfs-API运行报错(缺少winutils.exe hadoop.dll)_hadoop.dll和winutils.exe文件-CSDN博客
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。