当前位置:   article > 正文

Failed to locate the winutils binary in the hadoop binary path

failed to locate the winutils binary in the hadoop binary path

背景:hadoop服务运行在linux服务器上,windows环境下进行hadoop相关开发

报错:windows环境下进行hadoop开发时报错 Failed to locate the winutils binary in the hadoop binary path

  1. --------setUp---------
  2. [ERROR] method:org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:425)
  3. Failed to locate the winutils binary in the hadoop binary path
  4. java.io.IOException: Could not locate executable D:\software\hadoop\hadoop-2.6.0-cdh5.15.1\bin\winutils.exe in the Hadoop binaries.
  5. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:407)
  6. at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:422)
  7. at org.apache.hadoop.util.Shell.<clinit>(Shell.java:415)
  8. at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
  9. at org.apache.hadoop.fs.CommonConfigurationKeysPublic.<clinit>(CommonConfigurationKeysPublic.java:311)
  10. at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:536)
  11. at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:711)
  12. at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:694)
  13. at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
  14. at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2816)
  15. at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:98)
  16. at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2853)
  17. at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2835)
  18. at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:387)
  19. at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:176)
  20. at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:173)
  21. at java.security.AccessController.doPrivileged(Native Method)
  22. at javax.security.auth.Subject.doAs(Subject.java:422)
  23. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
  24. at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:173)
  25. at com.imooc.bigdata.hadoop.hdfs.HDFSApp.setUp(HDFSApp.java:46)
  26. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  27. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  28. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  29. at java.lang.reflect.Method.invoke(Method.java:498)
  30. at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
  31. at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
  32. at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
  33. at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
  34. at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
  35. at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
  36. at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
  37. at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
  38. at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
  39. at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
  40. at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
  41. at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
  42. at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
  43. at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
  44. at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
  45. at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
  46. at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
  47. at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
  48. at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
  49. [WARN ] method:org.apache.hadoop.util.NativeCodeLoader.<clinit>(NativeCodeLoader.java:62)
  50. Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  51. --------tearDown---------
  52. Process finished with exit code 0

原因:在windows环境下进行hadoop开发时也需要配置hadoop相关环境变量(与在linux下配置步骤相同),另外还需将缺少的hadoop.dll、winutils.exe两个文件拷贝到hadoop安装目录下对应的bin目录下。这两个文件(也可以使用hadoop源码自行编译)的下载地址:hdfs-API运行报错(缺少winutils.exe hadoop.dll)_hadoop.dll和winutils.exe文件-CSDN博客

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/羊村懒王/article/detail/686109
推荐阅读
相关标签
  

闽ICP备14008679号