win10编译Hadoop3.0.2源码遇到的坑_protoc --version' did not return a version

目录

背景

1、Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

3、Cannot run program "msbuild" (in directory "D:\hadoop-3.0.2-src\hadoop-common-project\hadoop-common"): CreateProcess error=2, 系统找不到指定的文件。

4、 Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part ...... @ 11:135 in D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml

5、 Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

编译结果

资源下载链接

---

背景

周五尝试用hive建表时，出现了问题，需要修改hadoop源码，而后编译。

但是在编译时，遇到了不少问题，周日耗时一天，排查完毕，编译通过

所有资源网盘链接见最后，请按需使用

1、Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version

 

解决方法：安装protobuf2.5-win

1、下载zip压缩包

2、解压压缩包，然后把解压路径添加到环境变量path里

 

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

 

javah出现了问题，解决方法：

修改源码目录下hadoop-common-project\hadoop-common\pom.xml文件，把所有的javahPath标签内容改成javah的绝对路径

 

3、Cannot run program "msbuild" (in directory "D:\hadoop-3.0.2-src\hadoop-common-project\hadoop-common"): CreateProcess error=2, 系统找不到指定的文件。

     

解决方法：安装VS Code，然后把msbuild可执行文件的绝对路径添加到path里(C:\Program Files (x86)\MSBuild\14.0\Bin)

 

4、 Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part ...<exec failοnerrοr="true" dir="D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="msbuild">... @ 11:135 in D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml

 

     cmake和MsBuild(MSB1009找不到)莫名报错，暂时在hadoop-hdfs-project\hadoop-hdfs-native-client\pom.xml中，屏蔽它们的错误

     

 

5、 Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

 

     解决方法：把ojalgo-43.0.jar下载下来，复制到本地maven仓库文件夹的org\ojalgo\ojalgo\jar\43.0\目录下

 

编译结果

资源下载链接

链接：https://pan.baidu.com/s/10PY2fhlBqJ0XNQiwlMpQIA 
提取码：l4bb 
 

包括hadoop3.0.2源码压缩包，apache-maven-3.6.1-bin.zip，cmake-3.14.5-win32-x86.msi，ojalgo-43.0.jar，protoc-2.5.0-win32.zip，VSCodeSetup-x64-1.25.1.exe，zlib-1.2.3.exe