赞
踩
在做项目时,把sqoop和mapreduce集成到springboot中做数据抽取和计算,并打成jar包上传到服务器,项目是使用eureka进行注册提供服务,父项目下创建子模块,通过maven的package打包子模块。
通过以下命令在服务器运行
# nohup和&组合在后台运行 运行时会在当前目录下生成一个nohup.out日志文件
[root@fda ~]# nohup java -jar XXX.jar &
Sqoop版本1.4.7
Hadoop版本2.7.7
当进行sqoop的数据抽取时报错信息如下
2021-02-27 19:24 | [34mINFO [0;39m | [1;33mtask-1[0;39m | [1;32mo.a.sqoop.orm.CompilationManager[0;39m | $HADOOP_MAPRED_HOME is not set /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:7: error: package org.apache.hadoop.io does not exist import org.apache.hadoop.io.BytesWritable; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:8: error: package org.apache.hadoop.io does not exist import org.apache.hadoop.io.Text; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:9: error: package org.apache.hadoop.io does not exist import org.apache.hadoop.io.Writable; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:10: error: package org.apache.hadoop.mapred.lib.db does not exist import org.apache.hadoop.mapred.lib.db.DBWritable; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:11: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.JdbcWritableBridge; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:12: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.DelimiterSet; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:13: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.FieldFormatter; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:14: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.RecordParser; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:15: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.BooleanParser; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:16: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.BlobRef; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:17: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.ClobRef; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:18: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.LargeObjectLoader; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:19: error: package com.cloudera.sqoop.lib does not exist import com.cloudera.sqoop.lib.SqoopRecord; ^ /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol public class Users extends SqoopRecord implements DBWritable, Writable { ^ symbol: class SqoopRecord /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol public class Users extends SqoopRecord implements DBWritable, Writable { ^ symbol: class DBWritable /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol public class Users extends SqoopRecord implements DBWritable, Writable { ^ symbol: class Writable /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:137: error: cannot find symbol public void loadLargeObjects(LargeObjectLoader __loader) ^ symbol: class LargeObjectLoader location: class Users /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:140: error: cannot find symbol public void loadLargeObjects0(LargeObjectLoader __loader) ^ symbol: class LargeObjectLoader location: class Users /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:217: error: cannot find symbol private static final DelimiterSet __outputDelimiters = new DelimiterSet((char) 9, (char) 10, (char) 0, (char) 0, false); ^ symbol: class DelimiterSet location: class Users /tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:221: error: cannot find symbol public String toString(DelimiterSet delimiters) {
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。