系统:ubuntu12.04
前提:
1.客户端已经配置好hadoop环境。
2.设置环境变量HADOOP_HOME。
版本:
1.hadoop:hadoop-1.0.3
2.sqoop: Sqoop 1.4.1-incubating
客户端配置:
1.配置sqoop
a.从http://mirror.bit.edu.cn/apache/sqoop/1.4.1-incubating/下载 sqoop-1.4.1-incubating__hadoop-0.20.tar.gz
解压 tar zxvf sqoop-1.4.1-incubating__hadoop-0.20.tar.gz
b.修改SQOOP的文件bin/configure-sqoop,注释掉hbase检查(除非你准备使用HABASE等HADOOP上的组件,否则每次都出现warining信息)
即注释掉:
if [ ! -d "${HBASE_HOME}" ]; then
echo "Warning: $HBASE_HOME does not exist! HBase imports will fail."
echo 'Please set $HBASE_HOME to the root of your HBase installation.'
fi
c.修改PATH,增加sqoop命令。
$ vi ~/.profile
#增加
export PATH=/path/to/your/sqoopHome/bin:$PATH
使修改生效:
$ source ~/.profile
现在在终端运行 sqoop就可以了。
Bin/sqoop help
导数的时候出错: ERROR tool.ImportTool: Imported Failed: Attempted to generate class with no columns!
经排查,问题出在 --username ahmt 这个选项,需要把用户名大写, --username AHMT,同时注意表名也要大写
12/03/01 14:54:03 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Cannot run program "hive": java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at java.lang.Runtime.exec(Runtime.java:593)
at java.lang.Runtime.exec(Runtime.java:509)
at org.apache.sqoop.util.Executor.exec(Executor.java:76)
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:344)
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:297)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:392)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 15 more
出现这个错误主要是因为hive没有配置环境变量,hadoop找不到程序路径导致的
在/etc/profile配置
HIVE_HOME=/download/hive-0.8.1
export HIVE_HOME
bin/sqoop import --connectjdbc:oracle:thin:@192.168.7.173:1521:orcl --usernameLYC --password lyc--table HIVETEST--hive-import -m 1