前提
安装Sqoop的前提是已经具备Java和Hadoop的环境。
准备安装包
- 下载地址：http://mirrors.hust.edu.cn/apache/sqoop/1.4.6/
或者 - 上传安装包sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz到虚拟机中
- 解压sqoop安装包到指定目录，
我解压到/root/soft目录下面了 ,解压命令
[root@zjj101 soft]# tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz重命名配置文件
Sqoop的配置文件与大多数大数据框架类似，在sqoop根目录下的conf目录中。
cd 到 /sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf 目录下面,重命名一下配置文件
mv sqoop-env-template.sh sqoop-env.sh修改sqoop-env.sh配置文件
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HIVE_HOME=$HIVE_HOME
export ZOOKEEPER_HOME=/opt/module/zookeeper-3.4.10
export ZOOCFGDIR=/opt/module/zookeeper-3.4.10/conf
export HBASE_HOME=/opt/module/hbase-1.3.1
拷贝JDBC驱动
准备mysql-connector-java-5.1.27-bin.jar 拷贝到
sqoop-1.4.6.bin__hadoop-2.0.4-alpha/lib/ ,具体jar版本看你要连接的MySQL的版本是多少.
验证Sqoop
在Sqoop安装的根目录执行 bin/sqoop help 命令验证一下 , 出现警告信息不要紧,只要不是error就行.
[root@zjj101 sqoop-1.4.6.bin__hadoop-2.0.4-alpha]# bin/sqoop help
# 这个意思是$HBASE_HOME没配置好,我本机没HBase,所以没配置这个
Warning: /opt/module/hbase-1.3.1 does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
# 这个也是一样
Warning: /root/soft/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
# 这个和HBase一样
Warning: /root/soft/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/module/zookeeper-3.4.10 does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/10/29 13:47:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
usage: sqoop COMMAND [ARGS]
Available commands:
codegen Generate code to interact with database records
create-hive-table Import a table definition into Hive
eval Evaluate a SQL statement and display the results
export Export an HDFS directory to a database table
help List available commands
import Import a table from a database to HDFS
import-all-tables Import tables from a database to HDFS
import-mainframe Import datasets from a mainframe server to HDFS
job Work with saved jobs
list-databases List available databases on a server
list-tables List available tables in a database
merge Merge results of incremental imports
metastore Run a standalone Sqoop metastore
version Display version information
See 'sqoop help COMMAND' for information on a specific command.
测试能否连接MySQL数据库
zjj101是我配置host映射的ip
在 根目录下面执行 bin/sqoop list-databases --connect jdbc:mysql://zjj101:3306/ --username root --password root
[root@zjj101 sqoop-1.4.6.bin__hadoop-2.0.4-alpha]# bin/sqoop list-databases --connect jdbc:mysql://zjj101:3306/ --username root --password root
Warning: /opt/module/hbase-1.3.1 does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /root/soft/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /root/soft/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/module/zookeeper-3.4.10 does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/10/29 13:51:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
20/10/29 13:51:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
# 说明连接MySQL了
20/10/29 13:51:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
hive
mysql
performance_schema
[root@zjj101 sqoop-1.4.6.bin__hadoop-2.0.4-alpha]#
