使用Java API访问HBase(Linux)
一、安装并配置HBase2.2.2
1.HBase2.2.2安装
(一)下载压缩包
虚拟机已安装hadoop3.2.1
下载网址 hbase2.2.2
(二)解压安装包hbase-2.2.2-bin.tar.gz至路径/home/hadoop
$cd ~
$sudo tar -zxf ~/下载/hbase-2.2.2-bin.tar.gz -C /home/hadoop
(三)将解压的文件名hbase-2.2.2改为hbase,以方便使用
$sudo mv ./hbase-2.2.2 ./hbase
(四)把hbase目录权限赋予给hadoop用户
$sudo chown -R hadoop ./hbase
(五)配置环境变量,将hbase下的bin目录添加到path中。
$vim ~/.bashrc
在~/.bashrc文件中添加如下内容:
export PATH=$PATH:/home/hadoop/hbase/bin
如果已经引入过PATH请在export PATH这行追加 :/home/hadoop/hbase/bin
(vim操作:esc + i 编辑文件、esc + :wq 保存并退出、esc + :q! 不保存强制退出、esc + :q 未修改并退出)
(六)执行source命令使上述配置在当前终端立即生效
$source ~/.bashrc
(七)添加HBase权限
$sudo chown -R hadoop ./hbase
(八)查看HBase版本,确定hbase安装成功
$/home/hadoop/hbase/bin/hbase version
2.HBase配置(伪分布式模式配置)
(一)配置/home/hadoop/hbase/conf/hbase-env.sh,配置JAVA环境变量,并添加配置HBASE_MANAGES_ZK为true,用vi命令打开并编辑hbase-env.sh,命令如下:
$vim /home/hadoop/hbase/conf/hbase-env.sh
写入:
export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_162
export HBASE_CLASSPATH=/home/hadoop/hbase/conf
export HBASE_MANAGES_ZK=true
添加完成后保存退出即可。
(二)配置/home/hadoop/hbase/conf/hbase-site.xml
打开并编辑hbase-site.xml,命令如下:
$vim /home/hadoop/hbase/conf/hbase-site.xml
写入:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>false</value>
</property>
</configuration>
(三)接下来测试运行HBase
首先登陆ssh,再切换目录至/usr/local/hadoop ,再启动hadoop,如果已经启动hadoop请跳过此步骤。输入命令jps,能看到NameNode,DataNode和SecondaryNameNode都已经成功启动,表示hadoop启动成功。
$ssh localhost
$cd /usr/local/hadoop
$./sbin/start-dfs.sh
输入jps,出现Jps、NameNode、SecondaryNameNode、DataNode,表示hadoop启动成功。
切换目录至/home/hadoop/hbase,再启动HBase。
$cd /home/hadoop/hbase
$bin/start-hbase.sh
输入jps,看到以下界面说明hbase启动成功。
进入shell界面:
$bin/hbase shell
二、Java API编程实例
使用Eclipse编写java程序,来对HBase数据库进行增删查等操作。
1.在以上启动hadoop然后启动hbase的基础上进行以下操作。
2.新建Java Project(HBaseExample)
3.在弹出的界面中用鼠标点击“Libraries”选项卡,然后,点击界面右侧的“Add External JARs…”按钮。
4.在弹出的“JAR Selection”界面中(如下图所示),进入到“/home/hadoop/hbase/lib”目录,选中该目录下的所有jar文件(注意,不要选中client-facing-thirdparty、ruby、shaded-clients和zkcli这四个目录)。“client-facing-thirdparty”目录下,选中所有jar文件。然后点击Finish。
5.建Class(ExampleForHBase)。
6.输入代码。
代码如下:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
public class ExampleForHBase {
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
init();
createTable("student",new String[]{"score"});
insertData("student","zhangsan","score","English","69");
insertData("student","zhangsan","score","Math","86");
insertData("student","zhangsan","score","Computer","77");
getData("student", "zhangsan", "score","English");
close();
}
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
public static void createTable(String myTableName,String[] colFamily) throws IOException {
TableName tableName = TableName.valueOf(myTableName);
if(admin.tableExists(tableName)){
System.out.println("talbe is exists!");
}else {
TableDescriptorBuilder tableDescriptor = TableDescriptorBuilder.newBuilder(tableName);
for(String str:colFamily){
ColumnFamilyDescriptor family =
ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes(str)).build();
tableDescriptor.setColumnFamily(family);
}
admin.createTable(tableDescriptor.build());
}
}
public static void insertData(String tableName,String rowKey,String colFamily,String col,String val) throws IOException {
Table table = connection.getTable(TableName.valueOf(tableName));
Put put = new Put(rowKey.getBytes());
put.addColumn(colFamily.getBytes(),col.getBytes(), val.getBytes());
table.put(put);
table.close();
}
public static void getData(String tableName,String rowKey,String colFamily, String col)throws IOException{
Table table = connection.getTable(TableName.valueOf(tableName));
Get get = new Get(rowKey.getBytes());
get.addColumn(colFamily.getBytes(),col.getBytes());
Result result = table.get(get);
System.out.println(new String(result.getValue(colFamily.getBytes(),col==null?null:col.getBytes())));
table.close();
}
}
7.运行成功
8.到HBase Shell交互式环境中(使用命令bin/hbase shell),使用如下命令查看student表是否创建成功。
9.使用如下命令查看student表中的数据。
三、表的操作实例
根据上文启动hadoop,启动hbase,并可在HBase Shell交互式环境中,使用命令查看student表中的数据。
1.建表并查看已有表
(一)代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//创建一个表,表名为Score,列族为sname,course
createTable("Score",new String[]{"sname","course"});
//查看已有表,应输出Score表(新建表)和student(原有表)
listTables();
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 建表。HBase的表中会有一个系统默认的属性作为主键,主键无需自行创建,默认为put命令操作中表名后第一个数据,因此此处无需创建id列
* @param myTableName 表名
* @param colFamily 列族名
* @throws IOException
*/
public static void createTable(String myTableName,String[] colFamily) throws IOException {
init();
TableName tableName = TableName.valueOf(myTableName);
if(admin.tableExists(tableName)){
System.out.println("talbe is exists!");
}else {
HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);
for(String str:colFamily){
HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(str);
hTableDescriptor.addFamily(hColumnDescriptor);
}
admin.createTable(hTableDescriptor);
System.out.println("create table success");
}
close();
}
/**
* 查看已有表
* @throws IOException
*/
public static void listTables() throws IOException {
init();
HTableDescriptor hTableDescriptors[] = admin.listTables();
for(HTableDescriptor hTableDescriptor :hTableDescriptors){
System.out.println(hTableDescriptor.getNameAsString());
}
close();
}
}
(二)运行成功
(三)建表成功
2.插入数据
(一)代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//在Score表中插入一条数据,其行键为95001,sname为Mary
//因为sname列族下没有子列所以第四个参数为空
insertRow("Score", "95001", "sname", "", "Mary");
//在Score表中插入一条数据,其行键为95001,course:Math为88
insertRow("Score", "95001", "course", "Math", "88");
//在Score表中插入一条数据,其行键为95001,course:English为85
insertRow("Score", "95001", "course", "English", "85");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 向某一行的某一列插入数据
* @param tableName 表名
* @param rowKey 行键
* @param colFamily 列族名
* @param col 列名(如果其列族下没有子列,此参数可为空)
* @param val 值
* @throws IOException
*/
public static void insertRow(String tableName,String rowKey,String colFamily,String col,String val) throws IOException {
init();
Table table = connection.getTable(TableName.valueOf(tableName));
Put put = new Put(rowKey.getBytes());
put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
table.put(put);
table.close();
close();
}
}
(二)运行
(三)成功插入数据
3.查询操作
(一)代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//查询Score表中,行键为95001,列族为course,列为Math的值
getData("Score", "95001", "course", "Math");
//查询Score表中,行键为95001,列族为sname的值(因为sname列族下没有子列所以第四个参数为空)
getData("Score", "95001", "sname", "");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 根据行键rowkey查找数据
* @param tableName 表名
* @param rowKey 行键
* @param colFamily 列族名
* @param col 列名
* @throws IOException
*/
public static void getData(String tableName,String rowKey,String colFamily,String col)throws IOException{
init();
Table table = connection.getTable(TableName.valueOf(tableName));
Get get = new Get(rowKey.getBytes());
get.addColumn(colFamily.getBytes(),col.getBytes());
Result result = table.get(get);
showCell(result);
table.close();
close();
}
/**
* 格式化输出
* @param result
*/
public static void showCell(Result result){
Cell[] cells = result.rawCells();
for(Cell cell:cells){
System.out.println("RowName:"+new String(CellUtil.cloneRow(cell))+" ");
System.out.println("Timetamp:"+cell.getTimestamp()+" ");
System.out.println("column Family:"+new String(CellUtil.cloneFamily(cell))+" ");
System.out.println("row Name:"+new String(CellUtil.cloneQualifier(cell))+" ");
System.out.println("value:"+new String(CellUtil.cloneValue(cell))+" ");
}
}
}
(二)查询成功
4.删除操作
(一)删除指定列数据
①代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//删除Score表中指定列数据,其行键为95001,列族为course,列为Math
deleteRow("Score", "95001", "course", "Math");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 删除数据
* @param tableName 表名
* @param rowKey 行键
* @param colFamily 列族名
* @param col 列名
* @throws IOException
*/
public static void deleteRow(String tableName,String rowKey,String colFamily,String col) throws IOException {
init();
Table table = connection.getTable(TableName.valueOf(tableName));
Delete delete = new Delete(rowKey.getBytes());
//删除指定列的数据
delete.addColumn(colFamily.getBytes(), col.getBytes());
table.delete(delete);
table.close();
close();
}
}
②删除指定列数据成功
(二)删除指定列族数据
①代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//删除Score表中指定列族数据,其行键为95001,列族为course
//95001的Math和English的值都会被删除
deleteRow("Score", "95001", "course", "");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 删除数据
* @param tableName 表名
* @param rowKey 行键
* @param colFamily 列族名
* @param col 列名
* @throws IOException
*/
public static void deleteRow(String tableName,String rowKey,String colFamily,String col) throws IOException {
init();
Table table = connection.getTable(TableName.valueOf(tableName));
Delete delete = new Delete(rowKey.getBytes());
//删除指定列族的所有数据
delete.addFamily(colFamily.getBytes());
table.delete(delete);
table.close();
close();
}
}
②成功
(三)删除指定行数据
①代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//删除Score表中指定行数据,其行键为95001
deleteRow("Score", "95001", "", "");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 删除数据
* @param tableName 表名
* @param rowKey 行键
* @param colFamily 列族名
* @param col 列名
* @throws IOException
*/
public static void deleteRow(String tableName,String rowKey,String colFamily,String col) throws IOException {
init();
Table table = connection.getTable(TableName.valueOf(tableName));
Delete delete = new Delete(rowKey.getBytes());
table.delete(delete);
table.close();
close();
}
}
②成功
(四)删除表
①代码如下:
package HBaseExample;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class ExampleForHBase{
public static Configuration configuration;
public static Connection connection;
public static Admin admin;
public static void main(String[] args)throws IOException{
//删除Score表
deleteTable("Score");
}
//建立连接
public static void init(){
configuration = HBaseConfiguration.create();
configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
try{
connection = ConnectionFactory.createConnection(configuration);
admin = connection.getAdmin();
}catch (IOException e){
e.printStackTrace();
}
}
//关闭连接
public static void close(){
try{
if(admin != null){
admin.close();
}
if(null != connection){
connection.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
/**
* 删除指定表
* @param tableName 表名
* @throws IOException
*/
public static void deleteTable(String tableName) throws IOException {
init();
TableName tn = TableName.valueOf(tableName);
if (admin.tableExists(tn)) {
admin.disableTable(tn);
admin.deleteTable(tn);
}
close();
}
}
②删除成功(Score表已不见)
四、遇到的问题及解决办法
1.启动hadoop再启动hbase后,使用jps命令发现DataNode节点没有启动
于是经过查找,采用以下方法:
$cd /usr/local/hadoop
$./sbin/stop-dfs.sh
$rm -r ./tmp # 删除 tmp 文件,注意这会删除 HDFS 中原有的所有数据
$./bin/hdfs namenode -format # 重新格式化 NameNode
$./sbin/start-dfs.sh # 重启
再次测试,成功:
2.代码运行出错(The package org.apache.hadoop.hbase is accessible from more than one module:),经过网络查找解决方法后,发现是导入jar包的位置不对。
错误:将jar包放到Modulepath下。
正确:将jar包放到Classpath下。
本文引用了http://dblab.xmu.edu.cn/blog/2442-2/