HTable是HBase提供的一个主要客户端接口， 通过它可以实现与HBase集群连接， 进而实现CRUD等一系列功能。
1. HTable如何实现与HBase集群的连接?
HTable是如何知道HBase集群中的HMaser和HRegionServer, 答案是通过Zookeeper, zookeeper是一个分布式协调系统， HMaser,HRegionServer在启动后会自动在zookeeper中注册自己的地址。
HTable实例构造后会查找zookeeper配置， 与zookeeper取的连接， 进而获取HMaser,HRegionServer信息
加载zookeeper配置流程如下:
通过org.apache.hadoop.hbase.zookeeper.ZKConfig 类来加载zookeeper配置:
- public static Properties makeZKProps(Configuration conf) {
- // First check if there is a zoo.cfg in the CLASSPATH. If so, simply read
- // it and grab its configuration properties.
- ClassLoader cl = HQuorumPeer.class.getClassLoader(); // zoo.cfg必须在CLASSPATH路径下
- final InputStream inputStream =
- cl.getResourceAsStream(HConstants.ZOOKEEPER_CONFIG_NAME); // 优先使用zoo.cfg配置
- if (inputStream != null) {
- try {
- return parseZooCfg(conf, inputStream);
- } catch (IOException e) {
- LOG.warn("Cannot read " + HConstants.ZOOKEEPER_CONFIG_NAME +
- ", loading from XML files", e);
- }
- }
// 在zoo.cfg配置加载失败的情况下再使用hbase的配置文件中指定的配置项- // Otherwise, use the configuration options from HBase's XML files.
- Properties zkProperties = new Properties();
- // Directly map all of the hbase.zookeeper.property.KEY properties.
- for (Entry<String, String> entry : conf) {
- String key = entry.getKey();
- if (key.startsWith(ZK_CFG_PROPERTY)) {
- String zkKey = key.substring(ZK_CFG_PROPERTY_SIZE);
- String value = entry.getValue();
- // If the value has variables substitutions, need to do a get.
- if (value.contains(VARIABLE_START)) {
- value = conf.get(key);
- }
- zkProperties.put(zkKey, value);
- }
- }
- // If clientPort is not set, assign the default
- if (zkProperties.getProperty(ZK_CLIENT_PORT_KEY) == null) {
- zkProperties.put(ZK_CLIENT_PORT_KEY,
- HConstants.DEFAULT_ZOOKEPER_CLIENT_PORT);
- }
- // Create the server.X properties.
- int peerPort = conf.getInt("hbase.zookeeper.peerport", 2888);
- int leaderPort = conf.getInt("hbase.zookeeper.leaderport", 3888);
- final String[] serverHosts = conf.getStrings(HConstants.ZOOKEEPER_QUORUM,
- "localhost");
- for (int i = 0; i < serverHosts.length; ++i) {
- String serverHost = serverHosts[i];
- String address = serverHost + ":" + peerPort + ":" + leaderPort;
- String key = "server." + i;
- zkProperties.put(key, address);
- }
- return zkProperties;
- }
加载过程主要是先从CLASSPATH中寻找zoo.cfg的配置文件， 没有再使用hbase的配置项，有一个优先级关系， 需要注意, 否则，在zoo.cfg配置文件存在的情况下， 再怎么修改hbase配置文件否无济于事.
2. 连上zookeeper后会监控哪些节点?
HTable连上zookeeper后就会监控一些节点， 以期获取HMaster, HRegionServer等信息, 通过org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher类的setNodeNames方法可以看出:
- /**
- * Set the local variable node names using the specified configuration.
- */
- private void setNodeNames(Configuration conf) {
- baseZNode = conf.get(HConstants.ZOOKEEPER_ZNODE_PARENT,
- HConstants.DEFAULT_ZOOKEEPER_ZNODE_PARENT);
- rootServerZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.rootserver", "root-region-server"));
- rsZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.rs", "rs"));
- masterAddressZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.master", "master"));
- clusterStateZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.state", "shutdown"));
- assignmentZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.unassigned", "unassigned"));
- tableZNode = ZKUtil.joinZNode(baseZNode,
- conf.get("zookeeper.znode.tableEnableDisable", "table"));
- }
监控的节点有:
/hbase/master
/hbase/root-region-server
/hbase/rs
/hbase/shutdown
/hbase/unassigned
/hbase/table
接下来验证下zookeeper中是否生成了这些节点,
zkCli -server 192.168.203.129:2181
- [zk: 192.168.203.129:2181(CONNECTED) 0]
- [zk: 192.168.203.129:2181(CONNECTED) 0] ls /
- [hbase, zookeeper]
- [zk: 192.168.203.129:2181(CONNECTED) 1] ls /hbase
- [root-region-server, rs, table, unassigned, master, shutdown]
- [zk: 192.168.203.129:2181(CONNECTED) 2]
通过zookeeper客户端可以确认zookeeper中确实创建了上述六个节点
