[HBase] HBase-2.2.0 安裝教學

Posted on by
Post Views: 5,266

在擁有 HDFS 這個儲存資料的平台之後，我們能夠做的應用就很多了，本篇要來介紹如何安裝 HBase 2.2.0，與 Hadoop 相同，HBase 也是使用 Java 來撰寫為主。Hadoop ecosystem 工具簡介, 安裝教學與各種情境使用系列 第 10 篇裡面教學使用 HBase2.0.x，但是只支援Hadoop 2.6.1+與2.7.1+版本，使用 Hadoop-3.1.2 目前可以支援的 HBase 版本為 HBase-2.2.0 (released at 2019/06/11) 參考連結。

修改 /opt/hbase-2.2.0/conf/hbase-site.xml

安裝在 local file system 上面則要標明：

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>file:///opt/data/hbase-2.2.0</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/opt/data/zookeeper</value>
    <description>預設路徑為/tmp</description>
  </property>
</configuration>

安裝在 hdfs 平台上面則要標明：

<property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:9000/hbase</value>
</property>
<property>
    <name>hbase.cluster.distributed</name>
    <value>false</value>
</property>

如果是運作在 Pseudo-Distributed 或是 Fully-Distributed 的 Cluster 則需要修改：

<property>
  <name>hbase.cluster.distributed</name>
  <value>true</value>
</property>

設定 JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-12.0.1.jdk/Contents/Home
使用以下的指令可以得到 JAVA 安裝的路徑：

/usr/libexec/java_home -verbose
執行 /opt/hbase-2.2.0/bin/start-hbase.sh
bash-5.0$ ./start-hbase.sh
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2358: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER：無效的變數名稱
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2453: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS：無效的變數名稱
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.GetJavaProperty
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.HBaseConfTool
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2358: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER：無效的變數名稱
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2453: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS：無效的變數名稱
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.GetJavaProperty
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Error: Could not find or load main class org.apache.hadoop.hbase.zookeeper.ZKServerTool
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.zookeeper.ZKServerTool
running master, logging to /opt/hbase-2.2.0/logs/hbase-linyuting-master-lindembp.out
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2358: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER：無效的變數名稱
/opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2453: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS：無效的變數名稱
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.GetJavaProperty
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Error: Could not find or load main class org.apache.hadoop.hbase.master.HMaster
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.master.HMaster
: running regionserver, logging to /opt/hbase-2.2.0/logs/hbase-linyuting-regionserver-lindembp.out
: /opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2358: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER：無效的變數名稱
: /opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2453: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS：無效的變數名稱
: Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
: Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.GetJavaProperty
: Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
: Error: Could not find or load main class org.apache.hadoop.hbase.regionserver.HRegionServer
: Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.regionserver.HRegionServer

不能啟動的原因主要是因為以下的原因：
: /opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2358: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER：無效的變數名稱
: /opt/hadoop-3.1.2/libexec/hadoop-functions.sh: 列 2453: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS：無效的變數名稱

具體原因目前不知道為什麼？暫時先利用 Downgrade 到 hbase-2.0.5 這個方法，結果就是可以成功啟動如下：

bash-5.0: jps
10932 NameNode
12326 HRegionServer
12375 Jps
10632 DataNode
12249 HMaster
10733 SecondaryNameNode

啟動 HBase Shell:

bash-5.0: ./start-hbase.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hbase-2.0.5/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.9.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.htmlmultiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
running master, logging to /opt/hbase-2.0.5/logs/hbase-root-master-lindembp.out
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hbase-2.0.5/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.9.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.htmlmultiple_bindings for an explanation.
: running regionserver, logging to /opt/hbase-2.0.5/logs/hbase-root-regionserver-lindembp.out

此時在 hdfs://localhost:9000/hbase 可以看到已經有檔案生成了！

bash-5.0: hdfs dfs -ls /hbase
19/08/07 22:45:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 13 items
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/.hbck
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/.tmp
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/MasterProcWALs
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/WALs
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/archive
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/corrupt
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/data
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/hbase
-rw-r--r--   1 root supergroup         42 2019-08-07 22:41 /hbase/hbase.id
-rw-r--r--   1 root supergroup          7 2019-08-07 22:41 /hbase/hbase.version
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/mobdir
drwxr-xr-x   - root supergroup          0 2019-08-07 22:41 /hbase/oldWALs
drwx--x--x   - root supergroup          0 2019-08-07 22:41 /hbase/staging