hbase 性能测试

原创

mtj66 2022-01-04 11:20:09 博主文章分类：hbase ©著作权

文章标签 数据 文章分类 代码人生

©著作权归作者所有：来自51CTO博客作者mtj66的原创作品，请联系作者获取转载授权，否则将追究法律责任

n_splits = 40

create ‘usertable’,’family’,{SPLITS => (1..n_splits).map {|i| “user#{1000+i*(9999-1000)/n_splits}”}}

create ‘usertable_snappy’,’family’,{SPLITS => (1..n_splits).map {|i| “user#{1000+i*(9999-1000)/n_splits}”}}

disable ‘usertable_snappy’

alter ‘usertable_snappy’, NAME => ‘family’, COMPRESSION => ‘snappy’

enable ‘usertable_snappy’

create ‘usertable_lz4’,’family’,{SPLITS => (1..n_splits).map {|i| “user#{1000+i*(9999-1000)/n_splits}”}}

disable ‘usertable_lz4’

alter ‘usertable_lz4’, NAME => ‘family’, COMPRESSION => ‘lz4’

enable ‘usertable_lz4’

create ‘usertable_gz’,’family’,{SPLITS => (1..n_splits).map {|i| “user#{1000+i*(9999-1000)/n_splits}”}}

disable ‘usertable_gz’

alter ‘usertable_gz’, NAME => ‘family’, COMPRESSION => ‘gz’

enable ‘usertable_gz’

alter ‘usertable’, NAME => ‘family’, COMPRESSION => ‘GZ’

bin/ycsb run hbase10 -P workloads/workloada -cp /var/rds/hbase-1.3.0/conf -p table=usertable -p columnfamily=family -threads 90

bin/ycsb run hbase10 -P workloads/workloadc -cp /opt/tools/hbase-1.3.0/conf -p table=usertable -p columnfamily=family -threads 90 -p maxexecutiontime=1800 -s > workloadc-result.txt

bin/ycsb run hbase10 -P workloads/workloadx -cp /opt/tools/hbase-1.3.0/conf -p table=usertable -p columnfamily=family -threads 90 -p maxexecutiontime=1800 -s 1> workloadx-run3.txt -s 2> workloadx-result3.txt

测试读操作前，需要先load数据，才能在读操作中被识别

bin/ycsb load hbase10 -P workloads/workloada -cp /var/rds/hbase-1.3.0/conf -p table=usertable -p columnfamily=family -threads 90 -p maxexecutiontime=1800 -s 1> workloada-run1.txt -s 2> workloada-result1.txt

获取OPS

cat current workloadx-result4.txt | awk ‘{print $7}’

P99时延

cat current workloadx-result4.txt | awk ‘{print 21}' | awk -F '=' '{print2}' | awk -F ',' '{print $1}’

平均时延

cat current workloadx-result4.txt | awk ‘{print 19}' | awk -F '=' '{print2}' | awk -F ',' '{print $1}’

cd /data/home/yinchunfeng/YCSB-master/YCSB-master

bin/ycsb load hbase12 -P /data/home/yinchunfeng/YCSB-master/YCSB-master/hbase12/target/ycsb-hbase12-binding-0.14.0-SNAPSHOT/workloads/workloada -cp /etc/hbase/conf -p table=usertable -p columnfamily=family -p recordcount=100000 -threads 100

cd /data/home/yinchunfeng/YCSB-master/YCSB-master

alter ‘usertable’, {NAME=>’f’, METHOD=>’delete’}

alter ‘usertable_snappy’, NAME => ‘family’, COMPRESSION => ‘snappy’

alter ‘usertable_lz4’, NAME => ‘family’, COMPRESSION => ‘lzf’

alter ‘obd_locus:locus_app’, NAME => ‘f1’, COMPRESSION => ‘lzf’