Easy to handle ten thousand read/write request per second with Hbase Phoenix

At this blog, i will show my smoke test read / write performance of Hbase (With Phoenix Sql layer).

- Test resource
  + 3 super micro server (16 core 2.1 Ghz, 32 GB RAM, 2TB HDD Hitachi)
  + Apache Hbase 0.98 (Hortonwork 2.1)
  + Apache Phoenix 4.0
  + JVM Master Maximum Java Heap Size: 3 GB
  + JVM RegionServer Maximum Java Heap Size: 10 GB
  + And some turning will explain later ...

- I wrote a multi threading java program that run 16 thread in each server and auto generate data for schema below, then insert data to Hbase via Phoenix:
CREATE TABLE IF NOT EXISTS test (id CHAR( 50) NOT NULL PRIMARY KEY, string_1 CHAR(100), int_1 INTEGER, float_1 FLOAT, time INTEGER);

- During put data into Hbase, some client still run a lot of query in Phoenix shell like: select * ... where ..., select count (*), select ... group by ...

- My monitoring screen during this test:

RegionServer requests [New highest performance]


RegionServer queue size [New highest performance]


Htop & Phoenix shell



Ganglia Monitoring tool



Graphite Jvm Monitoring tool

- As you can see, i can handle more request with my system, maybe more 30 or 40k request per second, but i think it good enought for a lot of system, in next blog entry, i will increase write performance and explain how to do it.



Comments