NoSQL Benchmark Testing

YCSB -- developed by Yahoo for cloud nosql benchmark tesing

Transcript

1. www.vmcd.org 基于 mongodb 的压力评测工具 YCSB 的一些概括 YCSB 是一款基于 NOSQL Benchmark 的工具,提供了多种方式模拟测试 nosql 的负载,基于现在对 nosql 的一些压力测试还没有得到重视. YCSB 的全面的测试功能可以给即将上线的 nosql 提供一种另类保障。 There are many new serving databases available, including:  PNUTS  BigTable  HBase  Hypertable  Azure  Cassandra  CouchDB  Voldemort  MongoDb  OrientDB  Infinispan  Dynomite  Redis  GemFire  GigaSpaces XAP  DynamoDB  Couchhase  Aerospike
2. www.vmcd.org 下面的数据仅供参考,测试于虚拟机服务器。 给一个简单的 load data 的例子： [root@mysqlstd ycsb-0.1.4]# ./bin/ycsb load mongodb -P workloads/workloada -p mongodb.url=mongodb://127.0.0.1:27017 -p mongodb.database=newdb -p mongodb.writeConcern=normal -s >data Loading workload... Starting test. 0 sec: 0 operations; 10 sec: 18448 operations; 1837.08 current ops/sec; [INSERT AverageLatency(us)=423.35] 20 sec: 42134 operations; 2366.71 current ops/sec; [INSERT AverageLatency(us)=373.44] 30 sec: 61185 operations; 1904.34 current ops/sec; [INSERT AverageLatency(us)=661.58] 40 sec: 85308 operations; 2411.09 current ops/sec; [INSERT AverageLatency(us)=324.83] 50 sec: 97785 operations; 1247.2 current ops/sec; [INSERT AverageLatency(us)=985.33] 50 sec: 100000 operations; 2662.26 current ops/sec; [INSERT AverageLatency(us)=371.24]
3. www.vmcd.org load data 之后 可以开始模拟压测了.YCSB 主要分为以下几种模式： Workload A: Update heavy workload This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions. Workload B: Read mostly workload This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags. Workload C: Read only This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop). Workload D: Read latest workload In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest. Workload E: Short ranges
4. www.vmcd.org In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id). Workload F: Read-modify-write In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity. 其中 E 模式的 short range 存在问题,当然我们也可以自定义模式: # Yahoo! Cloud System Benchmark # Workload A: Update heavy workload # Application example: Session store recording recent actions # # Read/update ratio: 50/50 # Default data size: 1 KB records (10 fields, 100 bytes each, plus key) # Request distribution: zipfian recordcount=100000 operationcount=100000 workload=com.yahoo.ycsb.workloads.CoreWorkload readallfields=true
5. www.vmcd.org readproportion=1 updateproportion=0 scanproportion=0 insertproportion=0 修改 read update insert 的百分比来达到模拟真实环境的目的, 下面给个压测例子 比如压测纯读操作: [root@mysqlstd ycsb-0.1.4]# ./bin/ycsb run mongodb -P workloads/workloadc -P large.dat -p mongodb.url=mongodb://127.0.0.1:27017 -p mongodb.database=newdb -p mongodb.writeConcern=normal -s >data Loading workload... Starting test. 0 sec: 0 operations; 10 sec: 49375 operations; 4922.24 current ops/sec; [READ AverageLatency(us)=192.56] 18 sec: 100000 operations; 6141.57 current ops/sec; [READ AverageLatency(us)=159.72] 95%读+%5 写： [root@mysqlstd ycsb-0.1.4]# ./bin/ycsb run mongodb -P workloads/workloadd -P large.dat -p mongodb.url=mongodb://127.0.0.1:27017 -p mongodb.database=newdb -p mongodb.writeConcern=normal -s >data Loading workload... Starting test.
6. www.vmcd.org 0 sec: 0 operations; 10 sec: 43497 operations; 4333.23 current ops/sec; [INSERT AverageLatency(us)=633.66] [READ AverageLatency(us)=196.33] 20 sec: 92795 operations; 4925.37 current ops/sec; [INSERT AverageLatency(us)=792.15] [READ AverageLatency(us)=167.74] 21 sec: 100000 operations; 5637.72 current ops/sec; [INSERT AverageLatency(us)=379.57] [READ AverageLatency(us)=163.45] 另外 thumbtack 对 YCSB 进行了修改,增强了一些功能 开源地址： https://github.com/thumbtack-technology/ycsb 主要增加支持了 Aerospike and Couchbase 目前 Aerospike 已经开源 针对 SSD 进行了专门优化 ： http://www.aerospike.com/blog/entrepreneurs-break-all-the-rules-aerospike-goes-open-source/ 同时 MongoDB 的驱动程序从 2.8.0 版（在 Mongo 2.2 发布之前的版本）升级到了 2.10.1 版,并实现了对 readPreference 配置的设置 mongodb.readPreference = primary|primaryPreferred|secondary|secondaryPreferred
7. www.vmcd.org 下面结合 thumbtack 修改的版本做个测试 可以得到具体数据: [root@mysqlstd ycsb]# fab ycsb_load:db=mongodb [10.0.32.38] Executing task 'ycsb_load' 2014-07-15 01:09:00-07:00 [10.0.32.38] run: echo "/root/ycsb/bin/ycsb load mongodb -s -p mongodb.url=mongodb://127.0.0.1:27017 -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p updateretrycount=1000 -p mongodb.writeConcern=normal -p mongodb.database=ycsb -p
8. www.vmcd.org recordcount=5000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=32 -p insertretrycount=10 -p readretrycount=1000 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p operationcount=2400000000 -p fieldnameprefix=f -p maxexecutiontime=2400 -p mongodb.readPreference=primaryPreferred -p measurementtype=timeseries -p reconnectiontime=1000 -p fieldlength=10 -p insertstart=0 -p insertcount=5000000 > /root/ycsb/2014-07-15_01-09_mongodb_load.out 2> /root/ycsb/2014-07-15_01-09_mongodb_load.err" | at 01:09 today [10.0.32.38] out: job 13 at 2014-07-15 01:09 Done. Disconnecting from 10.0.32.38... done. [mongo@mysqlstd ~]$ /data/mongodb/mongodb/bin/mongo MongoDB shell version: 2.6.1 connecting to: test > show dbs admin 0.031GB local 0.031GB newdb 0.500GB newdb1 0.500GB newdb2 0.500GB ycsb 1.500GB > use ycsb switched to db ycsb > db.usertable.c db.usertable.clean( db.usertable.convertToCapped( db.usertable.copyTo( db.usertable.createIndex( db.usertable.constructor db.usertable.convertToSingleObject( db.usertable.count( > db.usertable.count() 2675710
9. www.vmcd.org 模拟压力环境：workload mode=A [root@mysqlstd ycsb]# fab ycsb_run:db=mongodb,workload=A [10.0.32.38] Executing task 'ycsb_run' 2014-07-15 02:13:00-07:00 [10.0.32.38] run: echo "/root/ycsb/bin/ycsb run mongodb -s -P /root/ycsb/workloads/workloada -p mongodb.url=mongodb://127.0.0.1:27017 -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p updateretrycount=1000 -p mongodb.writeConcern=normal -p mongodb.database=ycsb -p recordcount=5000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=32 -p insertretrycount=10 -p readretrycount=1000 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p operationcount=1800000 -p fieldnameprefix=f -p maxexecutiontime=180 -p mongodb.readPreference=primaryPreferred -p measurementtype=timeseries -p reconnectiontime=1000 -p fieldlength=10 > /root/ycsb/2014-07-15_02-13_mongodb_workloada.out 2> /root/ycsb/2014-07-15_02-13_mongodb_workloada.err" | at 02:13 today [10.0.32.38] out: job 23 at 2014-07-15 02:13 [10.0.32.38] out: Done. Disconnecting from 10.0.32.38... done. 使用 merge.py 得到具体数值 [root@mysqlstd ycsb]# ./bin/merge.py OVERALL OVERALL READ READ READ READ READ READ READ READ READ UPDATE UPDATE UPDATE UPDATE UPDATE UPDATE UPDATE UPDATE UPDATE CLEANUP CLEANUP CLEANUCLEANUP CLEANUP CLEANUP CLEANUP CLEANUP CLEANUP RunTime Throughput Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency
10. www.vmcd.org 99thPercentileLatency Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency 99thPercentileLatency Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency 99thPercentileLatency 1 61156 28.58264111 1665 747000 918 747 927.7820691 0.132 6630.776 83 33000 50 33 2075.808675 0.504 9767.828 32 0 18.08734375 0.465 207.159 Total 61156 28.58264111 1665 747000 918 747 927.7820691 0.132 6630.776 83 33000 50 33 2075.808675 0.504 9767.828 32 0 0 0 18.08734375 0.465 207.159 [mongo@mysqlstd ~]$ /data/mongodb/mongodb/bin/mongostat connected to: 127.0.0.1 insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time *0 5568 5369 *0 0 5381|0 0 3.06g 6.47g 282m 9 admin:0.9% 0 0|0 0|0 1m 894k 95 02:14:05 *0 4298 6267 *0 0 6279|0 0 3.06g 6.47g 282m 6 admin:0.6% 0 0|0 1|0 1m 962k 96 02:14:06 *0 4675 6119 *0 0 6066|0 0 3.06g 6.47g 282m 2 admin:0.0% 0 95|0 1|0 1m 948k 92 02:14:07 *0 4137 4866 *0 0 4948|0 0 3.06g 6.47g 282m 18 admin:2.1% 0 0|0 0|0 1m 790k 91 02:14:08 *0 4568 5904 *0 0 5922|0 0 3.06g 6.47g 282m 4 admin:0.1% 0 0|0 0|0 1m 927k 92 02:14:09 *0 4727 6034 *0 0 6046|0 0 3.06g 6.47g 282m 5 admin:0.0% 0 0|0 0|0 1m 949k 90 02:14:10 *0 4991 5673 *0 0 5690|0 0 3.06g 6.47g 282m 3 admin:0.9% 0 0|0 0|0 1m 914k 94 02:14:11
11. www.vmcd.org *0 4740 5173 *0 0 5183|0 1 3.06g 6.47g 282m 7 admin:0.1% 0 0|0 0|0 1m 839k 94 02:14:12 *0 4332 5493 *0 0 5510|0 0 3.06g 6.47g 282m 8 admin:0.9% 0 0|0 0|0 1m 866k 94 02:14:13 *0 4980 5583 *0 0 5592|0 0 3.06g 6.47g 282m 8 admin:0.0% 0 0|0 0|0 1m 901k 97 02:14:14 insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time *0 5750 5030 *0 0 4997|0 0 3.06g 6.47g 282m 20 admin:1.8% 0 94|0 1|1 1m 853k 97 02:14:15 *0 4884 5509 *0 0 5578|0 0 3.06g 6.47g 282m 10 admin:0.1% 0 0|0 0|0 1m 894k 97 02:14:16 *0 5733 5773 *0 0 5784|0 0 3.06g 6.47g 282m 5 admin:0.0% 0 0|0 0|0 1m 952k 92 02:14:17 *0 5178 5202 *0 0 5219|0 0 3.06g 6.47g 282m 14 admin:0.0% 0 0|0 0|0 1m 861k 95 02:14:18 *0 4179 5680 *0 0 5688|0 0 3.06g 6.47g 282m 8 admin:0.0% 0 0|0 0|1 1m 884k 93 02:14:19 *0 4879 5695 *0 0 5707|0 0 3.06g 6.47g 282m 11 admin:0.1% 0 0|0 0|0 1m 911k 93 02:14:20 *0 5271 5402 *0 0 5413|0 0 3.06g 6.47g 282m 12 admin:0.0% 0 0|0 0|0 1m 887k 95 02:14:21 *0 4583 4852 *0 0 4867|0 1 3.06g 6.47g 282m 11 admin:0.0% 0 0|0 0|0 1m 795k 93 02:14:22 *0 6654 4956 *0 0 4967|0 0 3.06g 6.47g 282m 10 admin:1.5% 0 0|0 0|0 1m 881k 95 02:14:23
12. www.vmcd.org REF: http://www.aerospike.com/blog/entrepreneurs-break-all-the-rules-aerospike-goes-open-source/ https://github.com/thumbtack-technology/ycsb http://www.aerospike.com/wp-content/uploads/2013/02/Ultra-High-Performance-NoSQL-Benchmarking_zh-CN.pdf https://github.com/brianfrankcooper/YCSB/wiki http://labs.yahoo.com/news/yahoo-cloud-serving-benchmark/