Benchmark performance#
Redis®* memtier_benchmark* is a command line utility developed by Redis Labs for load generation and performing benchmark NoSQL key-value databases. It is recommended for performing benchmark of Aiven for Redis®* services.
Warning
redis-benchmark is not supported to work with Aiven services, since CONFIG command is not allowed to run.
Prerequisites#
An Aiven for Redis service running.
memtier_benchmarkinstalled. To install the tool, first download the source code from GitHub, and follow the instructions in the README to install all the dependencies, then build and install the tool. Now it’s ready to go.
Note
The Testing section in README is not mandatory, can be skipped.
Running benchmark#
Before using memtier_benchmark, read the help from mentier_benchmark -h or this Redis article to understand what the tool can offer.
The following variables need to be substituted when running the commands. You can find the information in the Overview tab of your Aiven for Redis service.
Variable |
Description |
|---|---|
|
User name of Aiven for Redis connection |
|
Password of Aiven for Redis connection |
|
Hostname for Redis connection |
|
Port for Redis connection |
Below is a sample command from the Redis blog. Each run of the benchmarking tool consists of executing 10000 (-n 10000) SET & GET operations with 1:1 ratio (--ratio 1:1) by launching 4 threads (-t 4) and each thread opening 25 connections (-c 25). The tool does 10 iterations (-x 10) of each run to collect meaningful aggregate averages.
memtier_benchmark -a 'USERNAME:PASSWORD' -s 'HOST' -p 'PORT' --tls --tls-skip-verify -t 4 -n 10000 --ratio 1:1 -c 25 -x 10 -d 100 --key-pattern S:S
Your output may be similar to this:
Writing results to stdout
[RUN #1] Preparing benchmark client...
[RUN #1] Launching threads now...
[RUN #1 100%, 216 secs] 0 threads: 1000000 ops, 1996 (avg: 4621) ops/sec, 277.88KB/sec (avg: 642.15KB/sec), 50.54 (avg: 21.63) msec latency
<<<< many lines for RUN #2 to RUN #9
[RUN #10] Preparing benchmark client...
[RUN #10] Launching threads now...
[RUN #10 100%, 224 secs] 0 threads: 1000000 ops, 4116 (avg: 4444) ops/sec, 572.83KB/sec (avg: 617.53KB/sec), 24.40 (avg: 22.49) msec latency
4 Threads
25 Connections per thread
10000 Requests per client
BEST RUN RESULTS
============================================================================================================================
Type Ops/sec Hits/sec Misses/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
----------------------------------------------------------------------------------------------------------------------------
Sets 2404.11 --- --- 21.62541 20.60700 48.89500 112.63900 339.90
Gets 2404.11 2404.11 0.00 21.62707 20.60700 49.15100 105.98300 328.16
Waits 0.00 --- --- --- --- --- --- ---
Totals 4808.23 2404.11 0.00 21.62624 20.60700 49.15100 111.10300 668.06
Request Latency Distribution
Type <= msec Percent
------------------------------------------------------------------------
SET 11.327 0.000
<<<< many lines
GET 66.559 100.000
---
WAIT 0.000 100.000
WORST RUN RESULTS
============================================================================================================================
Type Ops/sec Hits/sec Misses/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
----------------------------------------------------------------------------------------------------------------------------
Sets 2249.10 --- --- 22.94219 21.63100 47.87100 109.56700 317.98
Gets 2249.10 2249.10 0.00 22.94561 21.63100 47.87100 109.05500 307.00
Waits 0.00 --- --- --- --- --- --- ---
Totals 4498.20 2249.10 0.00 22.94390 21.63100 47.87100 109.56700 624.99
Request Latency Distribution
Type <= msec Percent
------------------------------------------------------------------------
SET 10.047 0.000
<<<< many lines
GET 191.487 100.000
---
WAIT 0.000 100.000
AGGREGATED AVERAGE RESULTS (10 runs)
============================================================================================================================
Type Ops/sec Hits/sec Misses/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
----------------------------------------------------------------------------------------------------------------------------
Sets 2312.01 --- --- 22.42681 21.24700 47.35900 101.88700 326.88
Gets 2312.01 2312.01 0.00 22.42914 21.24700 47.35900 101.88700 315.59
Waits 0.00 --- --- --- --- --- --- ---
Totals 4624.02 2312.01 0.00 22.42798 21.24700 47.35900 101.88700 642.47
Request Latency Distribution
Type <= msec Percent
------------------------------------------------------------------------
SET 9.791 0.000
<<<< many lines
GET 712.703 100.000
---
WAIT 0.000 100.000
This example demonstrates what performance data memtier_benchmark can collect. The beginning sections are the data of the 10 runs executed. The following sections present, among the 10 runs, the BEST RUN, WORST RUN and AGGREGATED AVERAGE results as well as the Request Latency Distribution of the operations.
Running the same command on different Redis services or on the same service in different conditions can effectively benchmark the performance.
Note
Aiven has rate limit on services. By default it’s 200 new connections per 0.5 second per CPU core. Also be aware of the connection limit depending on memory size as explained in Estimate maximum number of connection.