Choose the latency budget before the connection count

When sizing a Redis-compatible service, the useful number is often the throughput that fits your latency budget. Lavik’s September 18 SPDK benchmark makes that distinction concrete: doubling connections beyond its 10-million-key GET peak reduced throughput while sharply increasing p99. For teams considering NVMe-backed value capacity, the connection sweep is more useful than the winning bar alone.

Lavik is an Apache 2.0 beta project that keeps a compact key index in DRAM and stores values on storage. The benchmark used the downloaded standard 0.1.0-beta.1 package at revision 3955b98d43b312324aa8d52775df52cfb111c0d0, with SPDK storage and kernel TCP. Its results do not describe the minimal package’s io_uring path.

The point after the peak matters

For 10 million keys with 1 KiB values, Lavik delivered 803,016 GET QPS at 320 connections with 0.959 ms p99. At 640 connections, throughput reached 1,012,180 QPS and p99 rose to 3.599 ms. At 1,280 connections, throughput fell to 940,713 QPS while p99 reached 9.791 ms. More outstanding work bought no additional throughput at that last point. These measurements identify an unfavorable operating point, although they do not isolate its internal cause.

Average latency also helps explain what the connection count means. With pipeline=1, each connection has at most one request outstanding. At the 640-connection point, roughly 1.012 million requests per second multiplied by 0.632 ms average latency gives about 640 outstanding requests. At 1,280 connections, roughly 941,000 requests per second multiplied by 1.360 ms gives about 1,279. This arithmetic describes the measured request population; p99 cannot replace the average in that calculation.

An illustrative 1 ms budget changes the selection

Suppose the selection rule is a client-observed GET p99 below 1 ms. This is an illustrative filter, not a promised SLA. Among the supplied 10M-key measurements, Lavik’s highest qualifying throughput was 803,016 QPS at 320 connections, with 0.959 ms p99. Redis 8.8.0’s was 769,130 QPS with 16 I/O threads and 320 connections, at 0.975 ms p99. Valkey 9.1.0’s was 692,905 QPS with 16 I/O threads and 160 connections, at 0.519 ms p99. Each result is the best qualifying sampled point across that product’s supplied configurations, not an interpolated limit.

For Valkey, that filter retains the 16-I/O-thread configuration used in the report’s peak GET comparison, but changes the selected connection count from 1,280 to 160. It also exposes a percentile tradeoff: at those three qualifying points, p99.9 was 2.023 ms for Lavik, 1.663 ms for Redis, and 0.815 ms for Valkey. Meeting a p99 threshold does not establish the same result at p99.9. Lavik’s 0.959 ms p99 also leaves little margin below the illustrative threshold; a single 30-second observation cannot establish reliable operating headroom.

Keep the workload attached to the number

The server had an AMD EPYC 9V74, eight physical cores / 16 logical CPUs, approximately 126 GiB RAM, and six NVMe devices. The client ran on a separate host. The 10M group used uniform random GET or overwriting SET, 1,024-byte values, pipeline=1, no rate limit, and 30-second windows across 80–1,280 connections. Its default correlated client seeds are part of the scope. This is a concurrency-driven sweep, so it does not establish how the service handles an independently imposed arrival rate or a burst while preserving a latency target.

The independent billion-key group used 60-second windows, distinct client seeds, and a sweep extending to 2,560 connections. Lavik’s GET peak was 952,560 QPS at 640 connections with 3.599 ms p99. The matching p99 number from the smaller dataset does not make the workloads interchangeable. Writes give another reason to retain both metrics: Lavik peaked at 764,939 SET QPS with 10.879 ms p99, while Garnet 2.1.5 peaked at 751,057 QPS with 5.535 ms p99; both points used 1,280 connections. A throughput-only ranking hides that write-tail difference.

The report contains 22 newly measured Lavik SPDK points and 124 reused peer-control points from an earlier sweep on the same hosts. Products ran sequentially, once per command, connection count, and thread configuration. Cache policies, warmup, memory reservations, and persistence settings differ; Redis and Valkey had AOF and automatic snapshots disabled. All formal points reported zero connection errors and measured GETs had zero misses, but Garnet’s post-test exact count and value-length check remain unverified. Those checks and short windows cannot establish equal crash durability or an SLA.

Turn the sweep into an evaluation decision

For your own evaluation, write down the required throughput, p99 and p99.9 budgets, and acceptable failure semantics before choosing concurrency. Then measure representative value sizes, access skew, command mix, and pipeline depth across a connection sweep. Include sustained overwrites and background reclamation, and repeat measurements around the candidate operating point. The decision should be whether that point retains margin under your traffic; the supplied results are a starting hypothesis for this work.

A successful write response is not a synchronous fsync fence. Evaluate durability, replication, and recovery requirements for your workload.

Begin with the basic-commands recipe below to check a local installation, then move to application-specific compatibility and load evaluation. The recipe uses a different storage path from the SPDK benchmark and does not reproduce its performance. Consult Platform and packaging for Linux, io_uring, and glibc requirements. The existing article “Evaluate the Redis client’s second attempt” is a useful companion when planning compatibility checks.

set -euo pipefail
if [ ! -x ./lavik ]; then
  echo "Run this from the extracted Lavik package directory (the folder containing ./lavik)." >&2
  exit 1
fi
mkdir -p /tmp/lavik-example
if [ ! -e /tmp/lavik-example/data ]; then
  fallocate -l 512M /tmp/lavik-example/data
fi
./lavik --bind 127.0.0.1 --port 6379 --metrics-port 0 \
  --network kernel --storage uring \
  --threads 2 --no-pin-workers --registered-buffer-mb-per-worker 64 \
  --shutdown-checkpoint --data-file /tmp/lavik-example/data \
  --log-dir /tmp/lavik-example/logs

$ redis-cli -h 127.0.0.1 -p 6379 --raw PING
PONG

$ redis-cli -h 127.0.0.1 -p 6379 --raw SET greeting 'hello from Lavik'
OK

$ redis-cli -h 127.0.0.1 -p 6379 --raw GET greeting
hello from Lavik

$ redis-cli -h 127.0.0.1 -p 6379 --raw HSET user:42 name Ada
1

$ redis-cli -h 127.0.0.1 -p 6379 --raw HGET user:42 name
Ada

{
  "recipeId": "basic-commands",
  "platform": "Linux-7.0.12-linuxkit-aarch64-with-glibc2.39",
  "steps": [
    {
      "argv": [
        "redis-cli",
        "-h",
        "127.0.0.1",
        "-p",
        "6379",
        "--raw",
        "PING"
      ],
      "expected": "PONG",
      "actual": "PONG"
    },
    {
      "argv": [
        "redis-cli",
        "-h",
        "127.0.0.1",
        "-p",
        "6379",
        "--raw",
        "SET",
        "greeting",
        "hello from Lavik"
      ],
      "expected": "OK",
      "actual": "OK"
    },
    {
      "argv": [
        "redis-cli",
        "-h",
        "127.0.0.1",
        "-p",
        "6379",
        "--raw",
        "GET",
        "greeting"
      ],
      "expected": "hello from Lavik",
      "actual": "hello from Lavik"
    },
    {
      "argv": [
        "redis-cli",
        "-h",
        "127.0.0.1",
        "-p",
        "6379",
        "--raw",
        "HSET",
        "user:42",
        "name",
        "Ada"
      ],
      "expected": "1",
      "actual": "1"
    },
    {
      "argv": [
        "redis-cli",
        "-h",
        "127.0.0.1",
        "-p",
        "6379",
        "--raw",
        "HGET",
        "user:42",
        "name"
      ],
      "expected": "Ada",
      "actual": "Ada"
    }
  ],
  "status": "passed",
  "binaryVersion": "lavik 0.1.0-beta.1",
  "executableOnPath": false,
  "workingDirectory": "/opt/lavik",
  "sourceCommit": "3955b98d43b312324aa8d52775df52cfb111c0d0",
  "packageVersion": "v0.1.0-beta.1",
  "gracefulRestart": "passed"
}