From Redis

Migration With RedisShake

Use RedisShake v4.6.2 with sync_reader and redis_writer. It reads an initial RDB snapshot from Redis, restores it into Lavik, and continues forwarding the source replication stream. Your application can keep writing to Redis during import; reserve a write pause for final verification and cutover. This page covers both Redis and Redis Cluster sources.

This procedure targets a fresh Meta-managed Lavik Single or Cluster deployment. Create the target and its replicas before importing; RedisShake writes through the current Group Owners and Lavik replicates those writes. Direct REPLICAOF import into a non-Meta node is a different deployment path.

The flags describe each side’s client protocol, not its number of machines. A Single deployment can have replicas. Source shards and target Groups do not need a one-to-one mapping.

1. Prepare The Source And Empty Target

• Rehearse with a copy of your actual workload first. The release-qualified source is Redis 7.2.14; rehearse other versions separately, especially newer RDB encodings, module types, Lua Functions and Streams.
• Inventory every source primary, logical DB, data type, command and Function library. Lavik Single accepts DB0–15; Cluster accepts DB0 only. Resolve nonzero DBs and cross-slot application operations before selecting a Cluster target. Do not silently discard databases or commands with filters.
• Create an isolated, empty Lavik target using the deployment guides below. Require every Group to be serving and all intended replicas to be synchronized and readable. Use v0.2.0 on every target node. Keep all application reads and writes off the target until cutover.
• The migration host needs Linux x86_64 or ARM64, Bash, curl, tar, sha256sum, Python 3 and redis-cli. Budget disk for both the RDB snapshot and the incremental spool; watch disk usage, source snapshot load, replication backlog and target memory during rehearsal.
• Allow the tool to authenticate, read replication and discover source topology, and write the required commands to the target. Managed Redis services may restrict PSYNC; confirm availability before scheduling the migration. Keep source topology stable throughout the attempt.

Set Up A Managed Single Deployment →

Set Up A Primary–Follower Cluster →

The loopback examples below use Redis at 127.0.0.1:6379 (Cluster seed 127.0.0.1:7000) and Lavik at 127.0.0.1:6371, matching the binary deployment guides. Replace these with endpoints reachable from the migration process; Docker’s 127.0.0.1 means that container, not another container or the host.

For non-loopback traffic, use peer-authenticated encrypted tunnels for every source and target connection, including discovered Cluster nodes. RedisShake v4.6.2’s tls = true does not verify server certificates. Its status listener binds all interfaces: block external access to port 18080 and query it locally.

2. Install RedisShake v4.6.2

Run the following in Bash on the migration host. It selects the Linux architecture, verifies the release archive against its pinned SHA-256, and installs the binary in your user directory. Keep this shell open so SHAKE_BIN remains set.

# Run in Bash on a Linux migration host with curl, tar and sha256sum.
set -euo pipefail
case "$(uname -m)" in
  x86_64)
    arch=amd64
    checksum=c255565a60c15f9d1b4d38ef46d30056161266ffa75e11065c7b1901e236bff7
    ;;
  aarch64|arm64)
    arch=arm64
    checksum=5fc2b7a393f159f8981c1e8d712b63c0866628b99ad030ed3e46c9869c22ff04
    ;;
  *) echo "Use a Linux x86_64 or ARM64 host" >&2; exit 1 ;;
esac
export SHAKE_BIN="$HOME/.local/share/redis-shake-4.6.2/redis-shake"
archive=$(mktemp)
curl --fail --location --retry 3 \
  "https://github.com/tair-opensource/RedisShake/releases/download/v4.6.2/redis-shake-v4.6.2-linux-${arch}.tar.gz" \
  --output "$archive"
printf '%s  %s\n' "$checksum" "$archive" | sha256sum --check
mkdir -p "$(dirname "$SHAKE_BIN")"
tar -xzf "$archive" -C "$(dirname "$SHAKE_BIN")" ./redis-shake
rm "$archive"
test -x "$SHAKE_BIN"

Create a private directory for this attempt. If you choose a different directory or disk, update both absolute paths in the configuration. Never reuse an old spool after a failed attempt.

# Use a new directory for EVERY attempt. Choose a disk with space for the spool.
set -euo pipefail
export MIGRATION_DIR=/var/tmp/lavik-import-01
umask 077
# Deliberately fail if this attempt directory already exists.
mkdir "$MIGRATION_DIR"
mkdir "$MIGRATION_DIR/spool"
touch "$MIGRATION_DIR/shake.toml"
chmod 600 "$MIGRATION_DIR/shake.toml"
# Paste ONE of the configurations below into this file and edit its endpoints.
${EDITOR:-vi} "$MIGRATION_DIR/shake.toml"

3. Configure A Redis Source

Paste this complete TOML into shake.toml. It migrates Redis to managed Lavik Single. To write to Lavik Cluster, change only [redis_writer].cluster to true. Keep sync_rdb and sync_aof enabled to perform both baseline import and incremental catch-up.

[sync_reader]
address = "127.0.0.1:6379"
username = ""
password = "replace-source-password"
cluster = false
prefer_replica = false
tls = false
sync_rdb = true
sync_aof = true

[redis_writer]
address = "127.0.0.1:6371"
username = ""
password = "replace-target-password"
cluster = false
tls = false
off_reply = false

[advanced]
dir = "/var/tmp/lavik-import-01/spool"
log_file = "/var/tmp/lavik-import-01/shake.log"
status_port = 18080
log_interval = 1
rdb_restore_command_behavior = "panic"
target_redis_proto_max_bulk_len = 512000000

Replace both password placeholders with the credentials for each endpoint. Set the source username if Redis uses a named ACL user. Lavik 0.2.0 uses its default user with requirepass; leave the target username empty or use default. For an intentionally unauthenticated local rehearsal, set the corresponding password to "". TOML does not expand shell environment variables. Keep this file mode 0600 and out of Git.

4. Configure A Redis Cluster Source

Use this alternative complete configuration instead of the previous one. It discovers the source primaries from one seed and writes to a Lavik Cluster seed. If consolidating into managed Lavik Single, change only [redis_writer].cluster to false. Do not concatenate the two configurations or run both against the same target.

[sync_reader]
# One source seed; every discovered primary must also be reachable.
address = "127.0.0.1:7000"
username = ""
password = "replace-source-password"
cluster = true
prefer_replica = false
tls = false
sync_rdb = true
sync_aof = true

[redis_writer]
# One target seed; every advertised Lavik endpoint must be reachable.
address = "127.0.0.1:6371"
username = ""
password = "replace-target-password"
cluster = true
tls = false
off_reply = false

[advanced]
dir = "/var/tmp/lavik-import-01/spool"
log_file = "/var/tmp/lavik-import-01/shake.log"
status_port = 18080
log_interval = 1
rdb_restore_command_behavior = "panic"
target_redis_proto_max_bulk_len = 512000000

• Before starting, use CLUSTER NODES and CLUSTER SLOTS to inventory the source primaries, slots and advertised addresses. Verify authentication and reachability to every discovered source primary and target Owner from the migration host. A reachable seed alone is insufficient.
• Keep resharding, source failovers and topology changes out of the migration window. The source and target can have different shard counts, but you must validate every source primary and every target Group independently.
• Plan Function libraries and keyless commands explicitly per target Group. Combining multiple source primaries can expose conflicting library definitions or different meanings of FLUSH operations; shard-count matching does not resolve them.

5. Start Import And Follow Live Writes

: "${SHAKE_BIN:?Set the absolute path to the pinned redis-shake binary}"
: "${MIGRATION_DIR:?Set the absolute directory for this attempt}"
cd "$MIGRATION_DIR"
# Leave this terminal running through baseline import and incremental catch-up.
"$SHAKE_BIN" "$MIGRATION_DIR/shake.toml"

Keep this terminal running. In a second terminal, inspect the status and watch the configured shake.log; check free spool space with df -h /var/tmp/lavik-import-01. Errors about authentication, unsupported types, busy keys or failed commands require stopping and investigating the attempt. A running process does not establish successful import.

# Run in a second terminal on the migration host.
curl --fail --silent --show-error http://127.0.0.1:18080/status \
  | python3 -m json.tool

6. Freeze Source Writes And Prove Catch-Up

Stop all source writers, including jobs, administrative scripts and stream consumers that change group state. Keep Redis available for replication and verification. Record the following for each primary; for Cluster, connect directly to each primary’s address rather than repeatedly querying the seed.

# First stop ALL application, job and administrative writers on Redis.
# Repeat for every source primary (7000, 7001, 7002, ... for a Cluster).
# --askpass avoids putting a password in shell history or process arguments.
# For an ACL user, also add: --user migration-reader
redis-cli -h 127.0.0.1 -p 6379 --askpass INFO server
redis-cli -h 127.0.0.1 -p 6379 --askpass INFO replication
# Record run_id, role:master, master_replid and master_repl_offset per primary.

• Match each RedisShake reader to its source address. Every reader must report syncing aof, with both aof_received_offset and aof_sent_offset at or beyond that primary’s recorded master_repl_offset.
• Every target writer must have unanswered_entries = 0 and unanswered_bytes = 0. A single reader caught up, zero RDB work, or consistent = true alone is not a cutover decision.
• Recheck source identity and target ownership through the verification window. A source primary change, restart or target Owner transition invalidates that proof; stop the attempt and follow the recovery procedure below.

7. Verify Every Group, Replica And Data Type

After queues drain, confirm replica progress with a private write/delete marker and WAIT on the same connection to each current Owner. A fresh connection running only WAIT is not proof that the imported writes reached replicas. Pick an unused marker key in that Owner’s slot range; do not use redirection between the marker and WAIT.

# Connect directly to one CURRENT Lavik Owner. Keep this connection open.
redis-cli -h 127.0.0.1 -p 6371 --askpass
# Enter these commands at that interactive prompt, not in separate invocations:
# SET __lavik_migration__:{group1-probe}:barrier ready NX
# DEL __lavik_migration__:{group1-probe}:barrier
# WAIT 1 5000
# Require SET -> OK, DEL -> 1 and WAIT -> at least 1 for a one-replica Group.
# Choose a private, unused marker key whose slot belongs to THIS Owner.
# Repeat for each Group; adjust WAIT's count to its required replica count.

If SET does not return OK, stop: choose another unused key before deleting anything. If WAIT returns fewer replicas than required, keep the application off the target and investigate. This is a per-Group progress check, not a global commit barrier or a guarantee against acknowledged-write loss during asynchronous failover.

Compare the complete source and target key sets using a full cursor-based SCAN of every database and every primary/Group; reject missing AND extra keys. Also read every intended target replica. For a direct Cluster replica connection, issue READONLY and the reads on the same connection. Key counts and a few sampled GETs are useful diagnostics, but insufficient for final verification.

8. Stop RedisShake And Switch The Application

• Once the offset, queue, replica and data checks all pass with stable identities, stop RedisShake with Ctrl+C in its original terminal. Confirm the process has exited and no importer is still writing before allowing application traffic. Keep the source write-frozen.
• Update application endpoints and credentials. Use the normal Lavik Single/Sentinel or Redis Cluster client discovery appropriate to the target; Cluster clients must discover every advertised node. Reconnect existing pools and resume writers against Lavik only.
• Run your application’s real read/write paths, transactions, Lua/Function calls and stream consumers. Monitor errors, latency, replication health and capacity. Retain the stopped source and backups for the agreed rollback window.
• Before Lavik accepts new writes, reverting endpoints can return to the frozen source. After new writes reach Lavik, a switch back requires reconciling those writes; this procedure does not provide automatic reverse replication.

If An Attempt Fails

For this v4.6.2 workflow, do not assume reliable checkpoint resume. A lost reply may correspond to a committed write; replaying an uncertain INCR, list push or other non-idempotent suffix can change the result. RESTORE REPLACE does not make the whole migration idempotent.

• Stop the importer on process exit, connectivity loss, target Owner transition or verification failure. Keep application traffic off the target; let Meta complete target recovery and preserve logs for diagnosis.
• Prefer a fresh, isolated target and a new attempt/spool directory. Ensure all old importers have exited and in-flight commands have drained. Restart from a complete baseline with both sync_rdb and sync_aof enabled, then repeat every cutover check. Source writes may resume while rebuilding the new attempt.
• Reusing an isolated import target requires explicitly clearing both keys and Function catalogs on every current Owner and confirming the empty state on replicas after all writers stop. Never clear a serving production database to repair an import.

RedisShake forwards writes without preserving MULTI/EXEC transaction boundaries and does not create a global snapshot across shards. Keep the partial target out of service until the write-frozen comparison passes.

Redis Import Details For Lavik 0.2.0 →