Replication Type

RIOT-X offers two different mechanisms for reading and writing keys:

  • Dump & restore (default)

  • Data structure replication (--struct)

Dump & Restore

The default replication mechanism is Dump & Restore:

  1. Scan for keys in the source Redis database. If live replication is enabled the reader also subscribes to keyspace notifications to generate a continuous stream of keys.

  2. Reader threads iterate over the keys to read corresponding values (DUMP) and TTLs.

  3. Reader threads enqueue key/value/TTL tuples into the reader queue, from which the writer dequeues key/value/TTL tuples and writes them to the target Redis database by calling RESTORE and EXPIRE.

Data Structure Replication

There are situations where Dump & Restore cannot be used, for example:

  • The target Redis database does not support the RESTORE command (Redis Enterprise CRDB)

  • Incompatible DUMP formats between source and target (Redis 7.0)

In those cases you can use another replication strategy that is data structure-specific: each key is introspected to determine its type and then use the corresponding read/write commands.

Type Read Write

Hash

HGETALL

HSET

JSON

JSON.GET

JSON.SET

List

LRANGE

RPUSH

Set

SMEMBERS

SADD

Sorted Set

ZRANGE

ZADD

Stream

XRANGE

XADD

String

GET

SET

TimeSeries

TS.RANGE

TS.ADD
This replication strategy is more intensive in terms of CPU, memory, and network for all the machines involved (source Redis, target Redis, and RIOT-X machines). Adjust number of threads, batch and queue sizes accordingly.
Type-based replication example
riotx replicate --struct redis://source redis://target
Live type-based replication example
riotx replicate --struct --mode live redis://source redis://target