Package com.redis.vl.storage

Class BaseStorage

java.lang.Object
com.redis.vl.storage.BaseStorage
Direct Known Subclasses:
HashStorage, JsonStorage
public abstract class BaseStorage extends Object
Base class for internal storage handling in Redis.

Provides foundational methods for key management, data preprocessing, validation, and basic read/write operations.

  • Field Details

    • indexSchema

      protected final IndexSchema indexSchema
      The index schema for this storage instance.

    • defaultBatchSize

      protected int defaultBatchSize
      Default batch size for bulk operations.

  • Constructor Details

    • BaseStorage

      public BaseStorage(IndexSchema indexSchema)
      Creates a new BaseStorage instance.
      Parameters:
      indexSchema - The index schema for this storage

  • Method Details

    • createKey

      protected static String createKey(String id, String prefix, String keySeparator)
      Create a Redis key using a combination of a prefix, separator, and the identifier.
      Parameters:
      id - The unique identifier for the Redis entry
      prefix - A prefix to append before the key value
      keySeparator - A separator to insert between prefix and key value
      Returns:
      The fully formed Redis key

    • preprocessObject

      protected static Map<String,Object> preprocessObject(Map<String,Object> obj, Function<Map<String,Object>,Map<String,Object>> preprocess)
      Apply a preprocessing function to the object if provided.
      Parameters:
      obj - Object to preprocess
      preprocess - Function to process the object
      Returns:
      Processed object as a map

    • getIndexSchema

      protected IndexSchema getIndexSchema()
      Get the index schema.
      Returns:
      The index schema for this storage

    • createKeyForObject

      protected String createKeyForObject(Map<String,Object> obj, String idField)
      Construct a Redis key for a given object, optionally using a specified field from the object as the key.
      Parameters:
      obj - The object from which to construct the key
      idField - The field to use as the key, if provided
      Returns:
      The constructed Redis key for the object
      Throws:
      IllegalArgumentException - If the id_field is not found in the object

    • getKeys

      protected List<String> getKeys(List<Map<String,Object>> objects, List<String> keys, String idField)
      Generate Redis keys for a list of objects.
      Parameters:
      objects - List of objects
      keys - Optional iterable of keys
      idField - Field to use as the key
      Returns:
      List of generated keys

    • validate

      protected Map<String,Object> validate(Map<String,Object> obj)
      Validate an object against the schema.
      Parameters:
      obj - The object to validate
      Returns:
      Validated object

    • preprocessAndValidateObjects

      protected List<BaseStorage.KeyValuePair> preprocessAndValidateObjects(List<Map<String,Object>> objects, String idField, List<String> keys, Function<Map<String,Object>,Map<String,Object>> preprocess, boolean doValidate)
      Preprocess and validate a list of objects.
      Parameters:
      objects - List of objects to preprocess and validate
      idField - Field to use as the key
      keys - Optional list of keys
      preprocess - Optional preprocessing function
      doValidate - Whether to validate against schema
      Returns:
      List of tuples (key, processed_obj) for valid objects

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects, String idField, List<String> keys, Integer ttl, Integer batchSize, Function<Map<String,Object>,Map<String,Object>> preprocess, boolean doValidate)
      Write a batch of objects to Redis. This method returns a list of Redis keys written to the database.
      Parameters:
      redisClient - A Redis client used for writing data
      objects - An iterable of objects to store
      idField - Field used as the key for each object
      keys - Optional list of keys, must match the length of objects if provided
      ttl - Time-to-live in seconds for each key
      batchSize - Number of objects to write in a single Redis pipeline execution
      preprocess - A function to preprocess objects before storage
      doValidate - Whether to validate objects against schema
      Returns:
      List of keys written to Redis

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects)
      Write a batch of objects to Redis with default parameters.
      Parameters:
      redisClient - Redis client
      objects - Objects to write
      Returns:
      List of keys written

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects, String idField, List<String> keys)
      Write with id field.
      Parameters:
      redisClient - Redis client
      objects - Objects to write
      idField - Field to use as key
      keys - Custom keys
      Returns:
      List of keys written

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects, String idField, List<String> keys, Integer ttl)
      Write with TTL.
      Parameters:
      redisClient - Redis client
      objects - Objects to write
      idField - Field to use as key
      keys - Custom keys
      ttl - Time to live in seconds
      Returns:
      List of keys written

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects, String idField, List<String> keys, Integer ttl, Integer batchSize)
      Write with batch size.
      Parameters:
      redisClient - Redis client
      objects - Objects to write
      idField - Field to use as key
      keys - Custom keys
      ttl - Time to live
      batchSize - Batch size for pipeline
      Returns:
      List of keys written

    • write

      public List<String> write(UnifiedJedis redisClient, List<Map<String,Object>> objects, String idField, List<String> keys, Integer ttl, Integer batchSize, Function<Map<String,Object>,Map<String,Object>> preprocess)
      Write with preprocessing.
      Parameters:
      redisClient - Redis client
      objects - Objects to write
      idField - Field to use as key
      keys - Custom keys
      ttl - Time to live
      batchSize - Batch size
      preprocess - Preprocessing function
      Returns:
      List of keys written

    • get

      public List<Map<String,Object>> get(UnifiedJedis redisClient, Collection<String> keys, Integer batchSize)
      Retrieve objects from Redis by keys.
      Parameters:
      redisClient - Synchronous Redis client
      keys - Keys to retrieve from Redis
      batchSize - Number of objects to retrieve in a single Redis pipeline execution
      Returns:
      List of objects pulled from redis

    • get

      public List<Map<String,Object>> get(UnifiedJedis redisClient, Collection<String> keys)
      Retrieve objects from Redis by keys with default batch size.
      Parameters:
      redisClient - Redis client
      keys - Keys to retrieve
      Returns:
      List of objects

    • convertBytes

      protected Map<String,Object> convertBytes(Map<String,Object> map)
      Convert byte arrays in the map to appropriate types.
      Parameters:
      map - Map with potential byte arrays
      Returns:
      Map with converted values

    • set

      protected abstract void set(Pipeline pipeline, String key, Map<String,Object> obj)
      Set a key-value pair in Redis using a pipeline.
      Parameters:
      pipeline - The Redis pipeline to use
      key - The Redis key
      obj - The object to store

    • getResponse

      protected abstract Response<Map<String,Object>> getResponse(Pipeline pipeline, String key)
      Get a response for retrieving a value from Redis using a pipeline.
      Parameters:
      pipeline - The Redis pipeline to use
      key - The Redis key
      Returns:
      Response containing the retrieved object