Experimental / VCR Test System / VCR Test System
Version current

VCR Test System

This feature is experimental and the API may change in future releases.

Overview

The VCR (Video Cassette Recorder) test system provides a way to record and replay LLM and embedding API calls during testing. This approach offers several benefits:

  • Deterministic tests - Replay recorded responses for consistent test results

  • Cost reduction - Avoid repeated API calls during test runs

  • Speed improvement - Playback from local Redis is much faster than API calls

  • Offline testing - Run tests without network access after initial recording

The VCR system uses Redis for cassette storage with AOF/RDB persistence, allowing recorded data to be committed to version control and shared across team members.

Quick Start

Add Dependencies

To use the VCR test utilities, add the following test dependencies alongside RedisVL:

<!-- Maven -->
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.10.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.19.7</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.19.7</version>
    <scope>test</scope>
</dependency>
// Gradle
testImplementation 'org.junit.jupiter:junit-jupiter:5.10.2'
testImplementation 'org.testcontainers:testcontainers:1.19.7'
testImplementation 'org.testcontainers:junit-jupiter:1.19.7'

Basic Usage

The recommended approach uses @VCRTest on the test class and @VCRModel on model fields for automatic wrapping:

import com.redis.vl.test.vcr.VCRTest;
import com.redis.vl.test.vcr.VCRMode;
import com.redis.vl.test.vcr.VCRModel;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.chat.ChatLanguageModel;
import org.junit.jupiter.api.Test;

@VCRTest(mode = VCRMode.PLAYBACK_OR_RECORD)
public class MyLLMTest {

    // Models are automatically wrapped by VCR - initialize at field declaration
    @VCRModel(modelName = "text-embedding-3-small")
    private EmbeddingModel embeddingModel = createEmbeddingModel();

    @VCRModel
    private ChatLanguageModel chatModel = createChatModel();

    @Test
    void testEmbedding() {
        // First run: Records API response to Redis
        // Subsequent runs: Replays from Redis cassette
        Response<Embedding> response = embeddingModel.embed("What is Redis?");
        assertNotNull(response.content());
    }

    @Test
    void testChat() {
        String response = chatModel.generate("Explain Redis in one sentence.");
        assertNotNull(response);
    }

    private static EmbeddingModel createEmbeddingModel() {
        String key = System.getenv("OPENAI_API_KEY");
        if (key == null) key = "vcr-playback-mode";  // Dummy key for playback
        return OpenAiEmbeddingModel.builder()
            .apiKey(key)
            .modelName("text-embedding-3-small")
            .build();
    }

    private static ChatLanguageModel createChatModel() {
        String key = System.getenv("OPENAI_API_KEY");
        if (key == null) key = "vcr-playback-mode";
        return OpenAiChatModel.builder()
            .apiKey(key)
            .modelName("gpt-4o-mini")
            .build();
    }
}
Models must be initialized at field declaration time, not in @BeforeEach. The VCR extension wraps @VCRModel fields before @BeforeEach methods run.

VCR Modes

The VCR system supports six modes to control recording and playback behavior:

Mode Description

PLAYBACK

Always replay from cassettes. Fails if cassette not found.

RECORD

Always record new cassettes, overwriting existing ones.

RECORD_NEW

Record only if cassette is missing; otherwise replay.

RECORD_FAILED

Re-record cassettes for previously failed tests; replay successful ones.

PLAYBACK_OR_RECORD

Replay if cassette exists; record if not. Best for general use.

OFF

Disable VCR entirely - all API calls go through normally.

  • Development: Use PLAYBACK_OR_RECORD for convenience

  • CI/CD: Use PLAYBACK to ensure tests are deterministic

  • Initial setup: Use RECORD to capture all cassettes

Environment Variable Override

Override the VCR mode at runtime using the VCR_MODE environment variable. This takes precedence over the annotation mode:

# Force RECORD mode to capture new cassettes
VCR_MODE=RECORD OPENAI_API_KEY=your-key ./gradlew test

# Force PLAYBACK mode (no API key required)
VCR_MODE=PLAYBACK ./gradlew test

# Force OFF mode to bypass VCR
VCR_MODE=OFF OPENAI_API_KEY=your-key ./gradlew test

Valid values: PLAYBACK, PLAYBACK_OR_RECORD, RECORD, RECORD_NEW, RECORD_FAILED, OFF

Configuration

Data Directory

Configure where cassette data is stored:

@VCRTest(
    mode = VCRMode.PLAYBACK_OR_RECORD,
    dataDir = "src/test/resources/vcr-data"  // default
)
public class MyTest {
    // ...
}

The data directory contains Redis persistence files (RDB/AOF) that can be committed to version control.

Redis Image

Specify a custom Redis image:

@VCRTest(
    mode = VCRMode.PLAYBACK,
    redisImage = "redis/redis-stack:7.2.0-v6"  // default
)
public class MyTest {
    // ...
}

Method-Level Overrides

@VCRRecord

Force recording for a specific test method:

@VCRTest(mode = VCRMode.PLAYBACK)
public class MyTest {

    @Test
    void normalTest() {
        // Uses PLAYBACK mode from class annotation
    }

    @Test
    @VCRRecord
    void alwaysRecordThisTest() {
        // Forces RECORD mode for this test only
    }
}

@VCRDisabled

Disable VCR for a specific test:

@VCRTest(mode = VCRMode.PLAYBACK_OR_RECORD)
public class MyTest {

    @Test
    @VCRDisabled
    void testWithRealAPI() {
        // VCR bypassed - makes real API calls
    }
}

VCR Registry

The VCR Registry tracks the recording status of each test:

Status Description

RECORDED

Test has a valid cassette recording

FAILED

Previous recording attempt failed

MISSING

No cassette exists for this test

OUTDATED

Cassette exists but may need re-recording

Smart modes like RECORD_NEW and RECORD_FAILED use registry status to make intelligent decisions about when to record.

Architecture

The VCR system consists of several components:

  • VCRExtension - JUnit 5 extension that manages the VCR lifecycle

  • VCRContext - Manages Redis container, call counters, and statistics

  • VCRRegistry - Tracks recording status for each test

  • VCRCassetteStore - Stores/retrieves cassettes in Redis using JSON format

Cassette Key Format

Cassettes are stored in Redis with keys following this pattern:

vcr:{type}:{testId}:{callIndex}

For example:

vcr:llm:MyTest.testGeneration:0001
vcr:embedding:MyTest.testEmbedding:0001

Statistics

The VCR system tracks statistics during test execution:

  • Cache Hits - Number of successful cassette replays

  • Cache Misses - Number of cassettes not found (triggering API calls in record mode)

  • API Calls - Number of actual API calls made

Best Practices

Version Control

Commit your vcr-data/ directory to version control:

src/test/resources/vcr-data/
├── dump.rdb          # RDB snapshot
└── appendonlydir/    # AOF segments

CI/CD Integration

Use strict PLAYBACK mode in CI to ensure deterministic tests:

@VCRTest(mode = VCRMode.PLAYBACK)
public class CITest {
    // Tests will fail if cassettes are missing
}

Updating Cassettes

When API responses change, re-record cassettes using the VCR_MODE environment variable:

# Run tests in RECORD mode to update all cassettes
VCR_MODE=RECORD ./gradlew test

Sensitive Data

Be mindful of what gets recorded:

  • API responses may contain sensitive information

  • Consider filtering or redacting sensitive data

  • Use .gitignore patterns if needed

Example: Testing with SemanticCache

import com.redis.vl.test.vcr.VCRTest;
import com.redis.vl.test.vcr.VCRMode;
import com.redis.vl.extensions.cache.llm.SemanticCache;
import org.junit.jupiter.api.Test;

@VCRTest(mode = VCRMode.PLAYBACK_OR_RECORD)
public class SemanticCacheVCRTest {

    @Test
    void testCacheWithRecordedEmbeddings() {
        // Embeddings are recorded on first run
        SemanticCache cache = new SemanticCache(config, jedis);

        cache.store("What is Redis?", "Redis is an in-memory data store...");

        CacheResult result = cache.check("Tell me about Redis");
        assertTrue(result.isHit());
    }
}

Troubleshooting

Cassette Not Found

If you see "cassette not found" errors in PLAYBACK mode:

  1. Ensure cassettes were recorded (run in RECORD mode first)

  2. Check the data directory path matches

  3. Verify Redis persistence files exist

Inconsistent Results

If test results vary between runs:

  1. Ensure you’re using a fixed VCR mode

  2. Check for non-deterministic test logic

  3. Verify cassette data wasn’t corrupted

Container Issues

If Redis container fails to start:

  1. Ensure Docker is running

  2. Check port availability

  3. Verify the Redis image exists

Framework Integration

The VCR system provides drop-in wrappers for popular AI frameworks. These wrappers work standalone without requiring any other RedisVL components.

LangChain4J

Embedding Model

Use VCREmbeddingModel to wrap any LangChain4J EmbeddingModel:

import com.redis.vl.test.vcr.VCREmbeddingModel;
import com.redis.vl.test.vcr.VCRMode;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.openai.OpenAiEmbeddingModel;
import dev.langchain4j.data.embedding.Embedding;
import dev.langchain4j.model.output.Response;

// Create your LangChain4J embedding model
EmbeddingModel openAiModel = OpenAiEmbeddingModel.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .modelName("text-embedding-3-small")
    .build();

// Wrap with VCR for recording/playback
VCREmbeddingModel vcrModel = new VCREmbeddingModel(openAiModel);
vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);
vcrModel.setTestId("MyTest.testEmbedding");

// Use exactly like the original model - VCR handles caching transparently
Response<Embedding> response = vcrModel.embed("What is Redis?");
float[] vector = response.content().vector();

// Batch embeddings also supported
List<TextSegment> segments = List.of(
    TextSegment.from("Document chunk 1"),
    TextSegment.from("Document chunk 2")
);
Response<List<Embedding>> batchResponse = vcrModel.embedAll(segments);

The VCREmbeddingModel implements the full dev.langchain4j.model.embedding.EmbeddingModel interface, so it can be used anywhere a LangChain4J embedding model is expected.

Supported Methods

  • embed(String text) - Single text embedding

  • embed(TextSegment segment) - TextSegment embedding

  • embedAll(List<TextSegment> segments) - Batch embedding

  • dimension() - Returns embedding dimensions

Chat Model

Use VCRChatModel to wrap any LangChain4J ChatLanguageModel:

import com.redis.vl.test.vcr.VCRChatModel;
import com.redis.vl.test.vcr.VCRMode;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.output.Response;

// Create your LangChain4J chat model
ChatLanguageModel openAiModel = OpenAiChatModel.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .modelName("gpt-4o-mini")
    .build();

// Wrap with VCR for recording/playback
VCRChatModel vcrModel = new VCRChatModel(openAiModel);
vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);
vcrModel.setTestId("MyTest.testChat");

// Use exactly like the original model - VCR handles caching transparently
Response<AiMessage> response = vcrModel.generate(UserMessage.from("What is Redis?"));
String answer = response.content().text();

// Simple string convenience method
String simpleAnswer = vcrModel.generate("Tell me about Redis Vector Library");

// Multiple messages
Response<AiMessage> chatResponse = vcrModel.generate(
    UserMessage.from("What is Redis?"),
    UserMessage.from("How does it handle vectors?")
);

The VCRChatModel implements the full dev.langchain4j.model.chat.ChatLanguageModel interface, so it can be used anywhere a LangChain4J chat model is expected.

Supported Chat Methods

  • generate(ChatMessage…​ messages) - Generate from varargs messages

  • generate(List<ChatMessage> messages) - Generate from list of messages

  • generate(String text) - Simple string convenience method

Spring AI

Embedding Model

Use VCRSpringAIEmbeddingModel to wrap any Spring AI EmbeddingModel:

import com.redis.vl.test.vcr.VCRSpringAIEmbeddingModel;
import com.redis.vl.test.vcr.VCRMode;
import org.springframework.ai.embedding.EmbeddingModel;
import org.springframework.ai.embedding.EmbeddingResponse;
import org.springframework.ai.openai.OpenAiEmbeddingModel;
import org.springframework.ai.document.Document;

// Create your Spring AI embedding model
EmbeddingModel openAiModel = new OpenAiEmbeddingModel(openAiApi);

// Wrap with VCR for recording/playback
VCRSpringAIEmbeddingModel vcrModel = new VCRSpringAIEmbeddingModel(openAiModel);
vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);
vcrModel.setTestId("MyTest.testSpringAIEmbedding");

// Use exactly like the original model
float[] vector = vcrModel.embed("What is Redis?");

// Batch embeddings
List<float[]> vectors = vcrModel.embed(List.of("text 1", "text 2"));

// Document embedding
Document doc = new Document("Document content here");
float[] docVector = vcrModel.embed(doc);

// Full EmbeddingResponse API
EmbeddingResponse response = vcrModel.embedForResponse(List.of("query text"));

The VCRSpringAIEmbeddingModel implements the full org.springframework.ai.embedding.EmbeddingModel interface for seamless integration with Spring AI applications.

Supported Methods

  • embed(String text) - Single text embedding returning float[]

  • embed(Document document) - Document embedding

  • embed(List<String> texts) - Batch embedding returning List<float[]>

  • embedForResponse(List<String> texts) - Full EmbeddingResponse

  • call(EmbeddingRequest request) - Standard Spring AI call pattern

  • dimensions() - Returns embedding dimensions

Chat Model

Use VCRSpringAIChatModel to wrap any Spring AI ChatModel:

import com.redis.vl.test.vcr.VCRSpringAIChatModel;
import com.redis.vl.test.vcr.VCRMode;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.openai.OpenAiChatModel;

// Create your Spring AI chat model
ChatModel openAiModel = new OpenAiChatModel(openAiApi);

// Wrap with VCR for recording/playback
VCRSpringAIChatModel vcrModel = new VCRSpringAIChatModel(openAiModel);
vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);
vcrModel.setTestId("MyTest.testChat");

// Use exactly like the original model - VCR handles caching transparently
String response = vcrModel.call("What is Redis?");

// With Prompt object
Prompt prompt = new Prompt(List.of(new UserMessage("Explain vector search")));
ChatResponse chatResponse = vcrModel.call(prompt);
String answer = chatResponse.getResult().getOutput().getText();

// Multiple messages
String multiResponse = vcrModel.call(
    new UserMessage("What is Redis?"),
    new UserMessage("How does it handle vectors?")
);

The VCRSpringAIChatModel implements the full org.springframework.ai.chat.model.ChatModel interface for seamless integration with Spring AI applications.

Supported Chat Methods

  • call(String message) - Simple string convenience method

  • call(Message…​ messages) - Generate from varargs messages

  • call(Prompt prompt) - Full Prompt/ChatResponse API

Common VCR Operations

Both wrappers share common VCR functionality:

// Set VCR mode
vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);

// Set test identifier (for cassette key generation)
vcrModel.setTestId("MyTestClass.testMethod");

// Reset call counter between tests
vcrModel.resetCallCounter();

// Get statistics
int hits = vcrModel.getCacheHits();
int misses = vcrModel.getCacheMisses();
int recorded = vcrModel.getRecordedCount();

// Reset statistics
vcrModel.resetStatistics();

// Access underlying delegate
EmbeddingModel delegate = vcrModel.getDelegate();

Using with JUnit 5

Combine VCR wrappers with the @VCRTest annotation for a complete testing solution:

import com.redis.vl.test.vcr.VCRTest;
import com.redis.vl.test.vcr.VCRMode;
import com.redis.vl.test.vcr.VCREmbeddingModel;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

@VCRTest(mode = VCRMode.PLAYBACK_OR_RECORD)
public class EmbeddingServiceTest {

    private VCREmbeddingModel vcrModel;

    @BeforeEach
    void setUp() {
        EmbeddingModel realModel = OpenAiEmbeddingModel.builder()
            .apiKey(System.getenv("OPENAI_API_KEY"))
            .build();
        vcrModel = new VCREmbeddingModel(realModel);
        vcrModel.setMode(VCRMode.PLAYBACK_OR_RECORD);
    }

    @Test
    void testSemanticSearch() {
        vcrModel.setTestId("EmbeddingServiceTest.testSemanticSearch");

        // First run: calls OpenAI API and records response
        // Subsequent runs: replays from Redis cassette
        Response<Embedding> response = vcrModel.embed("search query");

        assertNotNull(response);
        assertEquals(1536, response.content().vector().length);
    }

    @Test
    void testBatchEmbedding() {
        vcrModel.setTestId("EmbeddingServiceTest.testBatchEmbedding");

        List<TextSegment> docs = List.of(
            TextSegment.from("Document 1"),
            TextSegment.from("Document 2"),
            TextSegment.from("Document 3")
        );

        Response<List<Embedding>> response = vcrModel.embedAll(docs);

        assertEquals(3, response.content().size());
    }
}

See Also