Architecture¶

This page covers the key architectural decisions behind fastapi-redis-sdk and the reasoning that shaped them.

Connection lifecycle¶

All components - DI factories and CacheBackend - share a single async Redis connection pool managed by the FastAPI lifespan.

Every request borrows from the same pool rather than opening a new TCP socket. This bounds the number of connections to Redis, avoids per-request TLS handshakes, and keeps connection limits predictable under load.

Pool construction is cheap - redis-py pools are lazy and allocate empty bookkeeping (~48 bytes, ~4 µs) without opening any sockets.

Tying the pool to the lifespan gives deterministic startup and shutdown. The pool is guaranteed to exist before the first request and is drained gracefully when the app stops - no leaked sockets, no race conditions.

sequenceDiagram
    participant App as FastAPI
    participant LS as FastAPIRedis(app).lifespan()
    participant PS as _PoolState
    participant R as Redis

    App->>LS: startup
    LS->>PS: build async pool (no TCP yet)

    loop Every request
        App->>PS: borrow connection
        PS->>R: connect on first use, then GET / SET / DEL
        R-->>PS: response
        PS-->>App: return to pool
    end

    App->>LS: shutdown
    LS->>PS: close pool
    PS->>R: disconnect all sockets

Lifespan wrapping¶

FastAPIRedis(app).lifespan() wraps the app's existing lifespan rather than replacing it. FastAPI only accepts one lifespan, so if the library owned it outright, the user would have to manually compose it with every other library's lifespan. Wrapping avoids this - multiple builder calls just nest around whatever is already there, starting up in registration order and tearing down in reverse.

This relies on app.router.lifespan_context, which is not part of Starlette's public API but has been stable since 0.20+ and is used internally by FastAPI's own router lifespan merging. Nesting order is determined by call order rather than being explicitly visible, which can make debugging startup hangs harder.

For scenarios where explicit ordering matters, skip .lifespan() and compose manually using redis_lifespan:

from contextlib import asynccontextmanager
from redis_fastapi import FastAPIRedis, redis_lifespan

@asynccontextmanager
async def my_lifespan(app):
    async with redis_lifespan(app):
        async with db_lifespan(app):
            yield

app = FastAPI(lifespan=my_lifespan)
FastAPIRedis(app).caching()   # no .lifespan() - user owns the lifespan

The builder methods are independent - .caching() does not require .lifespan() (but it is recommended).

Async-first design and sync endpoint support¶

fastapi-redis-sdk is async-only at the transport layer - the sole Redis connection pool is an redis.asyncio pool. There is no sync redis.Redis pool. This section explains why that works for both async def and plain def endpoints.

For background on how FastAPI handles async def vs def, see Concurrency and async / await (especially the Very Technical Details section).

How FastAPI dispatches endpoints and dependencies¶

Declaration	Where it runs	Blocking I/O safe?
`async def endpoint(…)`	Main event loop	No - would block all requests
`def endpoint(…)`	Worker threadpool (`anyio.to_thread`)	Yes
`async def dependency(…)`	Main event loop	No
`def dependency(…)`	Worker threadpool	Yes

All of fastapi-redis-sdk's DI factories (cache(), cache_evict(), cache_put(), get_cache_backend()) are async def. They run on the event loop and use await for every Redis call - no thread is blocked. Sync endpoints that declare these as Depends(…) still work correctly: FastAPI awaits the async dependency on the event loop, then hands the resolved value to the sync endpoint which runs in the threadpool.

`CacheBackendDep` - async endpoints¶

CacheBackendDep injects a CacheBackend whose methods (get, set, delete, has, delete_group) are all coroutines. Use it from async def endpoints:

@app.get("/items/{item_id}")
async def get_item(item_id: int, cb: CacheBackendDep) -> dict:
    cached = await cb.get(f"item:{item_id}")
    if cached:
        return cached
    item = await fetch_item(item_id)
    await cb.set(f"item:{item_id}", item, ttl=300)
    return item

`SyncCacheBackendDep` - sync endpoints¶

Sync (def) endpoints cannot await. SyncCacheBackendDep provides a blocking wrapper that bridges each call back to the event loop via anyio.from_thread.run:

@app.get("/items/{item_id}")
def get_item(item_id: int, cb: SyncCacheBackendDep) -> dict:
    cached = cb.get(f"item:{item_id}")      # blocking - runs on event loop
    if cached:
        return cached
    item = fetch_item(item_id)
    cb.set(f"item:{item_id}", item, ttl=300)
    return item

Under the hood, each SyncCacheBackend method does:

worker thread                         event loop
─────────────                         ──────────
cb.set("k", v, ttl=60)
  └─ anyio.from_thread.run(lambda)
       └─ schedules ──────────────► await backend.set("k", v, ttl=60)
          blocks thread ◄──────────  result / exception

This only works from threads managed by FastAPI's AnyIO threadpool (sync endpoints and sync dependencies). Calling SyncCacheBackend from the main thread or an arbitrary thread raises RuntimeError.

Why no sync Redis pool?¶

The library's purpose is high-level DI features (caching, rate limiting, sessions) - not raw Redis access. All of those features use the async client internally. Maintaining a parallel sync pool would mean:

Doubling pool-management code in the lifespan.
Keeping two client wrappers (Redis + AsyncRedis) in sync.
Opening a second set of TCP connections that most apps never use.

Users who need a raw sync redis.Redis client can create one outside the library in two lines.

The DI caching factories (`cache()`, `cache_evict()`, `cache_put()`)¶

These work with both async def and def endpoints without any extra setup. They are async def generators resolved by FastAPI's DI system before the endpoint runs. The endpoint function itself never touches Redis - caching is handled entirely in the dependency and middleware layers. See Caching § Sync endpoint support for details.

Why dependency injection, not decorators¶

Many FastAPI caching libraries - most notably fastapi-cache2 - use a @cache decorator that wraps the endpoint function. fastapi-redis-sdk deliberately avoids this pattern and uses FastAPI's native dependency injection (Depends()) instead. The decorator approach has five concrete problems in FastAPI:

Signature rewriting is fragile. A caching decorator must inject hidden Request / Response parameters via __signature__ manipulation. FastAPI relies on function introspection for validation, OpenAPI generation, and dependency resolution; rewriting the signature operates outside that system and is a known source of breakage (fastapi#1743, fastapi#5065, article).
Conflicts with other DI-based libraries. Decorator-based route wrapping prevents other Depends()-based libraries (pagination, security, DB sessions) from initializing correctly (fastapi-cache#557, fastapi-cache#89).
dependency_overrides cannot reach decorator internals. FastAPI's primary test-mocking mechanism only works with Depends() callables. Logic inside a decorator bypasses the DI container entirely (fastapi#4330).
Decorator order is silently significant. @app.get must come before @cache, which must come before @cache_evict. Reversing the order silently breaks caching. DI dependencies are resolved as a graph - no ordering constraints.
Ecosystem mismatch. FastAPI consistently models cross-cutting concerns as dependencies: authentication (Depends(get_current_user)), database sessions (Depends(get_db)). The official documentation specifically shows this pattern for side-effect-only concerns - which is exactly what caching is.

Our DI-based design (cache(), cache_evict(), cache_put()) resolves all five issues. On cache hit, the dependency raises a CacheHitException (caught by a registered exception handler) that returns the cached response directly - the endpoint never executes. On cache miss, a lightweight capture middleware stores the response in Redis after the endpoint returns. See Caching § Caching factories for usage.

Why not a full ASGI middleware for cache reads?¶

An earlier design used an ASGI middleware to intercept requests before routing and return cached responses without entering the FastAPI pipeline at all. In theory this is the fastest possible path - no DI resolution, no routing.

In practice, benchmarks showed no measurable improvement over the pure-DI approach. The ~0.5–2 ms saved by skipping FastAPI's pipeline is dwarfed by the Redis round-trip and client network latency (see Benchmarks). The middleware also introduced problems the DI path does not have:

Route registry fragility. The middleware runs before DI, so per-route config (TTL, eviction group, key builder) must be pre-computed into a lookup table at startup. This breaks with lazy route registration, dynamic routes, and mounted sub-applications.
Two mechanisms to coordinate. Users must register both the middleware and the per-route dependency; forgetting the middleware silently disables caching.
Cross-layer key consistency. Eviction and write-through dependencies must produce the same cache keys as the middleware - an error-prone coupling.
Harder to test. dependency_overrides covers the DI config but not the middleware; tests require ASGI-level fixtures.

The current design keeps a single lightweight middleware (CacheResponseCaptureMiddleware) solely for miss-path writes - it buffers the response body and stores it in Redis after the endpoint returns. Cache reads and short-circuiting happen entirely in the DI layer via CacheHitException.

Cache scope¶

cache() stores text representations — JSON, but also plain text, HTML, CSV, XML and JavaScript. The word JSON describes the envelope, not what you may return: an entry is a JSON document whose body is a text field, stored alongside the response's own header fields — so the media type it was served as, and the rest of its metadata, are replayed on a hit. See Headers a hit carries.

Unstructured payloads are covered. A FastAPI endpoint that returns a bare string or a number sends it as JSON — return "hello" goes over the wire as "hello" with Content-Type: application/json — so scalars need no special handling; they fall under the application/json row below.

The content type decides whether we store, and the library checks it against an allowlist, refusing anything it does not recognise rather than storing a representation it cannot reproduce:

Stored	Refused
`application/json`	`application/octet-stream`, `application/pdf`, `application/msgpack`, `application/x-protobuf`
Anything `application/…+json` or `application/…+xml`	`image/`, `audio/`, `video/*` — including `image/svg+xml`
`application/xml`, `application/javascript`	`multipart/*`
Any `text/*`	A response with no `Content-Type` at all
`charset=utf-8`, `charset=us-ascii`, or no charset	Any other charset — `iso-8859-1`, `utf-16`

A refusal never changes what the caller receives. The response is served whole, with its own headers and status, and only the Redis write is skipped. You can see it two ways: the response carries X-Redis-Cache: BYPASS, and the library logs the reason once per route:

WARNING  /thumbnails/{id} was served but not cached: content type 'image/png'
         is not a text representation.  cache() stores serializable text
         representations only; see 'Cache scope' in the caching guide.

Once per route and reason, not once per request — a refused route will not flood your logs.

Binary belongs somewhere else¶

cache() refuses binary by choice, not by necessity. An entry is a JSON envelope with a text body, and base64 could carry arbitrary bytes through it — an earlier version of this library did exactly that. So no claim below rests on the format being unable to hold the bytes. The reasons are about HTTP semantics and about where binary belongs, and they hold whatever your storage costs:

A CDN serves those bytes closer to the user, and serves them correctly. It answers from a node near the caller and implements Range and conditional requests properly — the two things this library refuses rather than half-supports. Binary is where Range matters most: seeking in video, paging a large PDF. When the endpoint sets no ETag of its own the stored validator is a weak one, and a weak validator cannot serve a range at all.
Binary is usually immutable and content-addressed — asset digests, thumbnail hashes. A long max-age at the edge then needs no invalidation, which is the one advantage Redis has over a CDN, and the one you would not be using.
Memory is a sizing question, not a wall. Ten thousand 200 KB thumbnails is 2 GB, and on open-source Redis that is 2 GB of RAM. On Redis Software or Redis Cloud, Flex tiers warm values onto locally attached NVMe: the RAM limit floors at 10% of total memory, keeping at least 20% of values in RAM is recommended, key names stay in RAM whatever happens to their values, and cold reads cost milliseconds rather than microseconds. If you run on Flex, read this bullet as capacity planning rather than as an objection.

MAX_CACHEABLE_BODY_SIZE is unrelated to all three: it bounds what your ASGI worker buffers in process memory while the middleware captures the body, which no storage tier affects.

cache() earns its keep on the opposite shape: small, costly-to-compute, often-requested payloads that change, where the saving is the computation rather than the bytes.

Caching a streamed response defeats the streaming¶

The capture middleware has to see a whole body before it can store it. A StreamingResponse on a cached route is therefore drained in full before the client receives its first byte, and the peak buffer is one body per request in flight.

Do not put cache() on a route that streams for a reason. If the point of streaming is time-to-first-byte or a body too large to hold in memory, caching it takes both away.

Bodies over MAX_CACHEABLE_BODY_SIZE (10 MiB) are passed through and marked BYPASS, so an oversized response is a refusal rather than a memory problem.

`Vary` is not honoured¶

The cache key comes from the request path and its sorted query parameters — never from a request header. default_key_builder cannot see Accept, Accept-Encoding, Authorization or Range, so a Vary header on the response does not affect which entry is read.

It is still stored and replayed. Dropping it would strip the origin's instruction from every cache downstream of you as well, so a CDN in front would treat your single stored variant as the only one there is. A response carrying Vary: * is refused outright: RFC 9111 §4.1 says such a response may never be reused, and the lookup ignores Vary, so refusing the store is the only place that rule can be honoured.

One URL that negotiates on a request header therefore has one entry, and the first variant stored is the one everyone gets. If a route serves WebP to clients that accept it and JPEG to the rest, or switches language on Accept-Language, pass a key_builder that folds the deciding header into the key:

def key_with_language(request, eviction_group="", prefix=""):
    base = default_key_builder(request, eviction_group=eviction_group, prefix=prefix)
    lang = request.headers.get("accept-language", "*")
    return f"{base}:lang={lang}"

@app.get("/articles/{slug}", dependencies=[Depends(cache(ttl=300, key_builder=key_with_language))])
async def article(slug: str): ...

Fold in only the header you negotiate on. A key that includes Accept-Encoding or the full Accept splits the entry across every browser variant and the hit rate collapses.

Headers a hit carries¶

An entry stores the body, the validator, and every header field the endpoint sent apart from five groups. A hit is rebuilt from that block, so the representation metadata your endpoint set — Link, Content-Disposition, Content-Language, X-Total-Count, anything of your own — arrives on the hit exactly as it did on the miss, repeated fields and their order included.

What an entry deliberately leaves out:

Group	Fields	Why
Owned by this library	`Cache-Control`, `ETag`, `X-Redis-Cache`	Re-emitted on every hit from the entry's own TTL and validator
Connection-specific (RFC 9110 §7.6.1)	`Connection` and the fields it names, `Keep-Alive`, `Proxy-Connection`, `TE`, `Transfer-Encoding`, `Upgrade`	A recipient must remove them before forwarding
Proxy-specific (RFC 9111 §3.1)	`Proxy-Authenticate`, `Proxy-Authentication-Info`, `Proxy-Authorization`	A `MUST NOT` unless the proxy's identity is in the key, which it is not
Framing	`Content-Length`	Recomputed from the replayed body
Policy	`Date`, `Set-Cookie`	See the two warnings below

A 304 carries less still: per RFC 9110 §15.4.5 only ETag, Cache-Control, Vary, Content-Location and Expires are replayed, since the rest describes a body the response does not contain.

Cache-Control is worth calling out: on a cached route this library owns the field outright, on the miss as well as the hit. An endpoint that sets Cache-Control: public, max-age=600 on a route wrapped in cache() has that value replaced by the library's own, so both responses carry one consistent policy. Use the private=True argument rather than the header.

Bodies have MAX_CACHEABLE_BODY_SIZE; header blocks have MAX_CACHEABLE_HEADER_SIZE (8 KiB). A response whose metadata exceeds it is served and marked BYPASS, like any other refusal.

A cached route cannot set cookies

Set-Cookie is not stored, which is the safe direction — a shared entry that replayed one caller's session cookie to the next caller would be a session leak. The cost is that a cached route silently stops setting cookies after the first request: the caller who takes the miss gets the cookie, everyone served from the entry does not.

Do not put cache() on a route that establishes a session, sets a CSRF token, or otherwise depends on Set-Cookie.

To supply one of the withheld fields on every response, set it in middleware rather than in the endpoint. Middleware runs outside the cache, so it is applied to a hit as well as to a miss, and its value is recomputed per response instead of being stored. That is the remedy for the cookie limitation below, and the right home for anything per-request — a request id, a trace header, a fresh signature.

Extension responses are not marked

A response the middleware cannot buffer — http.response.pathsend, zerocopysend, trailers, debug — is forwarded whole and never stored, but it carries no X-Redis-Cache header at all rather than BYPASS, since nothing about it was cached and it must not claim a MISS. It is the one served-but-not-stored case you cannot spot from the response alone.

A per-caller header leaks like a per-caller body

Because an entry now stores the headers your endpoint set, a header whose value depends on who asked is replayed to everyone the entry serves. A route returning X-Account-Tier or X-User-Id hands the first caller's value to every later caller, exactly as a per-caller body would.

The remedy is the same one: fold the caller into the key with a custom key_builder, as under Authenticated routes are not keyed per user. If a header is diagnostic rather than part of the representation — a request id, a trace id — set it in middleware instead, where it is recomputed per response and never stored.

Compression middleware must wrap the cache¶

The capture middleware stores the body it sees. If a compression middleware sits inside it, what it sees is already compressed, Content-Encoding is set, and every response is refused — caching is off for every client that sends Accept-Encoding: gzip, which is every browser, with nothing to show it but one log line.

Starlette applies the last registered middleware outermost, so register compression after caching():

app = FastAPI()
FastAPIRedis(app).lifespan().caching()
app.add_middleware(GZipMiddleware, minimum_size=1000)   # outside the cache

# Wrong - the cache sees gzip bytes and refuses every response
app.add_middleware(GZipMiddleware, minimum_size=1000)
FastAPIRedis(app).lifespan().caching()

With the correct order the cache stores the identity representation and the compression middleware compresses both the miss and the hit on the way out. The same applies to any middleware that rewrites the body — encrypt, sign, minify: it belongs outside caching(), or the entry stores its output instead of your endpoint's.

An endpoint that returns an already-compressed body of its own — setting Content-Encoding by hand — is refused whatever the ordering. RFC 9110 §8.4 makes Content-Encoding the instruction for decoding the body, and an entry that stored the bytes without it would replay something no client can read.

Storage model - strings vs hashes¶

Every cached entry is stored as a standalone Redis string key, with eviction groups encoded as key prefixes. Eviction-group deletion uses SCAN + DEL to find and remove matching keys.

Redis hashes would be a natural fit - one hash per eviction group, one field per entry. Since Redis 7.4 added per-field expiration and Redis 8.0 added HSETEX, the main historical blocker is gone. Group deletion becomes a single DEL (~59× faster than SCAN + DEL at 1000 entries), and memory drops ~5% from eliminating per-key overhead. Reads and writes are effectively identical in latency.

Strings remain the default for three reasons: HSETEX requires Redis ≥ 8.0 and many deployments still run 7.x; key-level features (keyspace notifications, MEMORY USAGE per entry) don't work on hash fields;

Rate limiting - command tiers and capability detection¶

The rate limiter uses the fixed-window (window-counter) pattern and executes it through the best command the connected server supports, degrading in two tiers:

Tier	Command	Requires	Atomic
1	`INCREX … BYINT … UBOUND … EX … ENX`	Redis 8.8+	Yes (single command)
2	`EVAL` Lua window counter	Server-side scripting (Redis 2.6+)	Yes (script)

INCREX is the preferred path: a single round trip that increments, enforces the upper bound, and sets the TTL only on window creation (ENX). When the server rejects it as an unknown command the limiter falls back to an equivalent atomic Lua script. Both tiers are atomic — there is deliberately no non-atomic degrade: a server supporting neither (scripting disabled) surfaces as a backend error and takes the fail-open/closed path rather than counting with a read-modify-write race.

Detection is per process, not per request¶

Which tier a server supports never changes at runtime, so probing for it on every request would waste a failed round trip on the hot path for the entire life of a pre-8.8 deployment. Detection state (INCREX support plus the registered Lua script) therefore lives on a shared, pool-lifetime capability cache held on _PoolState - the same object that owns the connection pool.

Observability¶

The tier that served a check is surfaced two ways: the RateLimitResult.backend field ("increx" / "lua") and, when OpenTelemetry is enabled, the ratelimit.backend attribute on the ratelimit.hit / ratelimit.global spans. This makes an unexpected fallback (e.g. INCREX silently unavailable in production) visible in traces.

Telemetry¶

Observability is layered into three independent tiers - HTTP request spans (FastAPI instrumentation), fastapi-redis-sdk operation spans and metrics for both caching and rate limiting, and redis-py's native driver metrics. Each tier can be enabled on its own, and the first two compose into one nested trace per request. The design is deliberately additive: the library's own layer never wraps or replaces the FastAPI or redis-py instrumentation - it only slots between them, which is why the three can be toggled independently.

See the Observability guide for the layer model, the full span and metric reference (caching and rate limiting), and how to enable each tier.