Redis integration guide for pandas¶
Redis must run separately with the Query Engine (Search). See installation and driver discovery for Python/native setup, supported wheel platforms, TLS and source builds. Tables must be created, ingested or adopted through the driver; arbitrary Redis keys do not automatically become SQL tables. Writes autocommit without rollback.
This guide describes current repository capabilities. A newly added helper or
extra may require a build containing its implementation. If a released wheel
lacks it, build the native driver from this checkout and install the relevant
extra from ./python; set REDIS_ADBC_LIBRARY to the built library. Do not infer
that an older published wheel contains every helper documented here.
pandas SQL reads¶
Install adbc-driver-redis[dbapi] and pandas to pass an open ADBC
connection directly to read_sql or read_sql_query:
import os
import pandas as pd
from adbc_driver_redis import dbapi
with dbapi.connect(os.environ["REDIS_URI"]) as conn:
df = pd.read_sql_query("SELECT 42 AS answer", conn, dtype_backend="pyarrow")
print(df)
Direct reads are covered by python/tests/test_pandas_direct_reads.py: default and Arrow
DataFrame types, SQL NULLs, Unicode, integers above 2^53, microsecond timestamps,
filtering, aggregation, index selection, empty results and multiple Arrow
batches. Local validation used pandas 3.1.0, PyArrow 26.0.0, ADBC manager 1.13.0
and Redis Software with Redis 8.6.2; see the
validation record.
The Arrow backend preserves nullable integer types. These tests cover SQL
queries; pandas table reads, parameters and chunksize are outside this
validation. For write coverage, see the next section.
pandas writes and round trips¶
pandas 2.2+ accepts an open ADBC connection directly for
read_sql
and DataFrame.to_sql.
Install adbc-driver-redis[dbapi] and pandas>=2.2. This example creates a
unique table, verifies the fetched rows and removes the table afterward:
import os
import uuid
import pandas as pd
from adbc_driver_redis import dbapi
table = "pandas_demo_" + uuid.uuid4().hex
source = pd.DataFrame({"id": [1, 2, 3], "amount": [10.5, 20.0, 30.25]})
with dbapi.connect(os.environ["REDIS_URI"]) as conn:
try:
source.to_sql(table, conn, index=False)
result = pd.read_sql(f'SELECT * FROM "{table}" ORDER BY id', conn)
pd.testing.assert_frame_equal(result, source)
print(result)
finally:
with conn.cursor() as cur:
cur.execute(f'DROP TABLE IF EXISTS "{table}"')
print("REDIS_ADBC_READ_WRITE_OK")
Use if_exists="append" to append rows. if_exists="replace" drops and
recreates the table; writes autocommit and cannot be rolled back. For nullable
Arrow dtypes on reads, pass dtype_backend="pyarrow". No SQLAlchemy engine is
needed for this path.
Run the focused integration tests against a disposable Redis with Search:
REDIS_URI=redis://localhost:6379/0 python -m pytest -q python/tests/test_pandas_writes.py
The recorded run used pandas 3.1.0, PyArrow 26.0.0 and ADBC manager 1.13.0; see validation.
Rerun this guide¶
From the repository root, with the guide's dependencies and native driver
installed, set REDIS_URI to the dedicated local Software fixture and run:
python python/scripts/check_integration_guide.py pandas
This executes the guide's code blocks or its linked runnable example. For Superset it exercises the documented backend workflow in an isolated application; it does not automate the browser UI. See the per-guide results.