Test Data Workbench#
Point it at a database. Get realistic, relationally intact test data back.
Test Data Workbench connects to a database, reads its schema, works out what each table and column represents, and writes Python generators that produce plausible data for it. The generators come out as readable source you can edit and commit, insertion order respects foreign keys, and no real data is ever read, so nothing personal enters the pipeline.
It supports SQLite and PostgreSQL, installs as a single command-line tool, and takes about a minute to try.
Try it#
pipx install test-data-workbench
# Zero-setup: build a small SQLite shop database and run the full flow
tdw demo --seed 42
# Or against your own database
tdw analyze "sqlite:///shop.db"
tdw deploy "sqlite:///shop.db" --output ./generated --seed 42
Installation covers the other install methods and the PostgreSQL extra.
Where to go from here#
Every page stands on its own, so start wherever your question is.
Deciding whether this is the right tool. The problem with test data explains why test data is harder than it looks and how the usual approaches fail. The approach taken explains what this tool does instead and why it emits code rather than rows. Scope and boundaries is the honest boundary: who it is for, when it is the wrong choice, and what it will never do.
Getting it running. Installation and Quickstart.
Understanding how it works. Architecture follows the pipeline end to end. Schema Adaptation covers how an unknown schema is classified, The Fallback Ladder covers what happens when that classification fails, and Privacy and Data Access covers the data-access guarantees and their limits.
Looking up a specific technology. Stack overview is a menu of the whole stack, one page each, covering what the library does here, the idea underneath it, the sharp edges, and verified links for going deeper.
Command and API details. CLI Reference, API Reference, and Testing and Quality.
Note
These docs describe the system as it actually behaves. Where a capability is deliberately out of scope, or an implementation has a limit worth knowing, the relevant page says so rather than leaving it to be discovered.
Introduction
Getting Started
Concepts
Foundations
Technology Stack
- Stack overview
- Python
- The idea underneath
- How it fits this project
- The src layout
- Type hints without enforcement
- Dataclasses versus Pydantic
- Enums for closed vocabularies
- Async: real concurrency in one place, none in another
- The standard library first
- Version floor: 3.10, and what depends on it
- Sharp edges and limits
- Learn more resources
- Packaging and distribution
- Why packaging and distribution
- The idea underneath
- How it fits this project
- The build backend: Hatchling
- Project metadata: classifiers, license, and requires-python
- The src layout and the packages option
- The tdw console script
- Optional dependency extras: postgres and dev
- Version handling
- The release workflow: build, check, and publish
- Sharp edges and limits
- Learn more resources
- SQLAlchemy
- pandas
- Faker
- Why Faker
- The idea underneath
- How it fits this project
- Two call sites
- Provider selection per column: names before types
- Primary keys: a counter or a uuid, and neither one is random
- Entity-specialized calls
- Seeding and reproducibility: two seed calls, not one
- Uniqueness: two different strategies, and one real gap
- Locale: none configured
- A custom provider that is defined but never registered
- Sharp edges and limits
- Learn more resources
- Jinja2
- Why Jinja2
- The idea underneath
- How it fits this project
- Inline
Template, not a configured environment - No autoescaping, and rightly so
- Whitespace control shapes the emitted Python
- Variables passed into the templates
- Conditional blocks that change the generated class shape
- Where the templates actually live
- Sharp edges and limits
- Learn more resources
- Rich
- Why Rich
- The idea underneath
- How it fits this project
- Console lifecycle: one instance per module
- Tables for structured output
- Panels for status and next steps
- Progress and Live: the two long-running commands
- Prompt and Confirm: the one interactive command
- Inline markup: how style gets into the text
- Sharp edges and limits
- Learn more resources
- FastAPI
- Why FastAPI
- The idea underneath
- How it fits this project
- App construction and lifespan
- Router organisation and versioned prefixes
- Request and response models
- Cross-cutting middleware
- Error handling: what a caller actually sees on failure
- Background tasks for the one real deferred job
- Testing the endpoints
- Sharp edges and limits
- Learn more resources
- Uvicorn
- Pydantic
- The idea underneath
- How it fits this project
- Model definitions and field types
- ConfigDict and the json_schema_extra example
- Aliasing around a name collision
- How FastAPI turns these models into the OpenAPI schema
- Validation error responses
- The boundary between API models and internal models
- Sharp edges and limits
- Learn more resources
- PyYAML
- aiofiles
- pytest
- Why pytest
- The idea underneath
- How it fits this project
- Fixtures and conftest
- Testing generated code by executing it
- asyncio_mode auto and testing async code
- Markers and selecting subsets
- Mocking, and mostly not needing it
- Integration tests against a real database
- Determinism and seeded tests
- The CI matrix
- Sharp edges and limits
- Learn more resources
Reference
About the documentation
Project information#
- Version:
1.0.0
- License:
Apache-2.0
- Repository: