Python¶
Python is the implementation language for the whole system: the analyzer, the generator factory, the CLI, and the optional API. There is no compiled extension and no second language anywhere in the stack. This chapter covers the language features the codebase actually leans on, not a general Python pitch.
Version floor: 3.10+¶
pyproject.toml declares requires-python = ">=3.10", and the CI matrix
in Testing and Quality runs 3.10, 3.11, and 3.12. Nothing in the
codebase currently requires a newer minimum; 3.10 was chosen as a floor that
is old enough to be widely available and new enough for modern type-hint
syntax.
Type hints throughout¶
Every public function and dataclass field in core and adaptation is
type-hinted. This is not decorative: the type hints are load-bearing for two
things elsewhere in the stack. Pydantic models in Pydantic build their
validation directly from annotated fields, and sphinx.ext.autodoc (see
API Reference) renders these same annotations into the API
reference without any separate documentation of parameter types.
Typing in this codebase favors the explicit typing module spellings
(Optional[int], Dict[str, List[Any]]) over the newer | union
syntax, for consistency with the 3.10 floor rather than because the newer
syntax is unavailable.
Dataclasses as the internal model layer¶
src/test_data_workbench/core/models.py defines the system’s internal
vocabulary, schema, entity types, relationships, business rules, using
@dataclass, not Pydantic:
@dataclass
class Column:
"""Database column metadata."""
name: str
data_type: str
nullable: bool = True
default: Optional[str] = None
max_length: Optional[int] = None
constraints: List[ConstraintType] = field(default_factory=list)
sample_values: List[Any] = field(default_factory=list)
@property
def is_primary_key(self) -> bool:
return ConstraintType.PRIMARY_KEY in self.constraints
This is a deliberate split, not an inconsistency. Pydantic earns its
validation and serialization overhead at the API boundary, where untrusted
input arrives over HTTP (see Pydantic). Internally, between the
analyzer, the factory, and the CLI, the data never crosses a trust boundary;
a plain dataclass with field(default_factory=...) for mutable
defaults is the lighter-weight, equally type-checkable choice. Table,
SchemaInfo, AdaptationResult, and every other internal record type
follow the same pattern.
Enum is used the same way, for closed vocabularies that should not be
free-text strings: EntityType and ConstraintType in core/models.py.
Async where fan-out actually pays for itself¶
asyncio appears in exactly one place with real concurrency:
SchemaAnalyzer.analyze_production_schema in
src/test_data_workbench/adaptation/schema_analyzer.py builds one
_analyze_table coroutine per discovered table and runs them together:
tasks = [
self._analyze_table(engine, inspector, table_name, metadata_only)
for table_name in table_names[:20] # Limit for rapid analysis
]
tables = await asyncio.gather(*tasks)
Each _analyze_table call does its own I/O (sample-value queries, row-count
estimates) against the database, so fanning them out with asyncio.gather
overlaps that I/O instead of doing it table by table. It is concurrency, not
parallelism; the underlying SQLAlchemy calls are still synchronous DB-API
calls running on one thread, but overlapping their wait time is exactly the
case asyncio is for. The 20-table cap alongside the fan-out is itself a
deliberate bound: unlimited concurrent connections to an unfamiliar
production database would be an odd thing for a test-data tool to risk.
The CLI is the synchronous/async boundary: cli/analyze.py and
cli/deploy.py both call into this async code with a single
asyncio.run(...) at the top, so the rest of the CLI stays ordinary
synchronous Python.
The standard library first¶
The CLI entry point, src/test_data_workbench/cli/main.py, is built on
argparse from the standard library rather than a third-party CLI
framework. Subcommands are registered lazily, imported only inside
build_parser(), so that tdw --version stays fast and free of the
import cost of every subcommand’s dependencies:
def build_parser() -> argparse.ArgumentParser:
# Imported lazily so that `tdw --version` stays fast and side-effect free.
from . import analyze, deploy, demo, team, validate
...
This mirrors the project’s broader dependency discipline described in
Technology Stack: reach for the standard library before adding a dependency, and
a CLI parser is squarely inside what argparse already does well.