Test Data Principles

The design of Test Data Workbench rests on a small number of principles about what makes generated test data useful. This chapter states them directly and connects each to how the tool behaves. It is deliberately scoped to ideas the tool actually implements; it does not survey techniques the tool does not use.

Referential integrity is the hard part

Generating a single realistic value is easy; Faker does it in one call. The difficulty in relational test data is between the rows: an order must point at a user that exists, a product must point at a category that exists. Data that violates these references is worse than useless, because it breaks the very foreign-key constraints your system relies on.

Workbench treats referential integrity as a structural property enforced by generation order. Foreign keys discovered during analysis define a dependency graph, and tables are generated in topological order so that every referenced row exists before the rows that reference it. Parents before children, always.

Dependency ordering needs cycle handling

A topological sort assumes the dependency graph is acyclic. Real schemas are not always so obliging: a table can reference itself (a category with a parent category), and two tables can reference each other. A sort that does not account for cycles either loops forever or crashes.

The ordering in Workbench detects cycles and resolves them to a defined order rather than failing. This is treated as a first-class correctness concern, not an edge case, because a schema-adaptive tool cannot assume the schemas it meets are well-behaved.

Determinism is a feature, not a default

A system built on random generation is nondeterministic unless determinism is engineered in. For test data this matters more than it first appears: a fixture that changes between runs turns a reproducible test into a flaky one.

Workbench makes determinism opt-in and total. With a seed, generation is reproducible to the byte, a property covered in Design Principles and enforced by tests. The important subtlety is that determinism is fragile: a single embedded timestamp or a single set-iteration ordering leak defeats it, which is why those leaks are closed deliberately rather than assumed absent.

Uniqueness under generation

Columns with unique constraints (an email, a username) cannot simply be generated independently, because random generation will eventually collide. Specialized generators track values they have produced and regenerate on collision, so that a unique column stays unique across a batch. This is a small mechanism with outsized importance: a duplicate in a unique column fails the insert and invalidates the whole dataset.

Realism versus simplicity

There is a genuine tension between data that is realistic (catches real edge cases, supports integration and load testing) and data that is simple (easy to read, fast, predictable). Neither end of the spectrum is correct in the abstract; the right point depends on what the data is for.

Workbench’s answer is to produce realistic-by-default output while keeping it fully editable, so you can dial toward simplicity where you want it. Because the output is generator source code, tuning the realism of a column is a code edit, not a limitation of the tool.

Partial and present beats perfect and absent

The principle that ties the others together, and the one that most distinguishes this tool: for test data, coverage matters more than perfection. A dataset that populates every table adequately is more useful than one that populates some tables beautifully and leaves the rest empty because the tool did not understand them. The The Fallback Ladder is the direct expression of this principle in the system’s behavior.