PyYAML#
Test Data Workbench generates several files a team member is expected to
open, read, and hand edit: a project’s config.yaml, its per-scenario
entity counts, a contribution guide describing what a beginner versus an
advanced contributor is safe to touch, and an environment variable template.
None of that is meant to be consumed only by another program; it is meant to
sit on a screen, take a stray comment explaining why a count was set the way
it was, and survive being opened in whatever editor a contributor already
has open. PyYAML is the library pyproject.toml pins (pyyaml>=6.0.1)
to produce that legible text, and, in exactly two places, to read a
contributor’s edited version of it back in.
The same project also has a machine facing serialization path: tdw
analyze --format json goes through the standard library’s json module,
not PyYAML, at all. Which module a given command reaches for tracks one
question: is the file meant to be typed and read by a person, or only ever
produced and consumed by a program. The “One command, two serializers”
section below shows the same CLI command choosing between the two.
The idea underneath#
YAML was designed as a human friendly superset of JSON. PEP 518, the Python
packaging proposal that chose a file format for pyproject.toml, puts it
this way: “The YAML format was designed to be a superset of JSON while being
easier to work with by hand.” Every JSON document is valid YAML; YAML adds
comments, unquoted scalars, block indentation instead of braces, and several
ways to write the same value.
That extra flexibility is also where the ambiguity comes from. A YAML
scalar with no quotes around it is not automatically a string: the parser
has to guess a type from its shape, and the guessing rules produce
surprises a JSON reader never has to think about. The best known case has a
name, the Norway problem: a bare NO, written as a country code, resolves
to the boolean False rather than the two letter string, because YAML’s
implicit typing rules treat no (in several cases) as a boolean literal.
Numbers carry the same kind of surprise. PyYAML’s own resolver, the
component that decides which type an unquoted scalar gets, registers
yes|Yes|YES|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF
as an implicit boolean, a leading zero followed by octal digits
(0[0-7_]+) as an implicit octal integer, and a run of digits broken up
by colons ([1-9][0-9_]*(?::[0-5]?[0-9])+) as an implicit sexagesimal
(base 60) integer. None of that is a PyYAML bug; it is what the YAML 1.1
Core Schema, the rule set PyYAML’s own project description says it
implements (“a complete YAML 1.1 parser”), specifies. YAML 1.2, published in
2009, tightened these rules specifically to remove the NO-becomes-false
surprise and to make YAML strictly JSON compatible, but a project pinning
PyYAML, as this one does, is bound by what PyYAML itself resolves, not by
what the newer specification recommends.
There is a second, sharper kind of ambiguity underneath the readability
tradeoff, and it is the one this page’s Sharp edges section is built around.
A format stays “just data” only if whatever reads it refuses to build
anything except data. JSON’s grammar has no way to say “construct this
class” or “call this function”; a JSON decoder can only ever hand back
dicts, lists, strings, numbers, booleans, and null, which is why the
standard library’s json module has one loading function, not a safe one
and an unsafe one. YAML’s grammar is not limited that way. PyYAML’s own
documentation states the consequence directly: “It is not safe to call
yaml.load with any data received from an untrusted source! yaml.load is as
powerful as pickle.load and so may call any Python function.” The library’s
restricted counterpart is explicit about the tradeoff it makes instead: the
safe_load docstring reads, “Resolve only basic YAML tags. This is known
to be safe for untrusted input.” A deserializer that can construct arbitrary
objects turns a configuration file into something closer to a program than
to data; safe_load is what puts that boundary back.
PEP 518 names both of these problems, size and safety, as the reasons
pyproject.toml uses TOML rather than YAML. On size: “the specification is
large: 86 pages if printed on letter-sized paper,” which leaves room for one
parser to accept a document another rejects. On safety, in the PEP’s own
words: “YAML itself is not safe by default.” And: “The specification allows
for the arbitrary execution of code which is best avoided when dealing with
configuration data.” The PEP goes on to note that PyYAML does offer a safe
loading function as a way around that default, which is exactly the function
this project uses everywhere it reads YAML back, as the next section shows.
TOML’s design favors being small and unambiguous, a good fit for build
metadata a tool reads automatically with no person in the loop; YAML’s
design favors being comfortable for a person to write and comment, a good
fit for the scenario and team files this project hands to contributors. This
project uses each format where its own tradeoff actually pays off.
How it fits this project#
pyyaml is imported in exactly five files, and the traffic runs in one
direction almost everywhere. src/test_data_workbench/adaptation/config_builder.py
(three calls), src/test_data_workbench/cli/analyze.py (one call, inside
save_analysis_results), and src/test_data_workbench/cli/team.py (one
call, inside create_team_config) only ever call yaml.dump: they turn
this project’s own in memory data into YAML text for a human to read, and
never parse that text back themselves. src/test_data_workbench/cli/validate.py
(inside ContributionValidator._validate_yaml_contribution) and
src/test_data_workbench/core/validators.py (inside
TeamContributionValidator._validate_yaml_contribution) are the only two
places this project ever parses a YAML file it did not just generate itself,
and both call yaml.safe_load. PyYAML in this codebase is entirely a
human boundary tool: it never carries data between two parts of the running
program, only between this project’s code and a person reading or writing a
file by hand.
Generating configuration a person will edit#
ConfigurationBuilder.create_team_config_file builds the main
config.yaml a generated project ships with. The dictionary it builds is
ordered deliberately, version first, then project metadata, then scenarios,
generators, and a team block of safety flags, and the dump call is
written to preserve that order rather than alphabetize it:
config = {
'version': '4.0',
'project': {...},
'scenarios': {},
'generators': {},
'team': {
'safe_mode': True,
'fallback_enabled': True,
'validation_required': True
}
}
...
return yaml.dump(config, default_flow_style=False, sort_keys=False)
default_flow_style=False is what makes the output block style, one key
per line with indentation, rather than the inline {a: 1, b: 2} flow
style YAML also allows; block style is what a person actually wants to
scroll and edit. sort_keys=False keeps the dict’s own insertion order
in the emitted file instead of PyYAML’s default of alphabetizing keys.
create_contribution_guide (the same class, building
contribution_guide.yaml) makes the identical two choices. create_env_template,
the third yaml.dump call in this file, does not:
return yaml.dump(template, default_flow_style=False)
With no sort_keys argument, PyYAML’s default (sort_keys=True)
applies, so the environment block’s database, generation, and
team keys come out alphabetized rather than in the order the dict
literal defines them, a small inconsistency with the other two methods in
the same file. The values that template writes are shell style placeholders,
for example '${DB_HOST:localhost}': to PyYAML that is nothing but a
string. Nothing in this project, and nothing in PyYAML itself, expands
${VAR:default} syntax; whatever process eventually reads the rendered
.env.template file is responsible for that substitution on its own.
cli/team.py’s create_team_config follows the same block style,
ordered dump pattern for team_config.yaml, including a value computed
from the registered team, not just copied from a template:
config = {
"team": team_info,
...
"safety": {
"validation_required": True,
"auto_backup": True,
"feature_flags": {
"experimental": False,
"advanced_features": any(m["skill"] == "advanced" for m in team_info["members"]),
},
},
...
}
...
yaml.dump(config, f, default_flow_style=False)
advanced_features is a genuine Python bool by the time it reaches
yaml.dump, so it always renders as the literal true/false PyYAML
uses for booleans, not as a word from the yes/no/on/off
family the Norway problem section above describes; that ambiguity only
becomes a live risk for values a person types by hand later, not for
anything this project’s own dump calls emit.
Reading a contribution back: the safe_load call sites#
The round trip this project actually implements is: ConfigurationBuilder
dumps a scenario or config file for a person to look at; a team member edits
it, most often the entity counts inside a scenario’s entities block; and
tdw validate reads the edited file back with yaml.safe_load before
treating it as a valid contribution. The two implementations of that read
are close to identical. cli/validate.py:
import yaml
with open(file_path, "r", encoding="utf-8") as f:
try:
data = yaml.safe_load(f)
except yaml.YAMLError as e:
return ValidationResult(
valid=False,
errors=[f"YAML parsing error: {e}"],
...
)
core/validators.py:
import yaml
with open(file_path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
Both then require data to be a dict, warn if name is missing, and
require an entities key whose value is itself a dict of non negative
integer counts. Past that point the two diverge in ways that matter to a
contributor. cli/validate.py sums every entity’s count and warns above a
combined total: if total_records > 1_000_000: ... elif total_records == 0:
..., so a scenario is flagged only if its grand total is either far too
large or exactly zero. core/validators.py instead checks each entity
individually: elif count > 100000: warnings.append(f"Entity '{entity}' has
very large count ({count})..."), with no check at all for an empty
entities block. A scenario with ten entities at 50,000 rows each, half a
million records total, passes cli/validate.py’s combined threshold
without comment and trips core/validators.py’s per-entity one five
times over. Whether that split is a deliberate second, stricter gate or
duplicated logic that has quietly drifted is not stated anywhere in the
code; what is certain is that the same file can be judged fine by one
validator’s numbers and flagged by the other’s, depending on which one is
asked.
Neither implementation checks the parsed structure against a real,
discovered schema. Both accept any dict shaped correctly, regardless of
whether its entity names, users, orders, or a typo of either,
correspond to a table SQLAlchemy reflection actually found.
TeamContributionValidator even accepts an optional schema_info in
its constructor and builds a SchemaValidator from it when one is given,
but _validate_yaml_contribution never touches self.schema_validator;
that cross check only runs from _validate_python_contribution, against
already generated data records, never against a submitted YAML file. A
scenario that references a table name nothing in the schema matches passes
tdw validate cleanly and would only fail later, somewhere further from
the point a contributor could have been told what was actually wrong.
One command, two serializers: analyze and its output formats#
tdw analyze (cli/analyze.py) is where this project’s JSON path and
YAML path sit closest together, one file apart. display_json_format
handles --format json with the standard library, not PyYAML:
if output_file:
with open(output_file, "w") as f:
json.dump(result, f, indent=2)
else:
console.print(json.dumps(result, indent=2))
save_analysis_results handles the other case, the default table
format combined with --output, and reaches for yaml instead:
import yaml
result = {
"analysis_results": {...},
"entity_mapping": {
table.name: table.entity_type.value
for table in schema_info.tables
},
"suggested_generators": sorted(set(...)),
}
with open(output_file, "w") as f:
yaml.dump(result, f, default_flow_style=False)
The same command, pointed at the same database, produces JSON when
--format json says the output is for a script or another tool to
consume, and YAML when the default, human oriented table format is
combined with --output to keep a copy of what was found. The choice of
serializer tracks the audience for the file, exactly the distinction “The
idea underneath” draws between the two formats.
Learn more resources#
Official documentation
PyYAML Documentation: the
load/safe_load/dump/safe_dumpAPI reference, including the project’s own warning against callingyaml.loadon untrusted input, quoted above.PyYAML yaml.load(input) Deprecation: the project’s own account of why the
Loaderargument toyaml.loadstopped being optional, and what changed for callers across releases.PyYAML on PyPI: the package’s own description, including the line stating it implements “a complete YAML 1.1 parser,” the fact this page’s Norway problem discussion depends on, for the
pyyaml>=6.0.1range this project pins.YAML Ain’t Markup Language (YAML) Version 1.2.2: the current specification, including the Core Schema and the tightened implicit typing rules that PyYAML’s older, 1.1 based resolver does not implement.
PEP 518, Specifying Minimum Build System Requirements for Python Projects: the Python packaging community’s own record of why
pyproject.tomluses TOML instead of YAML, quoted at length in “The idea underneath.”
Tutorials and blogs
Fully loaded: testing vulnerable PyYAML versions (Semgrep): which PyYAML releases and loaders are actually exploitable in practice, useful background for why this page’s clean
safe_loadaudit result is worth checking rather than assuming.YAML: The Norway Problem (Bram.us): the canonical write-up of the unquoted-
NO-becomes-falsefailure, with the real-world incident this page’s type-coercion warning draws on.
Videos
What about the YAML “Norway” Problem? (RWXROB): a short, live demonstration of the same implicit-boolean surprise against a YAML parser, the concrete version of the resolver behavior described above.