Validate Config Files with JSON Schema

A settings model validates configuration when the application starts. That leaves a gap on both sides: an author editing a YAML file gets no feedback until they run something, and files the application never loads — another team’s overlay, a template, a deployment manifest fragment — are never checked at all. JSON Schema fills the gap by describing the file’s shape in a language editors and CI understand, and generating it from the model you already have keeps the two definitions from drifting. This page shows the setup, extending the YAML and JSON parsing strategies topic.

Problem 1: two definitions that drift apart

# ANTI-PATTERN: a hand-written schema beside a model that already describes the same thing
SCHEMA = {"type": "object", "properties": {"pool_size": {"type": "integer"}}}   # written in March

class Settings(BaseSettings):
    pool_size: int = 20
    pool_timeout: float = 5.0        # added in June; the schema never learned about it

The moment a schema is maintained by hand alongside a model, it starts falling behind. Fields are added to the model because that is where the code needs them; nobody remembers the schema until a file passes validation and then fails to load, which is the worst of both worlds — a green check followed by a startup error.

Generate instead. Pydantic emits a JSON Schema from the model, so the schema is always exactly current, and regenerating it is a build step rather than a chore. The generated file lives in the repository so editors and CI can read it, and a check that regeneration produces no diff catches the case where somebody edits it by hand.

One generated schema serving editors, CI and the runtime The settings model generates a JSON Schema used by editors for completion and by CI for file validation, while the model itself remains the authority when the application starts. settings model the single definition generated schema committed, never edited editor completion and inline errors CI check on every config file runtime: the model itself validates
Generate once, use in three places — and never hand-edit the middle box.

Problem 2: expecting a schema to catch semantic errors

# passes every schema: correct types, correct shape, wrong database
database:
  host: db-staging.internal      # in the production overlay
  port: 5432

A schema checks structure: types, required keys, enum membership, string patterns, numeric ranges. It cannot know that this host is the staging database and this file is production’s. Treating a green schema check as “the configuration is correct” is the mistake that follows adopting one, and it is worth naming explicitly because the check feels comprehensive.

Use the schema for the class of errors it genuinely catches — a misspelled key, a string where a number belongs, an enum value that does not exist, a required section omitted — which is a large share of the mistakes people make while editing YAML by hand. Then keep the checks that catch semantic errors where they belong: cross-field validators in the model, and a deploy-time assertion that the host matches the environment being deployed to.

Problem 3: schema validation instead of loading the config

# ANTI-PATTERN: a CI job that only validates the file's shape
- run: check-jsonschema --schemafile config.schema.json config/production.yaml

This proves the file is well-formed and proves nothing about whether the application can start with it. It does not exercise the environment’s variables, does not run cross-field validators, does not resolve interpolation, and does not verify that the required variables exist in the target environment.

Run both. Schema validation is fast and gives precise messages about the file, so it belongs in a pre-commit hook and as an early CI step. Constructing the settings model against the target environment is the real gate, described on the CI validation page. The schema check makes the model check fail less often; it does not replace it.

Secure implementation

# schema_tools.py — generate the schema from the model, and validate files against it
import json
import sys
from pathlib import Path
import yaml
from jsonschema import Draft202012Validator
from config import Settings

SCHEMA_PATH = Path("config.schema.json")


def generate() -> None:
    # 1. one definition: the model produces the schema, never the other way round
    schema = Settings.model_json_schema()
    schema["$schema"] = "https://json-schema.org/draft/2020-12/schema"
    schema["additionalProperties"] = False        # 2. mirror extra="forbid" so typos fail here too
    SCHEMA_PATH.write_text(json.dumps(schema, indent=2) + "\n")


def validate(paths: list[Path]) -> int:
    schema = json.loads(SCHEMA_PATH.read_text())
    validator = Draft202012Validator(schema)
    failures = 0
    for path in paths:
        document = yaml.safe_load(path.read_text()) or {}
        # 3. iter_errors reports every problem at once, not just the first
        for error in sorted(validator.iter_errors(document), key=lambda e: e.path):
            location = ".".join(str(p) for p in error.path) or "(root)"
            print(f"{path}:{location}: {error.message}")   # 4. path first: greppable, clickable
            failures += 1
    return failures


if __name__ == "__main__":
    if sys.argv[1] == "generate":
        generate()
    else:
        sys.exit(1 if validate([Path(p) for p in sys.argv[2:]]) else 0)
# .pre-commit-config.yaml — the fast check, before the commit lands
- repo: local
  hooks:
    - id: config-schema
      name: validate config files against the generated schema
      entry: python -m schema_tools validate
      language: system
      files: ^config/.*\.ya?ml$

Setting additionalProperties: False in the generated schema is what makes the check catch typos, which is the single most valuable thing it does. Without it, pool_sze: 20 is simply an unknown key the schema ignores, the file validates, and the application starts with the default while somebody wonders why their change had no effect. With it, the misspelling is an error naming the exact key and its location, caught before the commit.

Reporting every error rather than the first also matters more than it sounds. A file edited by hand often has several problems at once, and a validator that stops at the first turns fixing them into a slow loop of run-fix-run. Sorting by document path keeps the output in the same order as the file, so working down the list is a single pass.

What each layer of checking catches An editor catches shape errors while typing, a pre-commit hook catches them before the commit, CI schema validation covers every file, and constructing the settings model catches semantic and cross-field errors. editor shape, while typing pre-commit before it lands CI schema every file, enforced model construction semantics and sources each layer is cheaper than the next and catches a different class
The schema's job is to make the last box fail less often, not to replace it.

Editor integration is most of the value

The CI gate is the part that enforces, but the editor integration is the part that changes how it feels to edit a configuration file. Associating the generated schema with a file pattern gives completion for key names, inline documentation from field descriptions, and an error underline the moment a value has the wrong type — all before anything is run.

# .vscode/settings.json (or the equivalent in your editor)
"yaml.schemas": {
  "./config.schema.json": ["config/*.yaml"]
}

The practical effect is that people stop guessing at key names. A nested configuration of any size has a discoverability problem — what goes under telemetry? is it sample_rate or sampling_rate? — and completion answers it without a trip to the documentation or the model. Adding description to the model’s fields makes those descriptions appear as hover text, which is the cheapest documentation you will ever write because it lives next to the field it describes.

There is a modest ongoing cost: the generated schema must be regenerated when the model changes, and a stale committed schema gives wrong completions. Wire the generation into the same task that runs formatting or linting, and add a CI check asserting that regeneration produces no diff. That turns “somebody forgot” into a failing job with an obvious fix rather than a slow drift back to the hand-maintained situation the generation was meant to end.

A schema is also the cheapest way to give another team a contract. When a platform team owns a base configuration and product teams write overlays, the generated schema tells those teams what is allowed without either side reading the other’s code, and it fails their pull request rather than the platform team’s deploy. That is a meaningful shift in where the feedback lands: the person who made the change sees the error, in their own pipeline, in the terms of the file they edited. Publishing the schema alongside a short example overlay is usually enough documentation to make an overlay a self-service change rather than a request.

Gotchas and version-specific behaviour

  • Generate the schema from the model; a hand-maintained second definition always drifts.
  • Set additionalProperties: False so an unknown key is an error, matching extra="forbid" in the model.
  • A schema validates shape, not meaning — a valid URL pointing at the wrong host passes everything.
  • Report all errors rather than the first, sorted by document path, so a broken file is fixed in one pass.
  • YAML must be parsed before validation, since JSON Schema operates on the loaded data structure.
  • Add a CI check that regeneration produces no diff, or the committed schema slowly goes stale.
  • Field descriptions in the model become hover documentation in the editor, which is the cheapest place to document configuration because it lives beside the field.
  • A schema cannot express a rule spanning two fields as clearly as a validator can, so keep those in the model rather than contorting the schema.

Production parity checklist

  • The schema is generated from the settings model and committed, never hand-edited.
  • additionalProperties: False is set so misspelled keys fail rather than being ignored.
  • A pre-commit hook validates every configuration file against the schema.
  • CI validates the files and separately constructs the settings model against the target environment.
  • A CI check asserts that regenerating the schema produces no diff.
  • Editor configuration associates the schema with the config file pattern, so authors get completion and inline errors from the same definition CI enforces.
Errors a schema catches and errors it cannot A schema catches misspelled keys, wrong types, invalid enum values and missing sections, but cannot catch a value that is well-formed and points at the wrong resource. the schema catches misspelled or unknown keys a string where a number belongs an enum value that does not exist most hand-editing mistakes it cannot catch the staging host in production a valid key for the wrong account a rule spanning two fields those need validators and deploy checks
A green schema check means well-formed, not correct — the distinction is worth stating out loud.

Frequently asked questions

Why use JSON Schema when pydantic already validates?

Because the schema works outside Python — in an editor while somebody types, in a pre-commit hook before a commit lands, and against files your application never loads, such as another team’s overlay or a template. Inside the process, pydantic remains the authority and nothing changes. The schema’s job is to move the same feedback earlier and wider, so a misspelled key is a red underline in an editor rather than a startup error in a container half an hour later.

Can I generate the schema from my pydantic model?

Yes, and you should. Generating it keeps one source of truth and makes drift impossible; hand-writing a second schema means two definitions that will eventually disagree, and the disagreement surfaces as a file that validates and then fails to load. Commit the generated file so editors and CI can read it, and add a check that regeneration produces no diff so nobody quietly edits it by hand and reintroduces the problem.

Does a schema catch a wrong value?

Only if wrongness is expressible structurally — a bad enum, an out-of-range number, a malformed pattern. A syntactically perfect URL pointing at the wrong host passes every schema ever written, as does a valid API key belonging to the wrong account. Those need cross-field validators in the model and a deploy-time assertion comparing the configuration against the environment being deployed to, which is a different kind of check entirely.

How do I get editor completion for a YAML config?

Publish the generated schema in the repository and associate it with the file pattern in your editor’s YAML settings. Completion and inline errors then come from the same definition CI enforces, so what the editor accepts is what the pipeline accepts. Adding description to fields in the model turns those descriptions into hover text, which is the cheapest configuration documentation available because it lives beside the field and cannot go stale independently.

Should schema validation replace loading the config in CI?

No. Schema validation checks shape; constructing the settings model checks everything else, including cross-field rules, interpolated variables, and the sources actually available in that environment. Run both, in that order — the schema check is fast and gives precise, file-relative messages, so it catches the easy problems cheaply, and the model construction remains the gate that decides whether a deploy proceeds.

Key takeaways

A generated JSON Schema extends your existing validation to places pydantic cannot reach: an author’s editor, a pre-commit hook, and configuration files the application never loads. Generating it from the settings model rather than writing it by hand is what keeps the two definitions identical, and setting additionalProperties: False is what makes it catch the misspelled key that would otherwise be silently ignored.

Be precise about what it proves. A schema validates structure — types, required keys, enums, ranges — which covers most hand-editing mistakes and none of the semantic ones. The staging host in the production file passes every check the schema can express, so keep cross-field validators in the model and an environment assertion at deploy time. Run the schema check first because it is fast and specific, and keep constructing the settings model as the gate that actually decides. See the YAML and JSON parsing strategies topic for the parsing choices underneath.

Back to YAML & JSON Parsing Strategies