CI/CD Config Validation

A configuration error should fail a pipeline, not a production pod. The other pages in this section repeatedly lean on “gate it in CI” — this is the dedicated page for that gate. It instantiates the settings model in the pipeline so a missing key, a malformed URL, or a committed secret stops the build. It sits within the core configuration patterns as the enforcement layer over everything else.

The insight that makes this cheap is that you already own a complete specification of valid configuration: the settings model. Every required field, every type, every constraint and validator is a machine-checkable assertion about what a correctly configured environment looks like. Constructing that model against a given environment’s variables is therefore a full configuration test, and it costs one short script and a few seconds of pipeline time. Nothing else in the deployment pipeline gives that much verification for that little effort.

Where configuration errors are caught Without a gate, a configuration error travels from commit through build and deploy to production; with a gate, it stops at the validation stage. the earlier the gate, the cheaper the failure commitpre-commit hook CI gatescan + validate pre-deploytarget env vars productiononly valid config arrives seconds to fix minutes to fix an incident The same check runs at every stage — only the environment it validates against changes.
One validation script, run at three points, with the cost of a failure rising sharply from left to right.

Secure implementation

# ci/validate_config.py — run as a standalone CI step
import sys
from pydantic import ValidationError
from config.settings import Settings   # your one BaseSettings model

def main() -> int:
    try:
        settings = Settings()           # extra="forbid" catches typo'd vars
    except ValidationError as exc:
        print("Config validation FAILED:", file=sys.stderr)
        print(exc, file=sys.stderr)     # field-level errors, no secret values
        return 1
    print("Config OK:", ", ".join(settings.model_dump(exclude={"api_key"}).keys()))
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
# .github/workflows/config.yml
jobs:
  validate-config:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements.txt
      - run: gitleaks detect --no-banner          # block committed secrets
      - run: python -m ci.validate_config         # fail build on bad config
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          API_KEY: ${{ secrets.API_KEY }}

The job constructs the model with extra="forbid", so a stray or misspelled variable fails the build. Secrets come from masked GitHub secrets and are never printed. The success line prints field names only, and even that goes through exclude, so a SecretStr slip cannot leak — the belt-and-braces matters because CI logs are frequently more widely readable than the secret store the values came from.

Printing the ValidationError directly is safe and deliberately chosen. Pydantic’s error output names the field and the failure reason but does not echo the offending value for most error types, which is exactly the level of detail an operator needs: database_url: Field required tells them what to add. If your model has custom validators, check that their messages follow the same rule — a validator that raises ValueError(f"invalid URL: {v}") turns a helpful message into a credential disclosure the moment the malformed value is a connection string with a password in it.

What the validation step reports On failure the step prints field names and reasons to stderr and exits non-zero; on success it prints only the set of resolved key names. failure database_url: Field required port: input should be a valid int exit code 1 → build stops names and reasons, never values success Config OK: database_url, port, redis_url, log_level exit code 0 → pipeline continues api_key excluded from the dump
Both branches print names only — the log is useful for diagnosis and useless to an attacker.

Configuration reference

Step Tool Fails build when Security implication
Secret scan gitleaks A credential is committed Stops leaks reaching history
Config validate Settings() Missing/malformed key Catches drift pre-deploy
extra="forbid" pydantic Unknown variable present Surfaces typos in review
Masked vars CI platform — Keeps secrets out of logs
PYTHONWARNINGS=error Python Any warning Catches deprecations early

extra="forbid" is the row that generates the most debate, because it fails builds for variables that are merely unused rather than wrong. That is the point. The overwhelmingly common cause of an unexpected extra variable is a typo — DATABSE_URL set in the environment while the model waits for DATABASE_URL — and without forbid that typo produces a “field required” error for a key the operator can plainly see is set, which is one of the more maddening debugging experiences in this area. With forbid, the error names both halves of the problem at once.

The one real friction is shared environments where other tooling’s variables land in the same namespace. Setting env_prefix on the model confines it to keys that belong to your service, after which forbid is unambiguous: anything with your prefix that the model does not declare is a mistake, and anything without your prefix is somebody else’s business.

PYTHONWARNINGS=error is a smaller lever with an outsized effect on maintenance. Deprecation warnings from pydantic, from a database driver, or from the standard library are invisible in normal CI output and accumulate silently until a major version upgrade turns them all into errors at once. Promoting them to failures means each one is fixed by whoever introduced it, on the commit that introduced it.

What each gate catches The secret scanner catches committed credentials, model construction catches missing and malformed keys, extra=forbid catches typos and stale variables, and warnings-as-errors catches deprecations. four checks, four distinct failure classes gitleaks a credential reached a commit — bypasses --no-verify Settings() required key missing, or value fails its type or validator extra="forbid" typo'd or stale variable set in the environment warnings as errors deprecation introduced today, not discovered at upgrade
None of the four overlaps with another — dropping any one leaves a class of misconfiguration unchecked.

Validating against the right environment

The subtlety that decides whether this gate is worth anything is which variables it validates against. A pipeline that constructs the model using a set of dummy CI values proves that the model can be constructed — nothing more. It will happily pass while production is missing a key, because production’s environment was never involved.

The fix is to run the same script once per target environment, with that environment’s real variable set, immediately before deploying to it. In GitHub Actions that is a job per environment using the environment’s own secrets; in GitLab it is a job per stage with the matching scoped variables. The script is identical each time; only the injected values differ, which is precisely the property that makes the check meaningful.

# .github/workflows/deploy.yml — validate against the target before deploying to it
jobs:
  validate-staging:
    environment: staging          # pulls staging's secrets, not the repo defaults
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python -m ci.validate_config
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          API_KEY: ${{ secrets.API_KEY }}
  deploy-staging:
    needs: validate-staging       # deployment cannot start unless config is valid
    runs-on: ubuntu-latest
    steps: [...]

The needs: edge is doing the real work. Without it the validation job is advisory — it goes red while the deploy proceeds beside it. With it, an invalid configuration makes the deployment unreachable, which converts the check from information into enforcement. Every pipeline that claims to gate on configuration should be readable as a dependency graph where nothing deploys without a passing validation upstream of it.

One thing this cannot catch is a variable that exists and is well-formed but points at the wrong place — a staging database URL set in the production environment. Validation proves the shape, not the intent. A cheap partial defence is a validator that asserts environment-appropriate patterns, such as refusing a hostname containing staging when APP_ENV is production; it is crude, but it catches the copy-paste mistake that causes most such incidents.

A more general version of the same idea is to validate relationships rather than individual fields. A model validator can assert that the database host and the cache host belong to the same region, that a callback URL’s domain matches the configured public hostname, or that debug mode and a production log destination are never both set. Each of those is a statement about a combination that no single field can express, and each one is checkable in the same pipeline job at no extra cost. They are also the assertions most likely to catch a genuinely dangerous misconfiguration, because a value that is individually plausible and jointly wrong is exactly what slips past review.

Per-environment validation gating each deployment Each environment has its own validation job using that environment's secrets, and the corresponding deploy job depends on it so an invalid configuration blocks the deployment. same script, each environment's own variables validate: stagingstaging secrets deploy: stagingneeds: validate validate: productionproduction secrets deploy: productionneeds: validate without needs: the job goes red while the deploy proceeds anyway
The dependency edge is what turns a red check into a blocked deployment.

Detecting drift between environments

Validation answers “is this environment configured?” It does not answer “are these environments configured the same way?”, and the gap between those two questions is where a large share of promotion failures live. Staging passes, production passes, and the deploy still misbehaves because production is running with a default that staging overrides — both valid, neither identical.

The check that closes the gap is an inventory rather than a validation: for each environment, record which source supplied each key, then diff the inventories. A key resolved from the environment in staging but from a field default in production is drift, and it is drift that no amount of per-environment validation will surface.

# ci/config_inventory.py — emit a source map, never values
import json, os, sys
from config.settings import Settings

def inventory() -> dict[str, str]:
    fields = Settings.model_fields
    return {
        name: ("env" if name.upper() in os.environ else "default")
        for name in fields
    }

if __name__ == "__main__":
    json.dump(inventory(), sys.stdout, indent=2, sort_keys=True)

Run it in each environment’s validation job, publish the JSON as a build artifact, and add a final job that fetches both and diffs them. The diff is small, readable, and reviewable — a handful of lines saying which keys differ in origin — and it turns an entire class of “worked in staging” incidents into a pipeline comment. Because it prints origins rather than values, the artifact is safe to keep and safe to attach to a pull request.

The same artifact has a second use during incidents. When production behaves unexpectedly, the inventory from the last successful deploy tells you which keys were coming from where at that moment, which is often faster than reconstructing the state from deployment manifests and secret store history.

Diffing the source map between environments Each environment publishes a map of key to source, and a final job diffs them so a key resolved from the environment in staging but from a default in production is flagged. staging database_url: env redis_url: env log_level: env production database_url: env redis_url: default log_level: env drift detected redis_url falls back to a default in production
Both environments validate successfully — only the source diff reveals that one is running on a fallback.

Testing the model, not just running it

The validation job proves that a given environment satisfies the model. A small test suite proves the reverse: that the model actually rejects what it should. Both are needed, because a model whose fields are all optional with defaults passes validation against an empty environment and gates nothing at all.

# tests/test_settings.py
import pytest
from pydantic import ValidationError
from config.settings import Settings

BASE = {"DATABASE_URL": "postgres://u:p@db/app", "API_KEY": "k"}

def test_required_keys_are_required(monkeypatch):
    monkeypatch.delenv("DATABASE_URL", raising=False)
    for k, v in {"API_KEY": "k"}.items():
        monkeypatch.setenv(k, v)
    with pytest.raises(ValidationError) as exc:
        Settings()
    assert "database_url" in str(exc.value)

def test_unknown_variable_is_rejected(monkeypatch):
    for k, v in {**BASE, "DATABSE_URL": "typo"}.items():
        monkeypatch.setenv(k, v)
    with pytest.raises(ValidationError):
        Settings()          # extra="forbid" catches the typo

def test_secret_is_not_in_repr(monkeypatch):
    for k, v in BASE.items():
        monkeypatch.setenv(k, v)
    assert "k" not in repr(Settings())      # SecretStr masks it

Three tests, three properties that the pipeline depends on and that would otherwise be assumed. The first asserts that a required field is genuinely required, which fails the day someone adds a convenience default to unblock a local run. The second asserts that extra="forbid" is in force, which fails the day someone relaxes it to silence an unrelated error. The third asserts that a secret field does not leak through repr, which fails the day a SecretStr becomes a plain str during a refactor.

Each of those three regressions is the kind that passes review easily — they all look like small simplifications — and each one silently disables part of the gate. Pinning them in tests means the pipeline’s guarantees are themselves tested, rather than being properties everyone believes are still true.

Tests that keep the gate from being silently disabled Three tests assert that required fields stay required, that unknown variables are rejected, and that secret fields stay masked in repr. each test guards one gate property required stays required breaks when someone adds a convenience default missing key still fails forbid stays on breaks when someone relaxes it to ignore typos still surface secrets stay masked breaks when SecretStr becomes a plain str logs stay clean
All three regressions look like harmless simplifications in review — which is exactly why they need tests.

Step-by-step deployment parity

  1. Local dev — run python -m ci.validate_config and the pre-commit secret scanner before pushing.
  2. CI — the pipeline runs the same validation plus gitleaks; a failure blocks merge.
  3. Staging — validate against staging’s injected variables before promotion.
  4. Production — a pre-deploy job validates against production variables; deployment is blocked on failure.

Notice that step one is not decoration. Making the same script runnable locally means a developer adding a required field can confirm their own environment satisfies it before pushing, and it means the CI failure they might otherwise hit is reproducible on their machine rather than mysterious. A validation gate that only exists inside the pipeline trains people to debug by pushing commits, which is slow and noisy.

Steps three and four are the ones most often collapsed into one. Validating once, against staging, and then promoting the artifact to production on the assumption that production is configured the same way, reintroduces the whole problem this page exists to solve — production’s variables were never checked. Each environment gets its own validation immediately before its own deployment, even when the artifact being deployed is byte-identical to the one already running in staging, because it is the configuration and not the artifact that differs.

The same validation script at four points The identical script runs locally before pushing, in CI on every merge, before promotion to staging, and before deployment to production. 1 · local before pushing fastest feedback 2 · CI every merge cannot be skipped 3 · staging staging's own vars blocks promotion 4 · production production's vars blocks deployment One script, four invocations — the environment changes, the assertions never do.
Because the script is the same everywhere, a local pass means something about what CI will do.

Keeping the gate fast and trustworthy

A gate that people work around is worse than no gate, because it creates the appearance of enforcement without the substance. Two failure modes cause that, and both are about trust rather than correctness.

The first is slowness. A validation job that installs the full dependency tree to construct one settings object adds minutes to every pipeline run, and the pressure to skip it grows with the wait. Most of that time is installation, not validation, so the fixes are ordinary: cache the dependency install keyed on the lockfile, install only what the settings module actually imports if the model can be isolated, and run the validation job in parallel with the test job rather than after it. A gate that finishes in twenty seconds is one that nobody ever argues about or asks to have made optional.

The second is flakiness. If the validation job fails intermittently — a network call in a validator, a secret store rate limit, a dependency resolved from the network at run time — people learn to re-run it rather than read it, and a genuine failure gets re-run too. Configuration validation should be hermetic: no network, no external service, nothing but the environment and the model. A validator that checks reachability of a database is a useful thing to have, but it belongs in a separate connectivity job, precisely so its flakiness cannot erode trust in the deterministic check.

# .github/workflows/config.yml — fast, hermetic, and cached
jobs:
  validate-config:
    runs-on: ubuntu-latest
    timeout-minutes: 5              # a hung gate is a broken gate
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip                # install is the slow part, not validation
      - run: pip install -r requirements.txt
      - run: python -m ci.validate_config
        env:
          PYTHONWARNINGS: error
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          API_KEY: ${{ secrets.API_KEY }}

The timeout-minutes entry is worth setting deliberately. Without it, a job that hangs — waiting on a prompt, retrying a network call — occupies a runner until the platform’s default timeout expires, which on some platforms is hours. A short timeout converts a hang into a fast, obvious failure, and for a job that should take seconds there is no downside.

What makes a gate trusted or worked around A fast, hermetic, deterministic job is read and acted on, while a slow or flaky job trains people to re-run or skip it. worked around minutes of dependency install network calls inside validators intermittent red people re-run instead of reading trusted cached install, seconds to run hermetic: env + model only red means red, every time a failure gets read and fixed
Determinism is a security property here: a check people re-run reflexively is a check that no longer blocks anything.

Security boundaries & operational guardrails

  • Secret scanning runs in both pre-commit and CI; CI is the gate that cannot be skipped.
  • Validation uses extra="forbid" so unknown keys fail instead of slipping through.
  • CI variables are masked; validation output reports key names only, never values.
  • Secrets are wrapped in SecretStr so an accidental print is masked.
  • The validation step runs with PYTHONWARNINGS=error to catch deprecations.
  • Pipeline definitions are reviewed like code — a change to a needs: edge can silently remove a gate.

Masking deserves one caution: it is a substring match on the log stream, not a guarantee. A secret that gets transformed before printing — base64-encoded, URL-escaped, split across lines, or interpolated into a JSON body — no longer matches the masked string and appears in the log in full. That is why “never echo configuration” is the primary rule and masking is the safety net, rather than the other way round. The same logic applies to SecretStr: it protects repr() and str(), but get_secret_value() returns a plain string that behaves like any other.

The last item is the one teams discover the hard way. Pipeline YAML is executable infrastructure, and a refactor that reorganises jobs can drop a dependency edge without any test noticing, leaving a validation job that runs, fails, and blocks nothing. A periodic read of the deployment workflow — asking only “can any deploy job start without a passing validation upstream?” — catches that in a minute.

Branch protection is the complement to that review. A required status check on the default branch makes the validation and scanning jobs non-optional for merges, which closes the gap where someone with push access bypasses the pull request entirely. Without it, every guarantee on this page holds only for changes that happen to go through review, and the one change that skips review is disproportionately likely to be the urgent fix made under pressure — which is exactly when a configuration mistake is most likely and least noticed.

Scanning for secrets that already exist

The scanner half of the gate has a different shape from the validation half, because it is looking for something that should never appear rather than checking that something does. Two properties decide whether it is useful: what it scans, and what it does with what it finds.

Scanning only the current working tree misses the common case. A credential committed three months ago and removed last week is still in history, still in every clone, and still valid unless someone rotated it — so the CI job should scan history on the default branch, not just the diff. Scanning the diff is right for the pull request job, where speed matters and the question is “does this change introduce a secret?”; scanning full history is right for a scheduled job, where the question is “is there anything in here we missed?”

# .github/workflows/scan.yml — diff on PRs, full history on a schedule
on:
  pull_request:
  schedule:
    - cron: "0 3 * * 1"           # weekly full-history sweep
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: ${{ github.event_name == 'schedule' && 0 || 1 }}
      - run: gitleaks detect --no-banner --redact

--redact matters more than it looks. Without it, the scanner’s own output prints the secret it found, into a CI log that is often more widely readable than the place the secret was supposed to live — so a tool that exists to prevent disclosure performs one. Redacted output still names the file, the line, and the rule that matched, which is everything a developer needs in order to act on the finding without the value ever being written down again.

False positives are the other thing to plan for, because an unmanaged allow-list is how scanners get disabled. Test fixtures with fake keys, documentation examples, and high-entropy strings that are not credentials all trip detectors. Keep the ignore list in the repository, require each entry to carry a comment explaining why it is safe, and review it like any other code — an allow-list nobody reads eventually contains a real secret somebody wanted to ship.

Diff scanning versus full-history scanning Pull request jobs scan the diff for speed, while a scheduled job scans full history to find credentials committed and removed in the past. on pull request scans the diff only seconds — runs on every push blocks the merge "does this change add a secret?" on a schedule scans full history minutes — runs weekly opens an issue on a hit "is there anything we missed?"
Two jobs, two questions — the diff scan blocks new leaks, the sweep finds the ones already in history.

Troubleshooting

  • Build passes but production fails — CI validated against the wrong environment’s variables; validate against the target environment.
  • Secret leaked despite pre-commit — someone ran git commit --no-verify; the CI gitleaks job is the backstop. See Pre-commit Hook to Block Committed Secrets.
  • extra not permitted in CI only — a variable is set in CI but not in the model; add or remove it deliberately.
  • Secret printed in CI logs — the field is a plain str; switch to SecretStr.
  • Validation passes, deploy uses different values — the deploy job reads a different secret scope than the validation job; make both use the same environment.
  • Scanner reports a finding in a deleted file — history is being scanned, which is correct; rotate the credential, then decide whether to purge.

The first entry is worth a longer look because it accounts for most false confidence in this area. A pipeline that validates against repository-level secrets and deploys using environment-level secrets is testing one configuration and shipping another, and everything about the run looks green. The diagnostic is to print the set of key names the validation job resolved and compare it with the deploy job’s environment definition; if the two lists differ, the gate is checking the wrong thing. Making both jobs declare the same environment: is the structural fix, since it removes the possibility of them diverging rather than merely detecting it.

The scanner entry catches people out in the opposite direction. A finding in a file that no longer exists looks like a bug in the tool and is usually a correct report about history — the file is gone from the working tree and the credential is still in the pack. Treat the credential as live until somebody has rotated it and confirmed the old value no longer works.

Symptom to cause for pipeline validation problems Four symptoms mapped to causes: validating against the wrong environment, a bypassed pre-commit hook, a variable missing from the model, and a plain string secret field. symptom cause green build, failed deploy validated against dummy CI values secret in history despite the hook git commit --no-verify — CI is the backstop extra not permitted, CI only variable set in CI but not declared on the model value visible in the job log plain str field — switch to SecretStr
Every row is a one-line fix, and the first is by far the most common.

Frequently asked questions

How do I validate configuration in a CI pipeline?

Add a stage that imports and instantiates your pydantic-settings model against the target environment’s variables. If construction raises a ValidationError, the stage fails and the build stops before the misconfiguration reaches production.

Should secret scanning run in CI or in pre-commit?

Both. A pre-commit hook blocks the obvious local mistake, and a CI job is the enforced gate that cannot be bypassed with --no-verify. Run gitleaks or detect-secrets in both places.

How do I keep real secrets out of CI logs?

Use the platform’s masked variables, never echo configuration, and run validation with output that reports key names only. Wrap secret fields in SecretStr so an accidental print is masked.

Key takeaways

The invariant: no configuration reaches an environment it has not passed validation for, and no secret reaches history without tripping a scanner. The pipeline, not production, is where misconfiguration dies. The whole gate is a dozen lines of Python plus a job definition, because the settings model already encodes what “valid” means — CI’s contribution is to run that specification against each environment’s real values at the moment before deployment.

Two structural details separate a gate that works from one that only looks like it does. Validate against the target environment’s variables, not a set of CI placeholders, or the check proves nothing about where you are deploying. And wire the deploy job’s needs: to the validation job, so a failure blocks rather than merely reports. Get those two right, add extra="forbid" so typos surface as themselves, and the class of incident where a service starts with the wrong configuration stops occurring. Everything beyond that — the source-map diff, the model’s own tests, the scheduled history sweep — is worth adding as the service grows, but none of it substitutes for the two structural details, and a pipeline that has them is already ahead of most.

Back to Core Configuration Patterns