Type-Safe Validation with Pydantic Settings

Configuration that loads is not the same as configuration that is correct. pydantic-settings turns the loose strings coming from the environment into a validated, typed object that either constructs cleanly at startup or refuses to start at all. This section covers how to build that object, control its coercion behaviour, extend it with domain rules, feed it secrets safely, and evolve its schema without breaking running services.

The distinction matters because the alternative — reading os.environ["PORT"] where you need it and hoping the value is sane — pushes every configuration mistake to the worst possible moment. A missing variable becomes a KeyError on the first request that touches it; a malformed one becomes a ValueError deep inside a library, three stack frames from anything you can act on; a value that is merely wrong (a staging database URL in production) becomes a data-integrity incident with no exception at all. Each of those is a failure that a typed, validated settings object converts into a single, precise error at process start, before the service ever binds a port. The whole discipline in this section is about moving failures left: from production to startup, and from startup to the pull request that introduced them.

pydantic-settings is the settings layer built on top of Pydantic v2. Pydantic supplies the validation engine — the field types, constraints, coercion rules, and validators; pydantic-settings adds the BaseSettings base class that knows how to source values from the environment, from .env files, from Docker secret mounts, and from custom providers, and then feed them through that engine. The two libraries are versioned together, so installing pydantic-settings pulls in a compatible Pydantic v2. Everything below assumes v2 idioms — SettingsConfigDict, Annotated types, and model_config — rather than the v1 inner class Config that older tutorials still show.

What this section covers

The seven topics below build on one another: fundamentals establish the model, coercion and validators tighten it, secrets and multi-environment overrides feed it real values, redaction decides what may ever be rendered, and schema evolution keeps the whole thing changeable. Each links to a dedicated page for depth.

Read in order, they answer a single question at increasing resolution: what does it take to trust the configuration a process is running on? Fundamentals answer “where do the values come from and how do I bind them to a model”; strict mode answers “when is "1" an integer and when must it not be”; custom validators answer “how do I express rules a type cannot”; schema evolution answers “how do I change all of that without a flag day”; and the two sourcing topics answer “where do the real values live in a managed environment”. You do not need every page to start — one typed BaseSettings model with extra="forbid" already moves most failures to startup — but each topic closes a gap that a growing system eventually hits.

One settings object, one entry point

Pick a single BaseSettings subclass that owns every environment read. Centralizing prevents the configuration sprawl that makes applications untestable, and it gives you one place to enforce the three security invariants that recur across this whole hub: SecretStr for credentials, extra="forbid" for unknown keys, and override=False where an existing environment value must win.

# config/base.py
from pydantic import SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict

class AppConfig(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_nested_delimiter="__",  # CACHE__HOST -> cache.host
        extra="forbid",             # reject unknown vars instead of ignoring them
    )
    database_url: str
    api_key: SecretStr              # never appears in repr() or model_dump()
    log_level: str = "INFO"

config = AppConfig()  # raises ValidationError if anything is missing or malformed

When you construct AppConfig(), pydantic-settings reads each field from the environment (falling back to the .env file), coerces it to the declared type, runs any validators, and only then hands you an object. Any process that imports config sees the same validated values, so tests, workers, and the web server can never disagree about what the configuration is.

There are two ways to expose that single instance, and the choice has consequences. A module-level config = AppConfig() constructs the settings at import time: the first module that imports config triggers the read, and if anything is wrong the import itself fails. That is the strongest form of fail-fast — the process cannot even finish loading with a broken configuration — but it makes the settings hard to substitute in tests, because the object is built before your test fixtures run. The alternative is a cached accessor:

# config/base.py
from functools import lru_cache

@lru_cache
def get_settings() -> AppConfig:
    return AppConfig()  # built once, on first call, then cached

Here nothing is read until the first get_settings() call, and tests can clear the cache (get_settings.cache_clear()) after patching the environment to force a fresh read. Frameworks such as FastAPI lean on exactly this shape, wiring get_settings in as a dependency so a request handler receives the validated object without importing a global. The rule of thumb: use the module-level instance for scripts and workers where import-time failure is a feature, and the cached accessor for web applications and anything with a test suite that needs to vary configuration.

Whichever you choose, resist the temptation to reach around the object. The moment one module calls os.getenv("REDIS_URL") directly while the rest read config.redis_url, you have two sources of truth that will drift the first time someone adds a default in one place but not the other. Centralization is not tidiness for its own sake; it is what makes the settings testable, greppable, and provably consistent across every entry point.

Key rule: use extra="forbid" so a typo’d DATABSE_URL fails loudly instead of being silently ignored. Initialization details — source precedence, .env discovery, and secrets directories — are in Pydantic Settings Fundamentals.

Know where each value actually comes from

A single field can be supplied from several places, and when two of them disagree the result is not random — pydantic-settings resolves it by a fixed precedence. From highest priority to lowest: arguments passed directly to the constructor, then real environment variables, then values in the .env file, then files in a secrets directory, and finally the default declared on the field. The moment a higher source provides a value, the lower ones for that field are ignored. Understanding this order is what lets you reason about why a setting has the value it does, and it is the difference between “the deploy shipped the wrong timeout” being a five-minute look and an afternoon of confusion.

The practical consequences fall out of the ordering directly. Because real environment variables outrank the .env file, a developer can override any single value for one run — LOG_LEVEL=DEBUG python -m app — without editing a committed file, which is exactly what you want for local experimentation. Because constructor arguments outrank everything, tests can pass values in explicitly and be certain nothing in the ambient environment leaks in. And because defaults sit at the bottom, a field with a sensible default is genuinely optional in every environment that is happy with it, while a field with no default is mandatory everywhere — the type signature alone tells you which settings an operator must provide.

# highest priority wins, top to bottom
AppConfig(port=9000)          # 1. explicit constructor argument
# PORT=8080 in the real environment   2. environment variable
# PORT=3000 in .env                    3. .env file
# /run/secrets/port contents           4. secrets directory
port: int = 8000              # 5. field default

This is also why the advice throughout this section is to route every read through the model. Once a value can arrive from five ordered sources, a stray os.getenv("PORT") elsewhere in the code sees only the second of those five and silently ignores the other four — the constructor argument your test passed, the .env your teammate relies on, the secret your platform mounts. The model is the one place that honours the full precedence, so it must be the only place that reads. The ordering itself is customisable — you can reprioritise sources or add your own, such as a call to a parameter store — and that mechanism is covered in Pydantic Settings Fundamentals.

Control coercion before it surprises you

By default Pydantic coerces "8080" to 8080 and "true" to True. That is convenient for environment variables — which are always strings — but dangerous when you actually need a strict type boundary, for example a feature flag that must be a real boolean and not the string "false" (which is truthy in plain Python).

# config/strict.py
from pydantic import Field
from pydantic_settings import BaseSettings

class ServerConfig(BaseSettings):
    port: int = Field(ge=1, le=65535)   # range-checked, not just typed
    workers: int = Field(default=4, ge=1, le=64)
    debug: bool = False                 # "0"/"false"/"no" all become False

A typed field rejects "abc" as a port automatically, and the Field(ge=1, le=65535) constraint additionally rejects 0 and 70000. Knowing exactly which fields coerce loosely and which are strict is the difference between a forgiving developer experience and a silent production bug.

The boolean field is where lax coercion earns its keep and also where naive code fails. Pydantic recognises "1", "true", "yes", and "on" (case-insensitively) as True, and "0", "false", "no", and "off" as False. Contrast that with the hand-rolled os.getenv("DEBUG") == "true", which quietly treats DEBUG=True (capital T) or DEBUG=1 as false, or worse, bool(os.getenv("DEBUG")), which is True for the string "false" because every non-empty string is truthy in Python. That single bug — a “disabled” feature flag that is actually on because someone wrote FLAG=false — is one of the most common ways a production toggle betrays its owner, and typing the field as bool on a settings model eliminates it outright.

Strict mode is the escape hatch for the fields where lax coercion is a liability rather than a convenience. You can set it per field with Field(strict=True) or model-wide in SettingsConfigDict(strict=True), and it changes coercion from “convert if plausible” to “accept only the exact type”. The judgement call is which fields deserve it. A port read from an operator-set variable benefits from lax coercion — everyone expects "8080" to become 8080. A value that arrives from an upstream system as structured JSON, where a string where you expected an integer signals a real bug in the producer, benefits from strict mode catching the mismatch instead of silently papering over it. Reaching for strict mode everywhere is a common over-correction; it makes the settings brittle against the very string inputs the environment is built to deliver.

Composite types have their own rules worth knowing before they surprise you. A field typed list[str] or dict[str, int] is parsed from the environment as JSON: HOSTS=["a","b"] works, but HOSTS=a,b raises a validation error, because pydantic-settings does not guess at a delimiter. If you want comma-separated input you either add a field_validator(mode="before") that splits the string, or you accept the JSON convention and document it. Nested models populate from a delimiter — with env_nested_delimiter="__", the variable CACHE__HOST fills cache.host — which lets one flat set of environment variables hydrate a structured configuration tree without any manual parsing.

Key rule: decide per field whether loose coercion is a feature or a hazard — the full rules, including strict=True, the model-wide strict config, and the JSON-vs-delimiter behaviour for collections, are in Strict Mode & Type Coercion.

Encode domain knowledge as validators

Basic types reject "abc" as a port, but only a domain validator rejects a malformed ARN or a non-TLS database URL. Push that knowledge into the model so it runs everywhere the config loads, not just in the one code path a developer remembered to guard.

# config/validators.py
from pydantic import field_validator
from pydantic_settings import BaseSettings

class DataConfig(BaseSettings):
    database_url: str

    @field_validator("database_url")
    @classmethod
    def require_tls(cls, v: str) -> str:
        if not v.startswith(("postgresql+psycopg://", "postgresql://")):
            raise ValueError("database_url must be a PostgreSQL DSN")
        if "sslmode=require" not in v:
            raise ValueError("database_url must require TLS (sslmode=require)")
        return v

Validators run at construction, so a bad value can never reach business logic — the process fails at startup with a precise message instead of throwing a connection error under load an hour later. The same pattern encodes ARN formats, Redis URL schemes, and API-key prefixes.

Two distinctions decide how a validator behaves. The first is mode: a @field_validator(..., mode="before") runs on the raw input before coercion — the right place to normalise, for example to strip whitespace or split a comma list into a real list — while the default mode="after" runs on the already-typed value, which is where format checks belong because you can trust the type. The second is scope: @field_validator sees one field in isolation, but some rules span fields — “if AUTH_MODE is oauth then OAUTH_CLIENT_ID is required” — and those belong in a @model_validator(mode="after"), which receives the fully-built model and can compare fields against each other. Reaching for a model validator when a rule genuinely couples two settings is what keeps invalid combinations out, not just invalid individual values.

The quality of a validator is measured by its error message as much as its logic. When AppConfig() fails, pydantic raises a single ValidationError that lists every field that failed, each with a loc pointing at the field name and the message your raise ValueError(...) supplied — so a good message reads like an instruction to the operator (“database_url must require TLS (sslmode=require)”) rather than a stack trace. This is a concrete reason to prefer validators over scattered runtime checks: one construction call surfaces all the configuration problems at once, so an operator fixes the whole set in a single edit instead of discovering them one failed deploy at a time. Write the message for the person who will read it in a CI log at the moment a deploy is blocked, not for yourself while the rule is fresh in your mind — a message that names the field, states the expectation, and shows the accepted form turns a red build into a self-service fix.

When the same rule recurs across models, lift it into a reusable annotated type instead of copying the validator:

from typing import Annotated
from pydantic import AfterValidator

def _require_https(v: str) -> str:
    if not v.startswith("https://"):
        raise ValueError("must be an https:// URL")
    return v

HttpsUrl = Annotated[str, AfterValidator(_require_https)]

class WebhookConfig(BaseSettings):
    callback_url: HttpsUrl        # the rule travels with the type

Now every field typed HttpsUrl carries the check automatically, the rule is unit-testable in isolation, and there is exactly one place to fix it. This is how a growing configuration stays consistent: domain constraints become named types, not scattered if statements.

Key rule: validators are the place to encode every “this string must look like X” rule you would otherwise scatter through the codebase. Field-versus-model scope, before/after mode, and ARN and URL recipes are in Custom Validators & Constraints.

Keep secrets out of logs and dumps

A database password typed as a plain str will eventually appear in a log line, a traceback, or a model_dump() sent to an error tracker. Typing it as SecretStr makes that leak structurally impossible: the value is masked in repr() and excluded from serialization unless you explicitly call .get_secret_value().

# config/secrets.py
from pydantic import SecretStr
from pydantic_settings import BaseSettings

class Secrets(BaseSettings):
    api_key: SecretStr
    db_password: SecretStr

s = Secrets()
print(s)                       # api_key=SecretStr('**********') ...
print(s.api_key.get_secret_value())  # explicit, greppable, auditable

Because reading a secret now requires the explicit .get_secret_value() call, every place your code touches a credential is greppable and reviewable. A SecretStr also compares by value, so settings.api_key == SecretStr(expected) works in tests without ever unmasking, and it composes: a nested DatabaseConfig(BaseModel) field typed SecretStr stays masked even when the parent settings object is dumped. The one habit to build is to defer unmasking to the last possible moment — pass the SecretStr around your code and call .get_secret_value() only at the boundary where you hand the credential to a driver or HTTP client, so the plaintext lives in as few frames as possible.

There is a matching pattern for reading a secret into a connection string, where the plaintext genuinely must appear. Build the string at the boundary, from get_secret_value(), and never store the assembled result on the model: f"postgresql://user:{cfg.db_password.get_secret_value()}@{cfg.db_host}/app" produces a plain string that you hand straight to the driver and let go out of scope, so the credential is unmasked for exactly one expression rather than living on a long-lived object that something might later serialize. The SecretStr stays the source of truth; the plaintext is a transient that exists only for as long as the connection call needs it, and the rest of your code continues to hold and pass the masked wrapper.

Knowing the boundary of that protection matters as much as using it. The wrapper changes how a value renders, so it covers printing, logged dumps and tracebacks that reference the settings object — and it covers nothing once you unwrap, nothing an error tracker captures from a frame’s local variables, and nothing about a parsed URL type whose str() includes a password. Secret types and redaction maps that surface in full, including the credentials that merely pass through a service in request headers and never touch the settings model at all.

The failure this prevents is mundane and constant. Error trackers serialize local variables; structured loggers dump context dictionaries; a hastily added print(settings) during a debugging session ends up in a log aggregator that a dozen teams can read. SecretStr makes all three of those paths safe by default, and turns “remember never to log the config” — an instruction that will eventually be forgotten — into a property of the type that cannot be forgotten. Feed these fields from a real secret manager rather than a committed .env — see enterprise secrets management for Vault, AWS Secrets Manager, and Doppler integrations.

Evolve the schema without downtime

Configuration schemas change: a field gets renamed, a default is tightened, an option is retired. Each of those is a breaking change unless you accept both the old and new shape during a migration window using validation_alias and AliasChoices, so old and new deployments can run side by side.

# config/evolve.py
from pydantic import AliasChoices, Field
from pydantic_settings import BaseSettings

class Config(BaseSettings):
    # accepts DATABASE_URL (new) and DB_DSN (legacy) during the migration
    database_url: str = Field(validation_alias=AliasChoices("DATABASE_URL", "DB_DSN"))

With both names accepted, you rename the variable in one environment at a time and remove the alias only after every deployment has moved. The sequencing is what makes it safe: you never have a moment where a running process expects a name that its environment does not yet provide, because during the alias window both names validate. Skip the alias and rename in a single deploy, and any instance still running the old code — a worker that has not cycled, a canary that lags the fleet — starts raising ValidationError the instant its environment flips.

Removing a field follows the mirror image of the same discipline. You first stop reading it in code while leaving it declared as an ignored, optional field, deploy that, and only then delete the field and the variable — otherwise extra="forbid" turns a leftover variable in someone’s environment into a hard startup failure. Tightening a constraint (say, making a previously free-form region an enum) deserves the same care: ship the stricter schema to a canary, watch for the values real environments actually carry, and widen the enum to cover the legitimate stragglers before it reaches the whole fleet. Every schema change is a small migration, and treating it as one — additive first, subtractive last, with a window where both shapes are valid — is what lets configuration evolve without a coordinated, downtime-inducing flag day. The full playbook — deprecation warnings, removed fields, and versioned schemas — is in Schema Evolution & Versioning.

Anti-patterns & common mistakes

• Any or bare Optional for required config — defeats the entire point of validation and pushes the failure to the first request.
• Reading os.environ directly alongside the model — two sources of truth that drift apart; route every read through the model.
• Printing the config object in logs — leaks anything not wrapped in SecretStr; add a test that asserts repr(config) contains no secret values.
• Catching ValidationError and continuing — start-up errors must be fatal, not swallowed; a half-configured process is worse than a stopped one.
• Per-environment subclasses with diverging fields — keep one schema; vary only the values via multi-environment overrides.
• Legacy inner class Config — pydantic v2 uses SettingsConfigDict; the old style raises deprecation warnings and will eventually break.

What unites these is a single anti-pattern: treating validation as optional decoration you can route around. The Any type, the stray os.environ read, and the swallowed ValidationError each punch a hole in the guarantee that the settings object exists to provide — that if the process is running, its configuration is known-good. A validated model is only as strong as its weakest bypass, so the discipline is not merely “use pydantic” but “let nothing else read configuration”. The most insidious of the six is catching ValidationError and limping onward with defaults: it converts a loud, actionable startup failure into a service that runs with the wrong values and looks healthy, which is precisely the outcome the whole exercise was meant to prevent.

Decision flow: which validation tool?

Not every field needs a hand-written validator. Reach for the lightest tool that expresses the rule: a typed field for fixed shapes, Field() constraints for ranges, a @field_validator for formats, and SecretStr for anything sensitive. Over-reaching costs you clarity — a five-line validator that reimplements ge=1, le=65535 is harder to read and slower than the constraint it duplicates, and it hides the intent from anyone skimming the model. Under-reaching costs you safety — a plain str where the value must be one of three log levels lets a typo through to the logging call. The skill is matching the tool to the shape of the rule, and the ladder below runs from the cheapest, most declarative option to the most expressive.

Read the decision left to right as a preference order. If a concrete type plus a Field() bound can express the rule, stop there; only escalate to a @field_validator when the constraint is a format a type cannot capture, and to a @model_validator when it couples fields. Secrets short-circuit the whole ladder: anything sensitive is SecretStr regardless of its other constraints, because the masking behaviour is orthogonal to and more important than the format check.

CI/CD integration checklist

The cheapest place to catch a bad configuration is a pull request, not a pager. These steps turn the settings model into a build-time gate.

1. Instantiate the settings model in a CI step; a failed construction fails the build.
2. Run with extra="forbid" so unknown variables are caught in review, not production.
3. Assert repr(config) contains no secret values as a regression test for SecretStr usage.
4. Pin the pydantic and pydantic-settings versions and test the v1→v2 behaviour explicitly.
5. Snapshot config.model_dump() (with secrets excluded) and diff it across environments to catch drift.

Step one is the whole game in miniature: a job that does nothing but python -c "from app.config import get_settings; get_settings()" against each environment’s real variable set turns “did anyone forget to add the new SENTRY_DSN to staging?” from a 2 a.m. discovery into a red check on the pull request. Pair it with extra="forbid" (step two) and the same job also catches the opposite mistake — a variable that is set but no longer read, which is the fingerprint of a half-finished migration. Step three closes the loop on secrets by asserting that a rendered repr(config) and a model_dump() contain none of the known credential values, so a future field added as a plain str instead of SecretStr fails the build rather than leaking in a log six weeks later.

Steps four and five defend against slower failures. Pinning both libraries and testing their behaviour explicitly means a Pydantic point release cannot silently change how a field coerces underneath you; a snapshot of the redacted model_dump(), diffed across environments, surfaces the drift — a timeout that is 30 in staging and 3 in production because someone tuned one and forgot the other — that no single-environment test can see. Together the five turn the settings model from a runtime object into a contract the pipeline enforces on every push.

Test the model the way production builds it

A settings model is code, and code that gates every startup deserves tests. The awkward part is that the model reads from the ambient environment, so a naive test is at the mercy of whatever variables the CI runner happens to export. The fix is to control the environment explicitly. Pytest’s monkeypatch fixture sets and clears variables around a single test, and passing _env_file=None at construction tells pydantic-settings to ignore any .env on disk, so the test sees exactly the values you set and nothing else.

# tests/test_config.py
import pytest
from pydantic import ValidationError
from app.config import AppConfig

def test_rejects_unknown_variable(monkeypatch):
    monkeypatch.setenv("DATABASE_URL", "postgresql://db/app?sslmode=require")
    monkeypatch.setenv("API_KEY", "k-live-xxx")
    monkeypatch.setenv("TYPOED_VAR", "1")        # not a declared field
    with pytest.raises(ValidationError):
        AppConfig(_env_file=None)                # extra="forbid" catches it

def test_secret_never_in_repr(monkeypatch):
    monkeypatch.setenv("DATABASE_URL", "postgresql://db/app?sslmode=require")
    monkeypatch.setenv("API_KEY", "k-live-supersecret")
    cfg = AppConfig(_env_file=None)
    assert "supersecret" not in repr(cfg)         # SecretStr masks it

Three kinds of test earn their place. A happy-path test proves a realistic environment constructs cleanly, so a tightened constraint that accidentally rejects a valid value is caught immediately. A rejection test asserts that a known-bad value — a port of 70000, a database URL without TLS, an unknown variable — raises ValidationError, which is the only way to know your validators actually fire. And a secrets test asserts no credential appears in repr() or model_dump(), guarding the one property that is invisible until it fails. If you expose settings through the cached get_settings() accessor, remember to call get_settings.cache_clear() in a fixture so each test builds against its own environment rather than the first one that ran.

Bringing it together

A single BaseSettings model is the contract between the raw configuration sources and your application. Control coercion, encode domain rules, evolve the schema safely, source values from AWS Parameter Store and per-environment overrides, and feed secrets in from enterprise secret managers as SecretStr. The result is a process that proves its configuration is correct before it accepts a single request.

The payoff compounds beyond the first startup. A validated model becomes documentation that cannot go stale — the field list is the exhaustive set of variables the service reads, their types, and their defaults, which is precisely the runbook an on-call engineer wishes existed at 3 a.m. It becomes a test surface, because “the config is valid” is now an assertion a CI job can make rather than a hope. And it becomes a refactoring anchor: renaming a variable, tightening a constraint, or splitting a monolith’s configuration into services all become mechanical changes guarded by the same validator that guards production. The upfront cost is a handful of type annotations; the return is that an entire category of failure — the misconfigured process that looks healthy until it isn’t — stops being possible.

If you take one thing from this section, take the sequencing. Type every field and forbid the unknown ones, so typos and drift fail loudly. Decide coercion per field, so a string flag is never mistaken for a boolean. Wrap every secret in SecretStr, so a credential cannot leak through a log line. Push domain rules into validators and named types, so invalid values never reach business logic. And treat every schema change as a small migration with a window where both shapes are valid, so configuration can evolve without downtime. Each is cheap on its own; together they turn configuration from the thing that pages you into the thing you stop thinking about.