Nested settings models in pydantic

Flat configuration becomes unreadable once you have CACHE_HOST, CACHE_PORT, CACHE_SSL, DB_HOST, DB_PORT. Nested settings models group related fields into sub-objects while still reading from flat environment variables. This page builds them, extending Pydantic Settings Fundamentals.

The payoff is not merely tidier code. When cache configuration lives in a CacheConfig sub-model, you can pass that whole object to the function that builds your Redis client, test it in isolation, give it its own defaults and validators, and reuse it across services — none of which is possible when the same five values are loose attributes on one giant settings class. Grouping turns configuration from a flat bag of strings into a small domain model, and it does so without giving up the flat PARENT__CHILD environment-variable form that every deployment platform already speaks. You get structure in your Python and flatness in your environment, which is exactly the right split, and you get it without adopting a new file format, a config server, or any machinery beyond a single line of model_config.

Problem 1: a flat soup of prefixed fields

# ANTI-PATTERN: related fields scattered flat
class Settings(BaseSettings):
    cache_host: str
    cache_port: int
    cache_ssl: bool
    db_host: str
    db_port: int        # no grouping, no reuse

There is no structure and no way to pass “the cache config” as one object. The damage compounds as the model grows. Every consumer that needs cache settings has to reach into the top-level settings object and pluck out cache_host, cache_port, and cache_ssl by name, so the knowledge of “which fields belong to the cache” is duplicated at every call site instead of living in one type. A function that builds a Redis client takes three loosely-related arguments rather than one CacheConfig, and nothing stops a caller from passing db_port where cache_port was expected — they are both int, so the type checker is silent. Testing suffers too: to exercise a cache-related code path you must construct an entire settings object with database fields, log levels, and everything else set, when all you needed was three cache values. That friction quietly discourages testing the very code that most needs it, because standing up a full valid configuration for one small assertion is tedious enough that people skip it.

The flat prefix is a naming convention pretending to be structure. cache_ in front of three fields tells a human they are related, but the code cannot act on that relationship — there is no cache object to pass, validate, or reuse. Nested models make the relationship real: the fields become attributes of a CacheConfig type that you can hand around as a unit, and the grouping the prefix only hinted at becomes something the language enforces.

Problem 2: wrong delimiter, empty sub-model

# ANTI-PATTERN: single underscore collides with field names
model_config = SettingsConfigDict(env_nested_delimiter="_")   # CACHE_HOST ambiguous

A single-underscore delimiter clashes with normal field names; the sub-model ends up empty or mis-parsed. The mechanism is worth understanding because the failure is silent. When the delimiter is _, pydantic-settings tries to split every variable on every underscore, so CACHE_HOST could be read as cache.host (nested) — but a field literally named cache_host on the model, or a sub-model whose own field contains an underscore, becomes ambiguous. The parser cannot tell which underscores are structural and which are part of a name, and the usual result is a sub-model that never receives its values and either falls back to defaults or raises a “missing field” error for a variable you plainly set. You spend an hour certain the environment is right, because it is — the delimiter is what is wrong.

The double underscore fixes this by choosing a separator that essentially never appears inside a real field name. CACHE__HOST splits unambiguously into cache and host; a stray single underscore inside a field name is left alone. This is why __ is the near-universal convention for nested pydantic-settings and why the platforms that inject configuration — Kubernetes, ECS, Compose — all handle a double-underscore variable name without complaint. Pick __ and the ambiguity simply cannot arise. It is a one-character change with an outsized payoff: the difference between a delimiter that fights your field names and one that never touches them.

Secure implementation

# config/nested.py
from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict

class CacheConfig(BaseModel):
    host: str
    port: int = 6379
    ssl: bool = True

class DBConfig(BaseModel):
    host: str
    port: int = 5432

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_nested_delimiter="__",      # double underscore avoids field-name clashes
        extra="forbid",
    )
    cache: CacheConfig
    db: DBConfig

# Env: CACHE__HOST=redis.local  CACHE__SSL=true  DB__HOST=pg.local
settings = Settings()
settings.cache.ssl   # typed bool, grouped under a reusable sub-model

env_nested_delimiter="__" maps CACHE__HOST to cache.host. The same flat variables that Kubernetes injects populate structured, reusable sub-models — exactly the format used in YAML config.

The key distinction in that code is that the sub-models are plain BaseModel, not BaseSettings. Only the top-level Settings class reads from the environment; the sub-models are ordinary pydantic models that receive their already-sourced values from the parent. This matters because making a sub-model a BaseSettings would give it its own independent source-reading behaviour, which is both unnecessary and a source of confusing double-reads. The rule is simple: one BaseSettings at the top that owns all sourcing, and plain BaseModel sub-objects beneath it that only provide structure and validation.

Because the sub-models are real pydantic models, everything you can do to a top-level field you can do to a sub-model field. CacheConfig.port has a default of 6379, so CACHE__PORT is optional; CacheConfig.host has no default, so CACHE__HOST is required and its absence raises at startup naming cache.host precisely. You can attach Field() constraints (port: int = Field(ge=1, le=65535)), @field_validators, and even a @model_validator that checks the sub-model’s fields against each other — for example, requiring a certificate path when ssl is true. The grouping does not weaken validation; it localises it, so the cache’s rules live on the cache type where they belong. Nesting can also go deeper than one level — a DatabaseConfig can itself contain a PoolConfig populated by DB__POOL__SIZE — though two levels is where most configurations sensibly stop before the nesting becomes harder to read than the flatness it replaced.

Where the grouping pays off is at the point of use. A cache-client factory can take the whole sub-model — def build_redis(cfg: CacheConfig) -> Redis — so its signature documents exactly which settings it needs and the type checker guarantees the caller passes a coherent set rather than three unrelated integers. That function can be unit-tested by constructing a CacheConfig directly, with no need to stand up the entire application settings object, so a test for cache behaviour depends only on cache configuration and stays fast, focused, and free of unrelated setup that would otherwise rot as the rest of the settings model changes. And because CacheConfig is a normal type, you can define it once in a shared library and import it into every service that talks to the same cache, giving a fleet of services one agreed shape for “how we configure Redis” instead of each re-declaring the same three fields with subtly different defaults. The sub-model becomes a small, reusable contract, which is precisely what the flat prefix could only gesture at but never actually enforce or hand to another function.

Gotchas & version-specific behaviour

• Use __ (double underscore) as the delimiter so it never collides with field names.
• Sub-models are plain BaseModel, not BaseSettings.
• Defaults on sub-model fields work normally; required sub-fields raise if unset.
• The same CACHE__HOST form works locally in a .env and in Kubernetes secrets — full parity.

A few behaviours surprise people the first time. There are actually two ways to populate a sub-model, and pydantic-settings accepts both: the per-field form (CACHE__HOST=redis.local, CACHE__PORT=6380) and a whole-object JSON form (CACHE={"host":"redis.local","port":6380}). The per-field form is what you want for platform-injected configuration because each value is a separate, individually-overridable variable; the JSON form is occasionally handy when a whole block is managed as one unit, but it is easy to get the quoting wrong and it collapses five clear variables into one opaque one. Prefer the double-underscore per-field form and reach for JSON only when a value is genuinely a structured blob.

Case-insensitivity interacts with nesting exactly as you would hope: with case_sensitive=False, CACHE__HOST fills cache.host regardless of case, so the upper-snake environment convention maps cleanly onto lower-snake Python at every level. And extra="forbid" applies inside the nesting too — a CACHE__HSOT typo is rejected as an unknown key on the cache sub-model, not silently dropped, which is the whole reason to keep forbid on. One genuine limitation to know: a sub-model field whose own name contains a double underscore would re-introduce ambiguity, so keep sub-model field names to single words or single underscores and let __ mean nesting and nothing else.

Production parity checklist

• Related fields grouped into BaseModel sub-models.
• env_nested_delimiter="__" set; variables use the double-underscore form.
• extra="forbid" rejects unexpected nested keys.
• Local .env uses the same PARENT__CHILD keys as production injection.
• Required sub-fields validated at startup.

The parity claim in that checklist is the one to test rather than trust. It is genuinely easy for a local .env to use CACHE_HOST (single underscore, matching an old flat model) while the Kubernetes manifest injects CACHE__HOST (double underscore, matching the nested one), so the two environments disagree about how the same value is spelled and the nested model is populated in production but not locally, or vice versa. A construction test in CI — build the model against the real staging variable set and assert the sub-models are populated — catches that drift before it ships. Because the whole point of nested models is that the flat key form is identical everywhere, a mismatch is a bug in your manifests, not a limitation of the approach.

How this maps onto Kubernetes and Compose

The double-underscore form is not a pydantic quirk you have to work around on real platforms — it is a plain environment-variable name, and every orchestrator handles it natively. In a Kubernetes Deployment you set name: CACHE__HOST in the container’s env list, or project it from a Secret or ConfigMap key of the same name; in Docker Compose you write CACHE__HOST: redis under environment; in an ECS task definition it is an ordinary key-value pair. Nothing special is required because a double underscore is a legal character sequence in an environment-variable name across all of them. That universality is what makes nested models safe to adopt: the structure lives entirely in your Python, the platform sees only flat names, and the two meet at the delimiter. A developer reads settings.cache.host in code; the operator sets CACHE__HOST in a manifest; and the model is the single place that connects the two, validated at startup.

Key takeaways

Nested sub-models give structure and reuse while reading the same flat variables every platform injects. The pattern is a small, high-leverage one: group related fields into plain BaseModel sub-objects, keep a single BaseSettings at the top to own all sourcing, set env_nested_delimiter="__" so the flat PARENT__CHILD form maps unambiguously onto the structure, and keep extra="forbid" so a typo in a nested key fails loudly rather than vanishing. What you get back is configuration you can pass around as domain objects — a CacheConfig to the cache-client factory, a DatabaseConfig to the connection setup — each with its own defaults and validators, tested in isolation, and reused across services.

The reason this composes so cleanly with the rest of your stack is that nothing about the environment changed. Kubernetes still injects CACHE__HOST as an ordinary environment variable; your .env still lists the same key locally; CI still constructs the model against the same flat inputs. Only the Python side gained structure. That is the whole trick — structure where you write code, flatness where platforms inject values — and it is why nested models are the right default for any service whose configuration has grown past a handful of unrelated fields into groups that clearly belong together, such as a cache block, a database block, and an outbound-HTTP block. For nested file config, see Handling Nested Configuration in YAML Safely.