Parse List and JSON Env Vars in Pydantic

Environment variables are strings, and configuration is not. A list of allowed hosts, a mapping of feature flags, a nested database section — each has to survive the trip through a flat string namespace, and pydantic-settings has clear rules about how. Knowing those rules turns a confusing JSONDecodeError at startup into a design decision about which form your operators should write. This page covers both, extending the strict mode and type coercion topic.

Problem 1: a comma-separated list that fails to parse

# ANTI-PATTERN: the obvious form, which pydantic-settings does not accept for a list field
ALLOWED_HOSTS=example.com,www.example.com

ValidationError: allowed_hosts
  Input should be a valid list [type=list_type, input_value='example.com,www.example.com']

For complex types — lists, dicts, nested models — pydantic-settings parses the variable as JSON before any type validation runs. ["example.com","www.example.com"] works; a bare comma-separated string does not, and the error mentions JSON rather than commas, which sends people looking in the wrong place.

You can either accept the JSON form, which is precise and unpleasant to write in a deployment template, or teach the field to accept the comma-separated form as well. The second is a five-line validator and is usually the right choice for anything a human edits, because the JSON form’s quoting is exactly the sort of detail that gets mangled in a YAML manifest.

Problem 2: JSON as the interface for hand-edited values

# ANTI-PATTERN: quoting that will eventually be got wrong
env:
  - name: FEATURE_FLAGS
    value: '{"new_checkout": true, "beta_search": false}'

Embedding JSON in a YAML manifest means two levels of quoting, and every layer between the author and the process — a templating tool, a shell, a secrets manager’s console — is an opportunity to mangle it. The failure is usually a stray escape that produces a string where an object was expected, and the error arrives at container start.

For structures an operator edits, prefer delimited keys: FEATURE_FLAGS__NEW_CHECKOUT=true with env_nested_delimiter="__" set on the model. Each variable is a simple scalar, no quoting is involved, and changing one flag touches one variable. Reserve the JSON form for values produced by tooling, where a single variable carrying a whole structure is genuinely convenient and nothing is typing quotes by hand.

Problem 3: an empty variable meaning two different things

# ANTI-PATTERN: what should this produce — an empty list, or a failure?
ALLOWED_ORIGINS=

An empty string is not valid JSON, so a complex field fails rather than falling back to a default. Sometimes that is correct — the variable was set by mistake and should be caught — and sometimes it is not, because a deployment template writes an empty value for a list that is legitimately empty in this environment.

Decide, and encode the decision in the before-validator. Treating an empty string as an empty list is reasonable for genuinely optional collections; treating it as “not set” and falling through to the default is reasonable for others; raising is reasonable for a list that must never be empty. What is not reasonable is leaving it to chance, because the three behaviours are indistinguishable until an environment writes an empty value.

Secure implementation

# config.py — accept the convenient form, keep the strict type
from typing import Annotated, Any
from pydantic import BeforeValidator, Field
from pydantic_settings import BaseSettings, SettingsConfigDict


def _split_commas(value: Any) -> Any:
    # 1. only strings are transformed; a real list from JSON passes straight through
    if not isinstance(value, str):
        return value
    stripped = value.strip()
    if stripped.startswith("["):
        return stripped                      # 2. looks like JSON: let the JSON step handle it
    if not stripped:
        return []                            # 3. an explicit decision about the empty case
    return [item.strip() for item in stripped.split(",") if item.strip()]


CommaList = Annotated[list[str], BeforeValidator(_split_commas)]


class DatabaseSettings(BaseSettings):
    host: str
    port: int = 5432
    pool_size: int = 20


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_nested_delimiter="__",           # 4. DATABASE__HOST populates database.host
        extra="forbid",
    )

    # 5. both "a,b" and '["a","b"]' work; the annotation stays list[str]
    allowed_hosts: CommaList = Field(default_factory=list)
    allowed_origins: CommaList = Field(default_factory=list)

    # 6. nested via delimited keys, no JSON quoting anywhere
    database: DatabaseSettings

# what an operator writes — scalars only, no quoting to get wrong
ALLOWED_HOSTS=example.com,www.example.com
DATABASE__HOST=db.internal
DATABASE__POOL_SIZE=40

The Annotated alias is what keeps this maintainable. Defining CommaList once means every list field gets identical behaviour, the annotation on each field still reads as list[str] for a type checker, and changing the parsing rule later is a single edit. Scattering @field_validator decorators achieves the same effect with more code and more opportunities for two fields to diverge.

The JSON passthrough in step 2 matters more than it looks. Tooling that generates configuration will emit JSON arrays, humans will write comma-separated strings, and both need to work in the same field — otherwise you end up with two variables for the same setting, or with a generator that has to know which fields are special. Detecting the leading bracket and deferring to the JSON step handles both forms with one line.

Documenting the accepted forms

A field that accepts two input forms needs to say so, or somebody will eventually discover the flexibility by accident and somebody else will remove it as dead code. The place to say it is the field’s description, which flows into the generated JSON Schema, into editor hover text, and into any documentation you generate from the model.

allowed_hosts: CommaList = Field(
    default_factory=list,
    description='Comma-separated hostnames, or a JSON array. Order is not significant.',
)

That last sentence is doing real work. For some lists the order matters — a sequence of fallback endpoints, a middleware chain — and for others it does not. Stating which is true saves a reader from having to find out empirically, and it tells a reviewer whether reordering a list in a deployment template is a cosmetic change or a behavioural one.

The corresponding tests are short and worth writing for every list field: the comma form, the JSON form, the empty case, and one malformed input that must raise. Four parametrised cases per field, run in milliseconds, and they pin behaviour that is otherwise implicit in a shared validator somebody may edit later. They also document the accepted forms in executable terms, which tends to outlast prose.

There is one more form worth knowing about, because it appears in almost every mature configuration eventually: a list of objects. A set of upstream backends, each with a host, a weight and a timeout, has no comfortable flat representation — delimited keys would need an index in the middle (BACKENDS__0__HOST), which works and reads poorly, and a comma-separated string cannot express structure at all. This is the case where JSON genuinely wins, and where the right move is usually to stop using environment variables for that particular setting and put it in a configuration file instead, leaving the variables for scalars.

That boundary is worth drawing deliberately rather than discovering. Scalars and flat lists belong in the environment, where they are easy to override per deployment and easy to inspect. Structures with repetition belong in a file, where they can be formatted, commented and reviewed as a unit. A settings model handles both without complaint — file values arrive as initialisation arguments, environment variables as their own source — so choosing the right home per setting costs nothing in code and makes both forms pleasant to work with.

Gotchas and version-specific behaviour

• Complex fields are JSON-decoded before type validation, so the error for a bare comma-separated value mentions JSON.
• mode="before" validators run ahead of the JSON step, which is what makes accepting both forms possible.
• An empty string is not valid JSON; decide explicitly whether it means an empty collection, unset, or an error.
• env_nested_delimiter populates nested models from flat keys and is generally kinder to hand-editing than JSON.
• Simple scalars — int, bool, float — are coerced directly and never go through the JSON step.
• A list[int] still needs the JSON or split step first; the element type is validated after the sequence is produced.
• Deduplicating or sorting inside a validator changes the value silently; if order or duplicates matter, validate rather than transform.

Production parity checklist

• List fields use one shared annotated type, so every list behaves identically.
• The empty-string case has an explicit, documented behaviour.
• Nested structures come from delimited keys where operators edit them, and JSON only where tooling generates them.
• Field descriptions state the accepted forms and whether order is significant.
• Parametrised tests cover the comma form, the JSON form, the empty case and a malformed input.
• No validator silently reorders or deduplicates a list whose order or multiplicity is meaningful; validate and reject instead of quietly transforming.
• Lists of objects live in a configuration file rather than in an environment variable.

Key takeaways

The rule to internalise is that complex fields go through a JSON decode before type validation, which is why a comma-separated list produces a JSON error. A mode="before" validator runs earlier still, so it is the place to accept the human-friendly form while letting generated JSON pass through untouched — defined once as an annotated alias so every list field behaves the same way.

For nested structures, choose by author rather than by taste: delimited keys give operators one plain scalar per setting with no quoting to get wrong, while a JSON blob suits values produced by tooling. Decide explicitly what an empty variable means, state the accepted forms and the significance of order in the field description, and pin all of it with four parametrised cases per field. That combination turns the most common startup error in this area into something nobody encounters. See the strict mode and type coercion topic for the wider coercion rules.