Settings from AWS Parameter Store

AWS Systems Manager Parameter Store is the cheap, IAM-scoped place to keep configuration and modest secrets — but pydantic-settings does not read it out of the box. This page adds Parameter Store as a first-class settings source so SSM parameters validate through the same model as everything else. It builds on type-safe validation with pydantic-settings alongside pydantic-settings fundamentals.

The important idea is that pydantic-settings is extensible about where values come from. Out of the box it reads init arguments, environment variables, .env files, and secret files, and it resolves them in a fixed precedence. But that source list is not closed — you can add your own source by subclassing PydanticBaseSettingsSource and slotting it into the precedence with settings_customise_sources. A Parameter Store source is exactly that: a class that fetches a path prefix from SSM and returns a plain dictionary, which the model then validates like any other input. The payoff is that SSM parameters flow through the same typed, validated model as your environment variables — same fields, same validators, same SecretStr masking — rather than being read out-of-band with raw boto3 calls scattered through the code.

The two AWS-specific concerns layered on top are decryption and access scoping. SecureString parameters are KMS-encrypted, so the source must request WithDecryption=True (and the reading identity needs kms:Decrypt), and the decrypted secret should land in a SecretStr field so it never leaks. Access is scoped by IAM to a specific path prefix and KMS key, so a workload can read only its own parameters. The rest of this page builds that source, orders it correctly beneath the environment, and covers the caching and IAM scoping that make it production-ready.

Parameter Store is a good fit for this because it is cheap and IAM-native: a standard-tier parameter costs nothing, access is governed by the same IAM the rest of your AWS workload already uses, and SecureString gives you KMS encryption for the sensitive values without a separate service. That makes it an attractive first home for a service’s configuration and its modest secrets, before you reach for the heavier, rotation-focused Secrets Manager. The integration on this page is what turns that store into a first-class part of your validated configuration rather than a set of ad-hoc API calls.

Secure implementation

# config/ssm_source.py
import boto3
from pydantic import SecretStr
from pydantic_settings import (
    BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict,
)

class SSMSource(PydanticBaseSettingsSource):
    """Reads a path prefix from SSM Parameter Store, decrypting SecureStrings."""
    def __call__(self) -> dict[str, object]:
        ssm = boto3.client("ssm")
        out: dict[str, object] = {}
        paginator = ssm.get_paginator("get_parameters_by_path")
        for page in paginator.paginate(Path="/myapp/", Recursive=True, WithDecryption=True):
            for p in page["Parameters"]:
                key = p["Name"].rsplit("/", 1)[-1].lower()
                out[key] = p["Value"]
        return out

    def get_field_value(self, field, field_name):  # required abstract hook
        return None, field_name, False


class Settings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")
    database_url: str
    api_key: SecretStr

    @classmethod
    def settings_customise_sources(cls, settings_cls, init_settings,
                                   env_settings, dotenv_settings, file_secret_settings):
        # env vars still win; SSM sits below them for parity.
        return (init_settings, env_settings, SSMSource(settings_cls), dotenv_settings)

WithDecryption=True resolves SecureString parameters; the model validates them like any other source, and environment variables still take precedence for local overrides. Secret fields are typed SecretStr.

Two parts of that code carry the design. The SSMSource.__call__ returns a flat dict mapping field names to values — it fetches the /myapp/ prefix recursively, strips each parameter name down to its last segment, lower-cases it to match the model’s field names, and returns the lot. Everything AWS-specific lives inside that method; the model itself has no idea its values came from SSM rather than the environment. The settings_customise_sources hook is where the source’s priority is set: by returning (init_settings, env_settings, SSMSource(...), dotenv_settings), it places SSM below environment variables, so a developer can still override any value with an env var locally while production reads from SSM. That ordering is deliberate — it preserves the “injected variables win” parity you want across environments.

The abstract methods deserve a note because they trip people up. PydanticBaseSettingsSource requires you to implement get_field_value, even though for a bulk-fetch source like this one the real work happens in __call__. The get_field_value hook exists for sources that resolve one field at a time; a path-prefix source that grabs everything in one API call does not need per-field resolution, so a minimal stub that returns (None, field_name, False) satisfies the interface. Understanding this split — __call__ returns the whole dict, get_field_value is the per-field hook you can stub — is what makes the boilerplate make sense rather than feeling arbitrary.

There is also a decision about when the source runs. As written, SSMSource.__call__ calls SSM every time the model is constructed, which is fine if you build the model once at startup behind a cached accessor. If your code reconstructs settings frequently, wrap the fetch in the TTL cache covered later so you are not hitting SSM on every construction. The source is just Python, so you have full control over its fetching strategy — eager at startup, lazy on first use, cached with a TTL — and choosing the right one for your access pattern is part of making the integration production-ready. For most web services the answer is eager-at-startup behind a cached accessor: the fetch happens once when the process boots, a bad or missing parameter fails the boot, and no request in the hot path ever pays the cost of an SSM API call.

Why a custom source beats scattered boto3 calls

The tempting alternative to a custom source is to call boto3 directly wherever you need a parameter — ssm.get_parameter(Name="/myapp/database_url") in the database module, another call in the cache module, and so on. It works, but it reproduces exactly the problems a settings model exists to solve. Each call is untyped, so the value is a raw string every consumer must coerce itself; each is unvalidated, so a malformed parameter fails at use rather than startup; each is a separate API call, multiplying your SSM traffic; and secrets fetched this way are plain strings with no masking. Worst of all, there is no single schema — the set of parameters the service reads is scattered across the codebase, exactly the configuration sprawl the model was meant to eliminate.

Routing SSM through a custom settings source fixes all of it at once. The parameters are fetched in one place, validated against the model’s fields, coerced to their declared types, masked if they are SecretStr, and cached together — and the field list becomes the single authoritative inventory of everything the service reads from Parameter Store. Downstream code reads settings.database_url, never touching boto3 or knowing the value came from SSM. This is the same single-boundary principle that governs environment variables, extended to a managed store: one source fetches, one model validates, and the rest of the application sees only typed, validated values.

Configuration reference

The rows split into two concerns. get_parameters_by_path, WithDecryption=True, and SecureString are the AWS mechanics — how you fetch and decrypt parameters, each with an IAM implication (scope the path, grant kms:Decrypt, use SecureString for anything sensitive). SecretStr and the custom source order are the pydantic mechanics — how the fetched values are handled once they reach the model (mask secrets, keep the environment above SSM). Reading the table this way clarifies where each responsibility lives: the source class owns the AWS side and hands the model a plain dict, and the model owns validation, masking, and precedence. Keeping that boundary clean is what lets you swap SSM for another store later by rewriting one source class, without touching the model or its fields.

The WithDecryption=True row deserves singling out because it is both easy to forget and consequential. Omit it, and SecureString parameters come back as their encrypted ciphertext rather than the plaintext value — a database_url field would receive an unusable KMS blob and, if it has a validator, fail with a confusing error about a value that looks nothing like a URL. Include it, and the source needs kms:Decrypt on the key or the whole GetParametersByPath call fails with AccessDenied. So the decryption flag ties together three things: the API call, the IAM permission, and the field’s expectation of plaintext. When a SecureString field misbehaves, the decryption flag and the KMS permission are the first two things to check.

Step-by-step deployment parity

1. Local dev — environment variables override SSM; developers need no AWS access for non-secret config.
2. CI — an OIDC role grants ssm:GetParametersByPath on the test path only.
3. Staging/Production — the pod’s role reads its own /myapp/ prefix with kms:Decrypt; the same model validates the result.

The parity here rests on two things: the same model everywhere, and the environment-above-SSM ordering. Locally, a developer sets a few env vars and never touches AWS for non-secret config, because env vars override the (absent) SSM source. In CI, a short-lived OIDC role reads a test path — no long-lived AWS keys stored in the CI platform. In production, the pod’s IAM role reads its own /myapp/ prefix and decrypts SecureString values with kms:Decrypt. In all three, the validating class is identical, so a value that constructs cleanly in CI constructs cleanly at boot in production, and the only thing that changes between them is which source and which IAM context supplied the value. Because SSM sits below the environment, an operator can still inject an override in any environment without editing Parameter Store, preserving the same precedence you rely on for plain env-var configuration.

Security boundaries & operational guardrails

• IAM scoped to the exact parameter path prefix and KMS key — no wildcards.
• Use SecureString for any sensitive parameter; never String.
• Wrap decrypted secrets in SecretStr; unwrap only at the point of use.
• Keep extra="forbid" so an unexpected parameter fails validation.
• Cache parameters in memory with a TTL to stay within SSM throughput limits.

The IAM scoping is the guardrail that limits blast radius. Grant the workload ssm:GetParametersByPath on its own /myapp/ prefix and kms:Decrypt on the one KMS key its SecureString values use — no wildcards on either. Scoped this way, a compromised workload can read only its own parameters, not the whole account’s, which is the difference between a small, contained incident and a fleet-wide secret exposure across the account. The SecureString-versus-String choice is the other security decision: String parameters are stored and returned in plaintext, so anything sensitive must be a SecureString, which is KMS-encrypted at rest and requires the explicit kms:Decrypt permission to read. Pairing SecureString at rest with SecretStr in the model closes the loop — the value is encrypted at rest in the store and masked in memory in the process — while the TTL cache keeps you inside SSM’s per-account throughput limits under load.

The IAM scoping is worth doing precisely rather than broadly, because it is easy to grant ssm:GetParametersByPath on /* and kms:Decrypt on all keys “to make it work”, and just as easy to leave that wildcard in place forever. Every wildcard widens the blast radius of a compromise from one workload’s parameters to the whole account’s. The disciplined grant names the exact path prefix the workload reads and the exact KMS key its SecureString values are encrypted under, and nothing else — so even a fully compromised process can read only what it was always allowed to read. Because the path prefix and key are stable, this scoping is a one-time setup cost that pays back every time you reason about what a service could access if it were breached.

The SecureString-versus-String decision compounds with the scoping. A String parameter is stored and returned in plaintext, visible to anyone with read access to the path, so putting a secret in a String is the SSM equivalent of committing it to a file — the encryption at rest and the kms:Decrypt gate simply do not apply. Reserve String for genuinely non-sensitive configuration (a timeout, a feature flag, a region name) and use SecureString for every credential, token, and connection string that embeds one. The model’s SecretStr typing then carries the protection through to the process, so the value is encrypted at rest in SSM and masked in memory once fetched.

Parameter Store or Secrets Manager?

Both AWS services store secrets, and choosing between them is a common decision that this integration forces you to make. Parameter Store is the cheaper, more general option: a standard-tier parameter is free, it holds configuration as easily as secrets, and SecureString gives you KMS encryption for the sensitive ones. Secrets Manager is the specialised option: it costs per secret and per API call, but it adds native, scheduled rotation (including built-in integrations for RDS and other databases) and higher throughput. The rule of thumb is to use Parameter Store for configuration and a modest set of secrets that you rotate manually or infrequently, and Secrets Manager when you need automatic rotation or are storing many high-value credentials that justify its features.

The good news for this page is that the integration pattern is identical either way. Whether the source class calls SSM get_parameters_by_path or Secrets Manager get_secret_value, it returns a plain dict that the same model validates, with the same SecretStr masking and the same source precedence. So the choice between the two stores is an operational and cost decision, not an architectural one — you can start with Parameter Store, and if a subset of secrets later needs rotation, add a Secrets Manager source alongside the SSM one and point the rotating fields at it, without changing the model. The two can coexist as two sources in the precedence tuple, each owning the parameters it is best suited to.

A concrete hybrid looks like this: put your configuration and your rarely-rotated secrets in Parameter Store under /myapp/, put the handful of database credentials that need automatic rotation in Secrets Manager, and register both as custom sources on the same model. The settings_customise_sources tuple then lists environment variables, the Secrets Manager source, and the SSM source in whatever precedence you want, and each field is populated by whichever source provides it. The model neither knows nor cares which store a given field came from — it validates them all identically. That composability is the deeper payoff of the custom-source mechanism: it is not a Parameter Store feature but a general way to teach pydantic-settings about any store, and once you have written one source you can add others cheaply — a HashiCorp Vault source, a Doppler source, or a source that reads a mounted secrets directory — each following the same shape of fetch-and-return-a-dict.

Troubleshooting

• AccessDeniedException on GetParametersByPath — the role lacks the path or kms:Decrypt; scope both.
• SecureString returned encrypted — WithDecryption=True was omitted.
• Throttled at scale — add the in-memory TTL cache. See Cache Parameter Store Values to Reduce API Calls.
• Env var ignored — that is correct; env sits above SSM in the source order.

Most of these trace to IAM or the source order. An AccessDeniedException on GetParametersByPath means the role lacks either the SSM path permission or the kms:Decrypt on the key — and because SecureString decryption needs both, the fix is usually to grant both, scoped to the exact path and key. A SecureString that comes back still encrypted is the tell that WithDecryption=True was omitted from the paginate call; add it. Throttling at scale is SSM’s per-account rate limit, solved by the in-memory TTL cache covered on its own page. And the “env var ignored” symptom is not a bug at all — the environment deliberately sits above SSM, so an env var overriding an SSM value is the precedence working as designed, which is exactly what preserves local-override parity. If you genuinely wanted SSM to win over a particular environment variable — which is unusual — you would reorder the sources in settings_customise_sources, but the default of environment-above-SSM is right for almost every case because it keeps the same precedence you rely on for plain env-var configuration and lets a developer override any value locally without touching AWS at all.

Mapping parameter names to fields

The one piece of glue the source owns is turning SSM parameter names into model field names. A parameter stored at /myapp/database_url needs to become the database_url field, so the source strips the path prefix down to the last segment and lower-cases it — p["Name"].rsplit("/", 1)[-1].lower(). This convention keeps the two namespaces aligned: parameters live under a path prefix in SSM, fields are lower-snake in Python, and the mapping is a single, predictable transformation. If your parameters use a nested path (/myapp/cache/host), you can extend the source to build a nested dict and pair it with env_nested_delimiter, mirroring how nested environment variables populate sub-models. Whatever convention you choose, keep it consistent, because a mismatch between the parameter name and the field name shows up as a missing error for a value you are sure you stored — the parameter is there in SSM, just under a name the source did not map to the expected field. A quick way to debug this class of problem is to log the keys of the dict the source returns (never the values, which may be secret) and compare them against the model’s field names — a mismatch is immediately visible.

Because extra="forbid" is on, the mapping also guards against stray parameters: a parameter under /myapp/ that does not correspond to a declared field is rejected as an unexpected key, which catches a typo in a parameter name or a leftover parameter from a field that was removed but whose SSM entry was never cleaned up. That strictness is what keeps the SSM prefix and the model schema in sync — the model’s field list and the store’s parameter set must agree, and forbid makes any drift between them a loud startup error rather than a silently ignored value.

Key takeaways

The invariant: Parameter Store is just another validated source below the environment, with SecureString for secrets and SecretStr in the model. Configuration and modest secrets share one cheap, IAM-scoped store. The mechanism is a custom PydanticBaseSettingsSource that fetches a path prefix and returns a dict, slotted below environment variables in settings_customise_sources so local overrides still win — and the model validates SSM parameters exactly as it validates env vars, with the same types, validators, and masking.

The reason to do this rather than call boto3 directly is the same reason to use a settings model at all: one validated boundary instead of scattered, untyped reads. Scope the IAM to the exact path and KMS key, use SecureString for anything sensitive and SecretStr for it in the model, cache with a TTL to respect SSM’s limits, and keep extra="forbid" so the store and the schema stay in sync. Do that, and Parameter Store becomes not a special integration but simply another source feeding the one typed configuration object your whole application depends on.