Cache Parameter Store values to reduce API calls

SSM Parameter Store has per-account throughput limits, and a process that re-reads its parameters on every request will hit ThrottlingException under load. A short-TTL in-memory cache fixes it without introducing staleness. This page adds caching to the custom source, extending Settings from AWS Parameter Store.

The caching problem has two failure modes that pull in opposite directions, and a TTL cache is the balance between them. On one side is fetching too often — reconstructing the settings on every request, which makes an SSM call every time and burns through the account’s rate limit until reads start failing on throttling. On the other side is caching forever — reading the parameters once at import and never again, which never throttles but also never notices a rotated SecureString or a changed value. Neither extreme is acceptable: the first is fragile under load, the second serves stale configuration. A cache with a time-to-live sits between them, serving from memory within the interval and refetching once the interval elapses, so it caps API calls at one per interval per process while still picking up changes within one TTL — the resilience of caching without the staleness of caching forever, and the freshness of re-reading without the throttling of re-reading on every request.

The TTL is a single dial you tune against your rotation cadence. Set it short and you notice changes quickly at the cost of more API calls; set it long and you make fewer calls at the cost of a longer staleness window during which a changed value goes unnoticed. The right value is bounded above by how quickly you need a rotated secret to take effect and below by the rate limit you must stay under — for most services a few minutes is the sweet spot, because it makes SSM calls rare while keeping any configuration change visible within minutes. If you find yourself wanting a TTL of seconds to react quickly to changes, that is usually a sign you have outgrown polling and would be better served by a store that pushes updates, or by triggering a cache invalidation on rotation rather than waiting for the TTL to lapse. The rest of this page builds that cache inside the custom SSM source and covers the details, chiefly using a monotonic clock for the timer and keeping the decrypted plaintext in process memory only.

Problem 1: re-reading on every settings construction

# ANTI-PATTERN: builds Settings() per request, calling SSM each time
def handler(event):
    return Settings()      # fresh get_parameters_by_path on every invocation

Each Settings() triggers a fresh SSM round-trip, throttling under traffic. Constructing the settings inside a request handler means every request re-reads the entire parameter prefix from SSM — a network call, and a charge against the rate limit, on the hot path of every invocation. At low traffic it merely wastes latency; at scale it exhausts the per-account throughput and SSM starts returning ThrottlingException, at which point requests fail to read configuration for a reason that has nothing to do with the configuration being wrong. This is the same anti-pattern as reading os.environ on every request, except each read is a real API call rather than a cheap dictionary lookup, so the cost is far higher.

The right structural fix pairs the cache with constructing the settings once rather than per request. In most web frameworks you build the settings object at startup, or lazily on first use behind a cached accessor, and inject it into request handlers — so a handler receives the already-built object without triggering any fetch. The TTL cache then handles the one remaining reason to refetch: picking up a value that changed after startup. Together, startup construction and a TTL cache mean the steady-state request path makes zero SSM calls, while the process still notices a rotation within one TTL. Constructing per request, by contrast, defeats even a cache’s benefit if the cache is not shared — which is exactly why the cache in this page is class-level, shared across every construction of the source within the process rather than per-instance.

Problem 2: caching forever

# ANTI-PATTERN: module-level read, never refreshed
PARAMS = boto3.client("ssm").get_parameters_by_path(Path="/myapp/")  # stale after update

Parameters that change (a rotated SecureString) are never picked up. A module-level read at import time is the opposite over-correction: it makes exactly one SSM call for the life of the process, so it never throttles — but it also freezes the configuration at whatever it was when the module first loaded. When a secret is rotated in Parameter Store, or an operator updates a value, the running process keeps using the old one indefinitely, because nothing ever refetches. This is especially dangerous with secrets: the whole point of rotating a credential is that the old one is retired, and a process caching it forever will keep presenting the revoked, no-longer-valid credential until it happens to restart, turning a routine, scheduled rotation into an outage for every process that cached the credential forever. The failure is also confusing to diagnose, because the credential is correct in Parameter Store and only wrong in the running process’s frozen copy.

Secure implementation

# config/ssm_cache.py
import time
import boto3
from pydantic import SecretStr
from pydantic_settings import PydanticBaseSettingsSource

class CachedSSMSource(PydanticBaseSettingsSource):
    _cache: dict[str, object] = {}
    _loaded_at: float = 0.0
    TTL = 300                                  # seconds

    def __call__(self) -> dict[str, object]:
        now = time.monotonic()
        if self._cache and now - self._loaded_at < self.TTL:
            return self._cache                 # serve from cache within TTL
        ssm = boto3.client("ssm")
        fresh: dict[str, object] = {}
        for page in ssm.get_paginator("get_parameters_by_path").paginate(
            Path="/myapp/", Recursive=True, WithDecryption=True,
        ):
            for p in page["Parameters"]:
                fresh[p["Name"].rsplit("/", 1)[-1].lower()] = p["Value"]
        type(self)._cache, type(self)._loaded_at = fresh, now
        return fresh

    def get_field_value(self, field, field_name):
        return None, field_name, False

The TTL caps SSM calls at one per interval per process, while still refreshing rotated values. Decrypted secrets become SecretStr once they reach the model.

The logic is simple and worth reading closely. On each call, the source checks whether it has a cached result that is still within its TTL; if so it returns the cache without touching SSM, and if not it refetches, stores the fresh result with a new timestamp, and returns it. That single check is what bounds both failure modes: within the TTL, no matter how many times the settings are constructed, there is at most one SSM call; and once the TTL elapses, the next construction refetches, so a rotated value is guaranteed to be picked up within one TTL interval of the change. The cache is class-level so it is shared across instances of the source within a process, and it uses time.monotonic() for the timestamp so a system clock adjustment cannot accidentally extend or shrink the window.

There is a subtlety in how a cache interacts with rotation that is worth spelling out, because it is the same seam covered in the secrets-rotation material: the TTL bounds the staleness window, and that window must fit inside the rotation’s overlap window. When a secret rotates, the old value typically stays valid for a grace period while the new one propagates; if your cache TTL is shorter than that overlap, a cached old value always expires and refetches before the old credential is actually revoked, so no request ever presents a dead credential. If the TTL were longer than the overlap, a process could still be serving a cached credential after it was revoked. Sizing the TTL comfortably inside the rotation overlap is what makes caching and rotation coexist safely — the cache never holds a value past the point where it stops working, so the two mechanisms reinforce each other rather than fighting.

For failure handling, decide what the source should do if a refetch fails when the TTL has expired. The conservative choice is to keep serving the last good cache for a bounded grace period rather than failing the construction outright, so a brief SSM blip does not take the process down — the values are slightly staler than the TTL intended but the service keeps running and serving, and the next successful fetch catches up. The stricter choice is to fail, which is right if serving a stale value is more dangerous than being unavailable. Either is defensible; the point is to choose the behaviour deliberately rather than letting an unhandled exception on a transient, self-healing SSM error crash an otherwise healthy process.

Gotchas & version-specific behaviour

• Use time.monotonic() so clock changes cannot extend the TTL.
• Class-level cache is per process; multi-worker servers each cache independently.
• Keep the TTL below any rotation interval so rotated SecureString values are picked up promptly.
• The cache holds plaintext in memory only — never persist it.

The monotonic-clock detail matters more than it looks. time.monotonic() returns a value that only ever increases and is immune to system clock adjustments, whereas time.time() can jump backward or forward when NTP corrects the clock — and a backward jump could make a cached value look fresh far longer than the TTL intended. For a cache that guards how long a rotated secret can linger, that is a real bug, so always compute the TTL from a monotonic source. The per-process-cache point is the operational reality to keep in mind: a class-level cache lives in one process, so a server with multiple workers has one independent cache per worker, and each refetches on its own schedule and its own timer. That is usually fine — it just means the effective refetch rate is per-worker — but it is why the TTL, not a shared cache, is what bounds the account-wide call rate.

Production parity checklist

• TTL caps SSM calls and stays below the rotation interval.
• Cache lives in memory only; nothing written to disk.
• WithDecryption=True with scoped kms:Decrypt.
• Secrets wrapped in SecretStr in the model.
• Throughput stays within SSM limits under peak load.

The load-bearing pair is the first two: the TTL both caps SSM calls and stays below the rotation interval. Those two constraints define the acceptable TTL window — long enough that you stay under the rate limit at your traffic, short enough that a rotated secret is picked up before the old one causes problems. If your credentials rotate hourly, a five-minute TTL leaves a comfortable margin on both sides; if they rotate every few minutes, the TTL has to be shorter still, which raises the call rate and is a signal you might want a store with push-based updates instead of polling. The memory-only rule is the security guardrail that rides alongside: the cache holds decrypted plaintext, so persisting it to disk would undo the encryption-at-rest that SecureString provides and put the decrypted secret somewhere a backup or a shared volume could expose it — keep it in process memory, masked as SecretStr in the model, and let it vanish entirely when the process exits.

Key takeaways

A monotonic-based TTL cache inside the SSM source removes throttling without serving stale secrets. It threads the needle between the two failure modes: fetching on every request, which exhausts the SSM rate limit, and caching forever, which never picks up a rotated value. The cache serves from memory within the TTL — capping API calls no matter how often the settings are constructed — and refetches once the interval elapses, so a change is picked up within one TTL. The TTL is the single dial you tune: bounded above by how quickly a rotated secret must take effect, and below by the account rate limit you must stay under, with a few minutes the usual sweet spot.

Two details keep it correct and safe. Compute the TTL from time.monotonic() so a clock adjustment cannot extend the staleness window, and keep the plaintext cache in memory only, masked as SecretStr in the model, so it never lands on disk to undo the encryption SecureString provides. Remember the cache is per process, so the TTL — not a shared store — is what bounds the account-wide call rate across a multi-worker fleet. For the base source it caches, see Load Pydantic Settings from AWS Parameter Store.