Secret Types & Redaction

The incident starts with a connection error. A database goes unreachable, the driver raises, the exception message includes the connection string it tried, and the error tracker faithfully records the whole thing — password included — into a system that a dozen people can read and that retains data for ninety days. Nothing was misconfigured, no rule was broken, and a credential now needs rotating. Redaction is the discipline that closes this class of leak, and it starts with using the right type for the value rather than remembering to be careful. This page sits under type-safe validation because the type system is where the protection belongs.

The mental model to adopt is that a secret has a disclosure surface: the set of places a value can end up without anybody intending it. For a running Python service that surface is larger than it first appears — application logs, the framework’s own request logs, captured tracebacks, an error tracker’s local-variable capture, metrics labels, a debug or health endpoint, an admin interface, a crash dump, and the terminal of whoever last ran the service by hand. SecretStr covers a large fraction of that surface cheaply, but knowing which fraction is what separates a service that is genuinely protected from one that merely looks protected in code review.

Where redaction belongs in the pipeline

Redaction belongs to the schema, not to the output layer, and the reason is ownership: the schema is the only component that knows which values are sensitive. Redaction is not a logging feature. Teams that treat it as one end up with a log-formatting filter that scrubs anything matching a regular expression, which fails in both directions: it misses secrets in shapes the pattern does not anticipate, and it mangles legitimate output that happens to match. The durable placement is at the type level, in the settings model, so the value arrives everywhere already knowing how it should render. A logging filter can then exist as a second line of defence rather than the only one, and it can be narrow enough to be correct.

Placing it in the model also puts it before every consumer. The framework adapters receive a wrapped value and unwrap at one line; a background task receives the same wrapped value; a management command that prints its configuration for debugging prints masked placeholders without its author having thought about it. That default-safe behaviour is the whole point — protection that depends on every future author remembering a rule is protection with a known expiry date, whereas protection that comes from the type is inherited automatically by code nobody has written yet.

The complement to masking is not holding the value at all where you do not need it. A field that only some processes require should be absent from the others rather than injected everywhere and masked. Combining scope with masking is what makes an incident survivable: masking limits where a held secret can appear, and scoping limits which processes hold it. The rotation practices covered elsewhere on the site assume both, because a rotation is only fast when you know the complete list of processes that had the old value.

Secure implementation

Wrap at the schema, unwrap at exactly one boundary, and render a safe summary anywhere a human needs to see what is configured.

# config.py — secret types and a safe summary
from pydantic import SecretStr, PostgresDsn, field_serializer
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")

    environment: str = "local"
    database_url: PostgresDsn                 # parsed: host/port/user/password are components
    api_token: SecretStr                      # masked in repr, str, f-strings and dumps
    signing_key: SecretStr

    @property
    def database_summary(self) -> str:
        """A loggable description of the database without the password."""
        return f"{self.database_url.host}:{self.database_url.port}/{self.database_url.path}"

    @field_serializer("database_url")
    def _redact_dsn(self, value: PostgresDsn) -> str:
        # a DSN is a str-like type, so serialising it would expose the password
        return str(value).replace(value.password or "", "***") if value.password else str(value)


settings = Settings()
print(settings)               # api_token=SecretStr('**********') — safe by default
print(settings.database_summary)   # db:5432/app — informative, no credential

# db.py — the single unwrap point, at the boundary that needs a raw string
import psycopg

def connect(settings: Settings) -> psycopg.Connection:
    # the ONE place the raw value exists; nothing here is logged or re-raised with the URL
    return psycopg.connect(str(settings.database_url))

The field_serializer is the part most implementations miss. SecretStr masks itself, but a parsed DSN type does not — it is a structured string type whose str() includes the password, so a model dump that includes database_url writes the credential into whatever consumed the dump. Adding an explicit serialiser makes the model’s serialised form safe as well as its repr, which matters because structured logging libraries and error trackers both prefer to serialise objects rather than print them. If you would rather not maintain the serialiser, the alternative is to keep the URL’s components as separate fields with the password as a SecretStr and assemble the DSN at the connection boundary.

The property is also the right place to put any other detail operators routinely need — a redacted account identifier, a region, a schema name — because a property is computed from already-validated fields and therefore cannot drift from what the process is really using.

Note what database_summary is for. Operators legitimately need to know which database a process is talking to, and if the safe way to find that out does not exist, somebody will log the whole URL to answer the question. Providing a property that renders host, port and database name — and nothing else — removes the motive for the unsafe alternative. The same reasoning applies to API tokens: log the key name and the last four characters if you must correlate, never the token.

Mechanism reference

SecretBytes earns its own row because binary credentials fail differently. A signing key supplied as base64 in an environment variable has to be decoded somewhere, and a decode that happens in application code tends to leave the decoded bytes in a local variable with a helpful name. Declaring the field as SecretBytes moves the decode into the model, keeps the result wrapped, and makes the mistake of comparing an encoded value against a decoded one a validation error at boot rather than a signature mismatch under load — which is a far more legible failure than the alternative.

The bottom two rows are the ones to weigh carefully. A logging filter is genuinely useful as a backstop — it catches the value a third-party library logs from inside its own code, which no type of yours can influence — but it is worthless as a primary control because it depends on predicting the shape of every leak. Scoping, by contrast, is the strongest control on the list and the least used: a process that never receives the value cannot disclose it under any circumstances, and no code review is required to keep that true.

Deployment parity, step by step

1. Type every credential field as SecretStr at the moment you add it. Retrofitting is what leaves one field plain, and the plain one is always the one that leaks.
2. Grep for get_secret_value and count the call sites. The number should be small, stable, and equal to the number of clients that need raw strings.
3. Add a safe summary property for anything operators will ask about. Database, message broker, object store — each gets a host-and-name renderer.
4. Configure the error tracker to scrub local variables. Most capture frame locals by default; the setting that disables it, or the denylist of variable names, belongs in the same commit as the first SecretStr.
5. Serialise a settings dump in CI and assert no secret appears. One test, run against a populated environment, catches a field somebody forgot to wrap.
6. Repeat for every process role. The worker’s settings model gets the same treatment as the web tier’s; leaks do not respect service boundaries.

Step 2 is more diagnostic than it looks. The count of get_secret_value call sites is a direct measure of how much of the codebase can leak, and its trend over time tells you whether the discipline is holding. A repository where that number is four — a database driver, an HTTP client, a queue connection, a signing helper — is one where a reviewer can check every dangerous site in a minute. A repository where it is forty has effectively reverted to plain strings with extra ceremony, and the fix is not more review but a helper: a small function per client that takes the settings object, unwraps internally, and returns a configured client, so callers never see the raw value at all.

Step 5 deserves emphasis because it is the only step that scales. Reviews catch the first few unwrapped fields and then attention drifts; a test that dumps the model and asserts the known secret values do not appear in the output holds the line permanently. It is easy to write — populate the environment with recognisable sentinel values, serialise, assert none of the sentinels are present in the resulting string — and it fails the moment somebody adds a plain str field for a token.

The secrets that never reach the settings model

A settings model covers the credentials your service holds. It does not cover the credentials that merely pass through it, and in most production incidents those are the ones that leak. An inbound Authorization header is a secret belonging to a caller. A webhook signature is a secret shared with a provider. A password field in a form submission is a user’s secret, transiting your process on its way to a hashing function. None of these are ever assigned to a settings field, so no amount of care in config.py affects them at all.

The mechanisms that cover them look different. Request logging middleware needs an explicit denylist of header names — authorization, cookie, x-api-key, proxy-authorization — applied before the log record is created, not after. Form and JSON bodies need field-name filtering, which most frameworks and error trackers support natively but rarely enable by default. And the highest-value control is a policy one: do not log request bodies in production at all, log a shape summary instead. Every team that turns body logging on “temporarily to debug something” discovers later that it was on for months.

Third-party libraries are the other category outside your reach. A database driver that embeds the connection string in its exception message, an HTTP client that logs full request URLs including query-string tokens, an SDK that prints its own configuration at DEBUG — none of these consult your types. The practical defences are to catch and re-raise at the client boundary with a message you control, to keep production log levels above the point where libraries become chatty, and to prefer passing credentials in headers rather than query strings so a URL-logging library has nothing to disclose. This is also the one place a formatted-output filter earns its keep: it is the only control that can act on text produced entirely inside somebody else’s code.

Security boundaries and operational guardrails

• SecretStr is not encryption. The value is a plain string in process memory. Anyone with the ability to read the process — a debugger, a core dump, a malicious dependency — has the secret regardless.
• Never build a URL by f-string from an unwrapped secret. The resulting str has no protection and tends to end up in a client’s error message.
• Turn off local-variable capture in the error tracker, or denylist the names. This is the single highest-value setting change on the list.
• Do not log at DEBUG in production with a library that dumps request bodies. Authentication headers are secrets that never went through your settings model.
• Treat the health endpoint as public. Report the environment name and the database host, never the credential, even behind an internal network.
• Rotate on suspicion, not on proof. If a secret may have been rendered into a retained log, the cheap action is a rotation and the expensive action is an investigation.

The rotation guardrail is the one that changes team behaviour most, so it is worth making concrete. When somebody asks “did that credential actually end up in the log?”, the honest answer is usually “probably not, but proving it would mean searching several retention systems, some of which are indexed by a third party”. That search costs hours and produces a probabilistic answer, while a rotation costs minutes if the credential is stored centrally and the application reads it at boot. Designing for cheap rotation therefore pays for itself in decision-making speed long before it pays for itself in security: teams that can rotate in minutes simply rotate, and stop spending afternoons reasoning about exposure windows.

The health-endpoint rule needs the same bluntness. “Internal network” is not a security boundary in any environment that has an ingress controller, a service mesh sidecar, a debugging proxy or a developer with port-forward rights — which is to say, in any environment. Treat every endpoint the application serves as though it will be reachable by someone you did not anticipate, and the question of what a health check may report answers itself: an environment name, a build identifier, a database host, a set of boolean readiness flags. Nothing that would be useful to an attacker who reached it and nothing that requires rotating if it were indexed.

Troubleshooting

A secret appears in the error tracker despite SecretStr. Look at the frame locals, not the settings object. A function that called get_secret_value() and then raised puts the raw string in that frame, and trackers capture locals by default. Either disable local capture, add the variable name to the tracker’s denylist, or restructure so the unwrapped value never lives in a named local that outlives the call.

model_dump() produces SecretStr('**********') where a real value was expected. That is the default and it is correct. Code that genuinely needs raw values — writing a config file for a subprocess, for example — should unwrap explicit fields at that boundary rather than asking the whole model to serialise in an unsafe mode, which affects every field including the ones nobody thought about.

The password is in the logs even though the field is a DSN type. Parsed URL types render fully under str(). Add a field_serializer that masks the password component, and use a summary property for anything you actually want to log.

A subprocess receives the secret on its command line. Command lines are visible in the process table to any local user. Pass the value through the environment or on standard input instead, and never assemble a command string containing an unwrapped credential.

A masked value reaches the client and authentication fails. Somebody passed the wrapper where a string was expected and the placeholder was sent as the credential. The signature is an authentication error whose logged payload shows a row of asterisks, which is at least a self-diagnosing failure. It happens most often when a helper is refactored to take the settings object instead of a string and one call site is missed, and it is prevented by typing the client helper’s parameter as SecretStr so a plain string is a type error rather than a silent mismatch.

A field was wrapped but the value is still readable in an old log. Wrapping a field changes the future, not the past. Anything already written to a retained log stays there until retention expires, so the wrapping commit should be accompanied by a rotation of the affected credential and, where the log system supports it, a targeted deletion. Treat the type change as a fix for recurrence and the rotation as the fix for the incident; doing only the first leaves a live credential sitting in searchable storage.

Tests fail comparing a SecretStr to a string. The wrapper does not compare equal to a plain string, which is deliberate. Compare get_secret_value() in the one test that verifies unwrapping, and compare the masked form everywhere else so a failure message never renders the credential into CI output.

Key takeaways

The invariant this topic enforces is that a credential renders safely by default and unsafely only on purpose. Typing the field as SecretStr makes every accidental path — printing, logging, dumping, most tracebacks — produce a placeholder, and it concentrates the dangerous act into a single greppable call at the client boundary. Parsed URL types need one extra step, a masking serialiser, because their str() includes the password; without it, a model dump undoes the protection the rest of the schema provides.

The parts the type cannot do are worth holding in mind as clearly as the parts it can. It does not protect memory, it does not follow the value once you unwrap it, and it has no influence over what a third-party library logs from inside its own code. Those gaps are closed by different tools: scrubbing frame locals in the error tracker, scoping fields so a process never holds a secret it does not use, keeping a safe summary property available so nobody needs to log the whole URL, and rotating quickly when a value may have been rendered somewhere retained. Together they turn secret disclosure from an ever-present risk into a small set of boundaries you can point at.