Custom Validators & Constraints

Basic types reject "abc" where an integer is expected, but they happily accept "arn:aws:s3:::" as a string even though it is a malformed ARN. Domain validation is the difference between configuration that is typed and configuration that is correct. This page encodes domain rules into the settings model so a bad value can never reach business logic.

Custom validators extend the settings fundamentals within the type-safe validation section: where field types stop, validators pick up and carry the rule the rest of the way.

The gap they fill is the space between “this is a string” and “this is a valid string for its purpose”. A type system can tell you queue_arn holds text; only a domain rule can tell you that text is a well-formed ARN pointing at a real service in a twelve-digit account. That distinction matters because the alternative — checking the format in the one code path that happens to use the value — means the check runs late, runs once, and is easy to forget in the second code path that also uses it. Pushing the rule into the model runs it at construction, everywhere, before any business logic sees the value, so a malformed ARN cannot reach the code that would try to publish to a queue that does not exist.

There is a ladder of tools for this, and most of the craft is picking the lightest rung that expresses the rule. A concrete type handles “is it an integer?”; a Field() constraint handles “is it in range or under a length?”; a @field_validator handles “does it match this format?”; and a @model_validator handles “are these two fields consistent with each other?”. Reaching too high wastes clarity — a five-line validator that reimplements ge=1 is harder to read than the constraint — while reaching too low lets bad values through. This page walks each rung and the security concerns, chiefly regex safety, that come with hand-written format checks.

It is worth being concrete about what “domain validation” buys you, because it is easy to dismiss as gold-plating. Consider a database_url. Typed as str, it accepts "localhost", "http://example.com", and an empty string — none of which is a usable database DSN, all of which construct without complaint, and each of which fails later with a driver error that says nothing about configuration. A validator that requires a postgresql:// scheme and sslmode=require turns every one of those into a startup ValidationError naming database_url, so the misconfiguration is caught at the boundary with a message that tells the operator exactly what is wrong. The validator is a handful of lines; the incidents it prevents are the kind that page someone at 2 a.m. and take an hour to trace back to a malformed connection string. That asymmetry — a cheap, one-time rule against an expensive, recurring failure prevented — is the entire case for domain validation captured in one example, and it repeats for every ARN, URL, key prefix, and connection string your service reads.

Secure implementation

# config/validated.py
import re
from pydantic import Field, field_validator, model_validator, SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict

ARN_RE = re.compile(r"^arn:aws:[a-z0-9-]+:[a-z0-9-]*:\d{12}:.+$")  # anchored, no nested quantifiers


class ResourceSettings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")

    queue_arn: str
    callback_url: str
    signing_key: SecretStr
    require_tls: bool = True

    @field_validator("queue_arn")
    @classmethod
    def valid_arn(cls, v: str) -> str:
        if not ARN_RE.match(v):
            raise ValueError("queue_arn is not a well-formed AWS ARN")
        return v

    @field_validator("callback_url")
    @classmethod
    def https_only(cls, v: str) -> str:
        if not v.startswith("https://"):
            raise ValueError("callback_url must use https")
        return v

    @model_validator(mode="after")
    def tls_consistency(self) -> "ResourceSettings":
        if self.require_tls and not self.callback_url.startswith("https://"):
            raise ValueError("require_tls is set but callback_url is not https")
        return self

Field-level validators check one value; the model validator enforces relationships between fields. Both run at construction, so the model cannot be built with an invalid combination.

Two properties of that code are worth making explicit. First, a @field_validator must be a @classmethod in pydantic v2 — the decorator does not make it one implicitly, and omitting it produces a confusing signature error where the validator silently fails to register. Second, a field validator returns the value: whatever it returns becomes the field’s value, so a validator can normalise as well as check (strip whitespace, lower-case a scheme, canonicalise a URL) by returning the cleaned version. The raise ValueError(...) path is what rejects; the return v path is what accepts, and the message you pass to ValueError is what an operator reads in the resulting ValidationError, so write it as an instruction (“queue_arn is not a well-formed AWS ARN”) rather than a code-level assertion.

The mode argument decides when a validator runs relative to type coercion. mode="after" — the default — runs on the already-coerced, correctly-typed value, which is where format checks belong because you can trust the type. mode="before" runs on the raw input before coercion, which is the right place to normalise or transform: splitting a comma-separated string into a list, mapping an empty string to None, or upper-casing a level name so the coercion that follows sees clean input. Choosing the mode wrongly is a common source of validators that “don’t run” — a normaliser placed in after mode never sees the raw string it was meant to clean. As a rule, transform in before, validate in after. A useful pattern combines both: a before validator normalises the raw input (trim, lower-case a scheme, split a delimited string) and an after validator or a Field constraint then checks the cleaned, typed value. Keeping the two responsibilities in their proper modes means each validator does one job on the shape of data it expects, rather than one tangled method trying to normalise and validate a value whose type it cannot rely on.

The @model_validator(mode="after") is the tool for rules that span fields, and it is fundamentally different from a field validator: it receives the whole constructed model and returns it, so it can compare fields against one another. The tls_consistency example — requiring callback_url to be HTTPS whenever require_tls is set — is a rule no single-field validator can express, because neither field is wrong on its own; only their combination is invalid. Whenever a rule reads “if this field is X then that field must be Y”, it belongs in a model validator, and putting it there keeps invalid combinations out just as field validators keep invalid values out.

Configuration reference

The ordering of that table is also a preference order. Start at the top and use the most declarative tool that expresses the rule: a Field() constraint for a numeric bound or a length limit is a single argument, needs no method, and shows its intent at the field definition. Drop to a Field(pattern=...) for a simple anchored regex where the rule is genuinely “match this shape”. Escalate to a @field_validator only when the rule needs real Python — cross-referencing a lookup, producing a tailored error message, or normalising the value — and to a @model_validator only when the rule couples fields. The reason to prefer the lighter tool is not dogma: a Field(ge=1, le=65535) is faster, more readable, and less error-prone than a validator that reimplements the same bound, and it keeps the constraint visible at the field rather than buried in a method below.

A useful way to read the “Runs” column: constraints and pattern are checked as part of parsing, @field_validator runs after its field is parsed, and @model_validator(mode="after") runs last, once every field is populated. That ordering is why a model validator can trust that all the fields it compares are already individually valid — the field-level checks have all passed by the time it runs — so it only has to worry about their relationships, not their individual correctness.

Reusable validators as Annotated types

When the same rule recurs across fields or models, copying the validator is a maintenance trap: three copies of “must be an https URL” drift apart the first time someone fixes one. Lift the rule into a named Annotated type and it travels with the type instead. Because AfterValidator, BeforeValidator, Field, and Strict all compose inside Annotated, a single alias can carry a field’s entire rule, and every field declared with that alias inherits it identically.

from typing import Annotated
from pydantic import AfterValidator

def _https(v: str) -> str:
    if not v.startswith("https://"):
        raise ValueError("must be an https:// URL")
    return v

HttpsUrl = Annotated[str, AfterValidator(_https)]

class WebhookSettings(BaseSettings):
    callback_url: HttpsUrl        # the rule lives in the type
    events_url: HttpsUrl          # same rule, no duplication

This does three things at once. It removes duplication, so the rule has exactly one place to change. It makes the rule testable in isolation — _https is an ordinary function you can unit-test without constructing a settings model. And it makes the intent visible at the field: a reviewer reading callback_url: HttpsUrl knows the constraint without opening a validator method. As a configuration grows, this is how it stays consistent — domain constraints become a small vocabulary of named types (HttpsUrl, Arn, Port, NonEmptyStr) that fields are built from, rather than a scattering of near-identical validators copied from field to field. The same technique also composes with Field constraints and Strict, so a named type can bundle a format check, a length bound, and a strictness setting together — Port = Annotated[int, Field(ge=1, le=65535)] reads as a single reusable concept, and every field typed Port is bounded identically without repeating the bounds.

Field constraints cover more than you think

Before writing any validator, check whether a Field() constraint already expresses the rule, because pydantic ships a broad set of them. Numeric fields have ge, gt, le, lt for bounds and multiple_of for step constraints; string fields have min_length, max_length, and pattern; collections have min_length and max_length on the container. A surprising fraction of “domain rules” are really just bounds — a worker count between 1 and 64, a timeout of at least one second, a name under 200 characters — and all of those are one Field() argument rather than a method. Using the constraint keeps the rule declarative and visible at the field, and it produces a clear, standard error message without you writing one.

The constraints also compose with everything else inside Annotated, so a field can carry a type, a strictness setting, and several bounds at once: Annotated[int, Field(ge=1, le=64)] is a bounded integer you can name and reuse. The habit worth building is to exhaust the declarative options first and only drop to a @field_validator when the rule genuinely needs logic a constraint cannot express — a format check, a normalisation, a lookup, or a tailored message. That ordering keeps most of your validation readable and pushes hand-written code to the minority of rules that truly require it.

Deployment parity: local to production

1. Local dev — invalid ARNs or non-TLS URLs fail at construction on the developer’s machine.
2. CI — a fixture that builds the model with representative values guards every validator.
3. Staging/Production — the same validators run at boot; a malformed injected value stops the rollout, not a request.

The parity here is stronger than for plain typed fields because domain rules encode assumptions that are easy to violate silently. A str field for queue_arn accepts a malformed ARN in every environment equally — the bug is uniform but present everywhere. A validated queue_arn rejects the malformed value at construction, so if staging’s ARN is wrong, the staging deploy fails to boot immediately and the bad value never reaches a code path that would otherwise fail obscurely at first use, long after the deploy looked successful. The CI fixture that builds the model with representative values is what makes this a pre-deploy guarantee rather than a boot-time surprise: it proves the validators accept the shapes your real environments carry, so a validator that is accidentally too strict (rejecting a legitimate value that a real environment actually carries) is caught on the pull request, not discovered during a failed production rollout.

Security boundaries & guardrails

• Anchor every regex (^...$) and avoid nested quantifiers to prevent ReDoS.
• Prefer explicit prefix/scheme checks over broad patterns for ARNs and URLs.
• Keep extra="forbid" so unvalidated extra keys cannot slip through.
• Validate, never sanitize-and-continue — reject bad config rather than silently repairing it.
• Wrap signing keys and tokens in SecretStr even inside validated models.

Regular expressions are where a validator can turn from a guard into a liability, so they deserve the most care. Catastrophic backtracking — the mechanism behind a ReDoS, or regular-expression denial of service — happens when a pattern with nested or overlapping quantifiers ((a+)+, (.*)*, (\d+)*$) is given an input that almost matches: the engine tries an exponential number of ways to split the string before concluding it fails, and a modest input can hang the process for seconds or minutes. For configuration this is usually a lower risk than for user-facing input, because config values are set by operators rather than attackers, but it is not zero — a config file, a fetched remote fragment, or a value an attacker can influence turns a bad pattern into a hang. The defences are simple: anchor the pattern with ^ and $ so the engine is not searching for a match at every offset, and never nest quantifiers.

The stronger move is to prefer an explicit check over a clever regex entirely. For an ARN or a URL, a few str operations — v.startswith("https://"), v.count(":") == 5, a split on : with a length check — are faster, impossible to make backtrack, and far easier to read than a single monster pattern that tries to validate the whole structure at once. A validator that says “starts with arn:aws:, has six colon-separated parts, and a twelve-digit account segment” as plain Python is both safer and clearer than the equivalent regex. Reserve regex for the parts that genuinely need pattern matching, keep those patterns anchored and quantifier-flat, and let ordinary string logic handle the structure.

The “validate, never sanitize-and-continue” guardrail is a philosophy as much as a rule. A validator’s job is to reject a bad value, not to quietly repair it into something plausible — because a silent repair hides the fact that the input was wrong, and the “fixed” value may not be what anyone intended. If callback_url arrives without a scheme, raising “must use https” tells the operator to fix their configuration; silently prepending https:// guesses at their intent and may point the webhook somewhere unexpected. Normalisation that is unambiguous (trimming whitespace, lower-casing a scheme) is fine and belongs in a before validator; repair that guesses at meaning is not. When in doubt, reject with a clear message and let a human decide what the correct value should be, rather than guessing on their behalf and shipping the guess.

Troubleshooting

• ValidationError lists multiple fields — pydantic collects all failures; fix them together.

The multiple-errors behaviour is a feature you learn to appreciate. Pydantic does not stop at the first bad field; it runs every field validator it can and collects all the failures into one ValidationError, so an operator fixing a broken configuration sees the complete list — the malformed ARN and the non-TLS URL and the missing key — and fixes them in a single edit rather than one failed boot at a time. When you read one of these errors, read the whole list before changing anything, because a second failure below the first is common and fixing them together saves a round trip.

• Validator not running — missing @classmethod under @field_validator, or the method references self instead of cls.
• Regex hangs on long input — catastrophic backtracking; re-anchor and simplify the pattern. See Validators for AWS ARNs and URLs.
• Cross-field rule ignored — it belongs in @model_validator(mode="after"), not a field validator.

The “validator not running” symptom is the most common and has two usual causes, both mechanical. The first is a missing @classmethod beneath @field_validator — in pydantic v2 the method must be an explicit classmethod, and without it the validator either fails to register or raises a signature error that does not obviously point at the missing decorator. Add @classmethod as a reflex whenever you write @field_validator. The second is a mode mismatch: a normaliser written to clean raw input but left in the default after mode never sees the raw string, because coercion has already run by the time it fires. If a validator seems inert, check the decorator stack and the mode before suspecting anything deeper.

The cross-field symptom is really a scope error. A rule like “if require_tls then callback_url must be https” cannot live in a field validator because a field validator sees only its own field — the callback_url validator has no visibility into require_tls, and vice versa. Moving the rule to a @model_validator(mode="after"), which receives the whole constructed model, is the fix, and the tell that you need one is any rule whose statement mentions two field names. When you catch yourself trying to reference another field from inside a field validator, that is the signal to promote the rule to the model level. Model validators also run last, after every field is individually valid, so they are the right place for any check that only makes sense once the whole configuration is assembled — a consistency rule, a mutual-exclusion rule, or a “at least one of these must be set” rule that no single field can enforce alone.

Key takeaways

The invariant: every domain rule lives inside the model as a validator, runs at construction, and rejects rather than repairs. Configuration that builds is configuration that is correct. That sentence is the whole point of custom validators — they extend the “if the process is running, its configuration is known-good” guarantee from types to domain shapes, so a malformed ARN or a non-TLS URL is as impossible to run on as a non-integer port.

The practical guidance is a preference order and a safety rule. Prefer the lightest tool: a Field() constraint over a pattern over a @field_validator over a @model_validator, escalating only when the rule genuinely needs the heavier tool. Lift recurring rules into named Annotated types so a constraint has one home and one test. Put cross-field rules in a model validator, because only it sees the whole model. And on the safety side, keep regexes anchored and quantifier-flat, prefer explicit string checks to clever patterns, and reject bad values with a clear message rather than silently repairing them. Do that and the model becomes the single, trustworthy place where “what does a valid configuration look like?” is both written down and enforced.