Validate database and Redis URLs with pydantic

A malformed DATABASE_URL does not fail at startup — it fails on the first query, with a stack trace that points at the driver, not the config. A pydantic validator turns it into a clear boot-time error. This page validates connection URLs, extending Custom Validators & Constraints.

Connection URLs are a special case worth their own page because they combine three properties that make them dangerous when unvalidated. They are complex — a scheme, optional credentials, a host, a port, a path, and query parameters all packed into one opaque string — so there are many ways to malform them. They are typed as plain str, so pydantic’s type check does nothing to validate their internal structure. And the failure of a bad one is deferred: the value sits inert until the first connection attempt, at which point it surfaces as a driver exception three stack frames deep, disconnected from the configuration that caused it. A validator collapses that deferred, misattributed failure into a precise startup error that names the field and the problem.

The two rules that matter most are a scheme allow-list — rejecting anything that is not a driver you actually support — and TLS enforcement, ensuring an encrypted scheme in the environments that require one. Both are cheap to express and both prevent a class of production failure that is otherwise diagnosed the hard way.

There is a broader principle here that applies to any connection string, not just databases and caches: the value your application holds is a contract with an external system, and the earliest, cheapest place to check that contract is at configuration load. Everything the driver will eventually require of the URL — a supported scheme, a real host, the right transport security — is knowable at startup, so checking it there converts a runtime dependency failure into a configuration error you own and can message clearly. The driver’s own error, by contrast, arrives late, is written in the driver’s vocabulary, and often obscures whether the problem is the URL, the network, or the remote service. Validating up front is how you keep “the config is wrong” and “the database is down” as two distinct, separately-diagnosable failures instead of one confusing failure that sends an incident down the wrong path.

Problem 1: any string accepted

# ANTI-PATTERN: a typo'd scheme passes, fails at first connect
class Settings(BaseSettings):
    database_url: str        # "postgres//db" (missing colon) is a valid str
    redis_url: str

The error appears later as an opaque driver exception, far from the bad value. Consider postgres//db — a real typo, missing the colon after the scheme. As a str it is perfectly valid, so the settings model constructs without complaint and the process starts healthy. The problem only surfaces when some request finally opens a database connection, at which point the driver raises a parse or connection error whose message is about DSNs and sockets, not about a configuration variable. An on-call engineer reading that traceback has no obvious thread back to “someone mistyped DATABASE_URL”, so a five-character typo becomes a multi-person debugging session under incident pressure. Validating the URL at construction moves the failure to boot, where the error names the field and the process refuses to start rather than starting healthy and failing on the first request that touches the database.

Problem 2: plaintext where TLS is required

# ANTI-PATTERN: non-TLS Redis accepted in production
redis_url = "redis://cache:6379"   # should be rediss:// in prod

A redis:// URL silently disables TLS where rediss:// was required. This is more dangerous than a malformed URL because it does not fail at all — the connection succeeds, over plaintext, and traffic flows unencrypted between your application and its cache. The single missing s in redis:// versus rediss:// is the entire difference between an encrypted and an unencrypted channel, and nothing about the running system signals which one you got: the cache works, requests are served, and the credentials and data crossing the wire are simply exposed to anyone on the network path. It is the kind of misconfiguration a security review finds months later, if at all, by which point unencrypted traffic carrying credentials and data has been flowing the whole time.

Because a scheme check cannot know your intent on its own — redis:// is perfectly valid for local development — the enforcement has to be conditional: require the TLS scheme when a require_tls flag is set, which you turn on in production and staging and leave off locally where a plaintext cache on localhost is harmless and convenient. That makes the model reject a plaintext URL exactly in the environments that must not have one, while still allowing it where TLS would be needless friction. The cross-field validator that expresses this is the reason the URL rules cannot be a simple pattern — they depend on another setting. A plain regex or a Field(pattern=...) can check that a URL looks like a Redis URL, but it cannot express “must be the TLS variant when require_tls is set”, because that rule reads a second field. This is exactly the boundary between what a declarative constraint can do and what needs a validator with visibility into the rest of the model, and connection URLs land squarely on the validator side because their most important rule is conditional on the environment’s security posture.

Secure implementation

# config/urls.py
from urllib.parse import urlparse
from pydantic import field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")
    database_url: str
    redis_url: str
    require_tls: bool = True

    @field_validator("database_url")
    @classmethod
    def valid_db(cls, v: str) -> str:
        parsed = urlparse(v)
        if parsed.scheme not in {"postgresql", "postgresql+psycopg", "mysql+pymysql"}:
            raise ValueError(f"unsupported database scheme: {parsed.scheme!r}")
        if not parsed.hostname:
            raise ValueError("database_url has no host")
        return v

    @field_validator("redis_url")
    @classmethod
    def valid_redis(cls, v: str, info) -> str:
        parsed = urlparse(v)
        if parsed.scheme not in {"redis", "rediss"}:
            raise ValueError("redis_url must use redis:// or rediss://")
        if info.data.get("require_tls", True) and parsed.scheme != "rediss":
            raise ValueError("require_tls is set but redis_url is not rediss://")
        return v

urlparse plus a scheme allow-list catches malformed and plaintext URLs at startup; the Redis validator cross-checks require_tls. You can also use pydantic’s PostgresDsn/RedisDsn types for parsing, but a custom validator gives a clearer message and enforces TLS.

The validator does three checks, and each catches a distinct failure. urlparse splits the URL into components so you can inspect them without a regex — parsed.scheme, parsed.hostname, and so on. The scheme allow-list (postgresql, postgresql+psycopg, mysql+pymysql) rejects both typos like postgres//db, which parse to an empty or wrong scheme, and unsupported drivers you never intend to accept. The host check rejects a URL with no host, which is a common shape of a truncated or half-filled value where a substitution failed silently. And the Redis validator adds the cross-field TLS check by reading info.data.get("require_tls") — the mechanism pydantic v2 provides for a field validator to see another field’s already-validated value. Note the ordering dependency: info.data contains only fields validated before this one, so require_tls must be declared above redis_url on the model for the cross-check to see it.

A note on the built-in DSN types: pydantic ships PostgresDsn and RedisDsn that parse and normalise connection URLs, and they are fine when you only need parsing. The reason to write a custom validator anyway is control over two things the built-ins do not give you — a clear, domain-specific error message that reads well in a startup log, and enforcement of application-specific rules like “TLS required in production” or “only these two drivers are supported” that a generic parser has no way to know about. Use the DSN types for convenience where a generic parse suffices; reach for a custom validator when the message quality or a cross-field rule matters.

The same pattern generalises to the other connection strings a service holds. A message-broker URL wants a scheme allow-list (amqps:// over amqp:// in production); an object-store endpoint wants an HTTPS scheme and a real host; a search-cluster URL wants TLS and a port check. Rather than writing a bespoke validator for each, you can lift the shared logic — “parse, allow-list the scheme, require a host, enforce TLS when a flag is set” — into a small reusable helper or an Annotated type, so every connection field on the model inherits the same discipline with a one-line declaration. That keeps a growing configuration consistent: adding a new external dependency means adding a typed, validated URL field, not another chance to accept a malformed or plaintext endpoint. The database and Redis validators on this page are simply the two most common instances of a rule that every outbound connection deserves, and the helper that expresses them is the same one you extend for the next external system you integrate.

Gotchas & version-specific behaviour

• urlparse does not validate credentials — it only splits the URL; the scheme/host checks do the work.
• pydantic’s PostgresDsn and RedisDsn parse and normalize, but raise generic errors; custom validators read better in logs.
• Access sibling fields in a v2 field validator via info.data (only fields validated before it are present — order matters).
• Never log the full URL on error if it embeds a password; report the scheme/host only.

The credential-logging gotcha is the one with security consequences. A database URL routinely embeds a password — postgresql://user:secret@host/db — so an error handler that helpfully prints the offending value into a startup log has just written the credential to wherever those logs go. When a URL validator raises, put only the scheme and host in the message (unsupported database scheme: 'postgres'), never the full value, and if you must reference the URL, reconstruct a redacted form from urlparse components with the password stripped. The info.data ordering gotcha is more subtle but easy to trip over: because a v2 field validator sees only fields validated before it through info.data, the field you cross-check against must be declared earlier on the model — put require_tls above redis_url on the model, or the TLS check silently reads a missing value from info.data and defaults through without enforcing anything.

Production parity checklist

• DB and Redis URLs validated against a scheme allow-list at startup.
• TLS enforced (rediss://, sslmode=require) when require_tls is set.
• extra="forbid" rejects stray connection variables.
• Error messages avoid printing embedded credentials.
• A CI fixture exercises valid and invalid URLs.

The CI fixture is what proves the validator actually works rather than just existing. It is easy to write a URL validator that is subtly too strict — rejecting a legitimate postgresql+asyncpg scheme you forgot to add to the allow-list — or too loose, and a test that constructs the model with both known-good and known-bad URLs catches either. Assert that a realistic production URL constructs cleanly, that a plaintext redis:// raises when require_tls is on, and that a malformed URL raises with a message that does not contain a password. Those three cases together verify the allow-list, the TLS cross-check, and the credential redaction, so a future edit that breaks any of them fails the build instead of shipping to production unnoticed.

Key takeaways

A scheme allow-list and a TLS cross-check turn a bad connection URL into a precise startup error instead of a runtime mystery. Connection URLs earn dedicated validation because they are complex strings that a str type cannot vet, whose failures are deferred to the first connection and misattributed to the driver, and one of whose most dangerous mistakes — plaintext where TLS was required — does not fail at all. A validator built from urlparse, a scheme allow-list, a host check, and a conditional TLS rule collapses all of that into a clear boot-time error.

Two habits keep the validator safe and maintainable: never put a password-bearing URL in an error message, and mind the field order so a cross-field TLS check can actually see require_tls through info.data. Back the whole thing with a CI fixture that exercises a good URL, a plaintext-in-production URL, and a malformed URL, and the connection configuration becomes something you have proven correct rather than something you merely hoped was. For ARN and webhook validation, see Validators for AWS ARNs and URLs.