.env File Management

A .env file is the most convenient way to run an app locally and the most common way teams leak credentials into git. The danger is twofold: the file overwrites real platform secrets when loaded carelessly, and it ends up committed. This page makes .env loading safe in both directions.

Within the configuration pipeline, the .env file is a development-only source that must sit below the real environment variables in the precedence order — it fills gaps locally, never overrides production.

Those two hazards pull in opposite directions, which is why .env handling goes wrong so reliably. The overwrite hazard is a runtime problem: a file that reaches production and wins against an injected value redirects the service somewhere unintended, and the symptom is confusing because the environment genuinely contains the right value. The leak hazard is a repository problem: once a credential lands in git history it is exposed to everyone with clone access and to every backup, mirror, and CI cache of that repository, and deleting the file in a later commit does nothing. One is fixed by precedence, the other by tooling, and a .env setup is only safe when both are handled.

Why the file sits below the environment

The ordering question has a single decisive argument behind it: the .env file ships with the working copy, while environment variables are injected by whatever is running the process. Anything that ships with the code is, by construction, older and less specific than a value the deployment supplied moments ago. Placing the file above the environment inverts that relationship and gives the stalest source the final word.

Play out the failure concretely. A developer adds DATABASE_URL=postgres://localhost/dev to their .env. Months later a build accidentally copies the file into an image — a broad COPY . . with an incomplete .dockerignore is all it takes. The container starts, the orchestrator injects the real DATABASE_URL, and then application startup calls load_dotenv(override=True), replacing it with localhost. The service reports healthy, connects to nothing, and every log line about the environment shows the correct value, because the environment did have the correct value until the loader overwrote it. With override=False the same accident is harmless: the file is present, the key is already set, the loader skips it, and nobody notices.

The mirror-image case matters too. Locally there is no injected value, so the file supplies everything, and the developer experience is exactly what people want from .env — one file, edit and run. Nothing about override=False degrades local use; it only removes the file’s ability to win where it should not. That asymmetry is what makes it the right default rather than a compromise: it costs nothing locally and prevents the entire overwrite class in production.

Secure implementation

# config/dotenv_loader.py
import os
from pathlib import Path
from dotenv import dotenv_values

def load_env(path: str = ".env") -> None:
    env_path = Path(path)
    if not env_path.exists():
        return                                  # production has no .env; that's fine
    file_values = dotenv_values(env_path)       # isolated dict, not a mutation
    for key, value in file_values.items():
        if value is not None and key not in os.environ:  # override=False semantics
            os.environ[key] = value             # only fill genuinely missing keys

By reading into a dict and writing back only the missing keys, a platform-injected DATABASE_URL is never clobbered by a stale local one. This is override=False implemented explicitly. Writing it out rather than calling load_dotenv() is worth the six extra lines for one reason: the precedence decision becomes visible. A reviewer reading if key not in os.environ sees the rule; a reviewer reading load_dotenv() has to know the library’s default, and defaults change across major versions while explicit conditions do not.

dotenv_values is the important choice here. It parses the file and hands back a dictionary without touching the process environment at all, which means the merge policy is yours to write. That opens up options a mutating loader forecloses: you can log which keys the file supplied, refuse keys that are not in an allow-list, or skip the merge entirely and feed the dict into a settings model as an explicit source. The value is not None guard covers a genuine parser case — a bare DEBUG line with no = yields None rather than an empty string, and assigning None into os.environ raises a TypeError at startup.

The early return when the file is missing is not a nicety either. Production has no .env, and a loader that raises or warns when the file is absent trains everyone to ignore its output. Silence when the file is missing, action when it is present, and never any behaviour that depends on which environment the code thinks it is in — that is the whole contract.

Configuration reference

The .env.example row is the one teams skip and later regret. Because the real file is untracked, nothing in the repository records which keys a developer needs, so onboarding becomes an exercise in reading tracebacks until the application stops complaining. A committed example file with every required key and obviously fake values — DATABASE_URL=postgres://user:pass@localhost/dbname — turns that into a copy, and it doubles as a review surface: a pull request that adds a required setting should add it to the example in the same diff, which makes the new requirement visible to reviewers rather than discovered on the next deploy.

Keep the example honest about which keys are secrets. A short comment marking a block as credentials tells a new developer to fetch real values from the secret store rather than inventing them, and it tells whoever writes the deployment manifests which keys need to come from a managed source rather than a plain variable. The file costs nothing to maintain and removes an entire category of “how do I run this locally?” questions.

Feeding the file into a settings model

Merging into os.environ is the compatibility-friendly approach — every library that reads environment variables sees the values, including ones you did not write. But if the application already uses a settings model, there is a cleaner route that never mutates the process environment at all.

# config/settings.py
from pydantic import SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",            # read directly; no os.environ mutation
        env_file_encoding="utf-8",
        extra="ignore",             # tolerate unrelated keys in a shared file
    )
    database_url: str               # required — no default
    api_key: SecretStr              # never printed by repr()
    log_level: str = "INFO"

settings = Settings()               # env vars still outrank the file

The model reads the file itself, so the values land in typed fields and nowhere else. Nothing is written back to os.environ, which means a stray print(os.environ) in a debugging session cannot spill a credential that only the model knows about, and a library that reads the environment directly cannot accidentally pick up a value intended for your service. The precedence is unchanged — pydantic-settings ranks real environment variables above the env_file — so the production behaviour is identical to the explicit merge.

extra deserves a deliberate choice here. extra="ignore" is right when the .env is shared with other tooling, such as a Compose file or a frontend build, because those keys are not the model’s business. extra="forbid" is right for a service that owns its file outright, since it turns a typo into a startup error rather than a silently missing value. What you should not do is pick one by accident and discover the consequences during an incident — the two behave very differently the first time somebody misspells a key.

The one thing the model approach gives up is reach. Third-party libraries that read os.environ for their own configuration — an AWS SDK looking for AWS_PROFILE, a database driver reading PGSSLMODE — will not see values that only exist inside your settings object. If your .env carries keys for those libraries as well as for your own code, you need the merge, or a hybrid where the model owns your keys and a short explicit list is exported for the libraries that need it.

Parsing rules that surprise people

.env is not a standardised format, and the details that differ between implementations are exactly the ones that bite. Quoting is the first: python-dotenv strips matching surrounding quotes, so PASSWORD="s3cr3t" yields s3cr3t, but a value containing a literal # needs quoting or everything after the # is treated as a comment. TOKEN=abc#123 silently becomes abc, which produces an authentication failure whose cause is invisible in the file — the value looks correct when you read it.

Interpolation is the second. python-dotenv expands ${VAR} references against values already parsed and against the process environment, so BASE_URL=https://${HOST}/api resolves if HOST is defined earlier in the file. That is convenient until a password contains a literal dollar sign, at which point the expansion mangles it. Single quotes disable interpolation, so PASSWORD='pa$$word' survives intact while PASSWORD="pa$$word" may not. When a credential mysteriously stops working after a rotation, an unquoted special character is the first thing to check.

Third, shell habits do not transfer cleanly. A leading export is accepted and ignored, which is helpful when a file is shared between source .env in a shell and the Python loader. But shell command substitution — VALUE=`date` — is not evaluated; you get the literal characters. And multi-line values need explicit quoting across the newline, which matters for PEM-encoded keys: a private key pasted raw into a .env file parses as one assignment plus several broken lines, and the resulting value is a truncated key that fails with a confusing cryptography error rather than a parse error.

The practical rule that covers all three is to single-quote anything you did not choose yourself. Machine-generated credentials — API keys, connection strings, base64 blobs — are exactly the values likely to contain #, $, quotes, or equals signs, and they are also the values whose corruption produces the least informative error. Values you typed by hand, like LOG_LEVEL=INFO, need no quoting at all. And for anything genuinely multi-line, the honest answer is that .env is the wrong container: put the PEM file on disk, reference its path in a variable, and let the application read it.

Layered files and per-developer overrides

Once a team grows past a handful of people, one .env stops fitting. Somebody runs a local Postgres on a non-standard port, somebody else points at a shared staging database, and a third person needs verbose logging permanently. The temptation is to commit .env.development, .env.staging, and friends so each person can pick one — but that puts real values back in the repository and, worse, makes the repository enumerate the environments again.

The pattern that holds is a two-file layer: a tracked .env.defaults containing only non-secret, genuinely shared values, plus an untracked .env holding each developer’s real credentials and overrides. Load the defaults first, then the personal file, then let the real environment win over both:

# config/dotenv_loader.py (layered)
from pathlib import Path
from dotenv import dotenv_values
import os

def load_env(root: Path | None = None) -> dict[str, str]:
    root = root or Path(__file__).resolve().parent.parent
    merged: dict[str, str] = {}
    for name in (".env.defaults", ".env"):        # later file wins over earlier
        merged.update({k: v for k, v in dotenv_values(root / name).items()
                       if v is not None})
    applied = {}
    for key, value in merged.items():
        if key not in os.environ:                 # the real environment still wins
            os.environ[key] = value
            applied[key] = "file"
    return applied                                # log the keys, never the values

The ordering inside the loop is the whole design: files layer among themselves in the order listed, but the entire stack of files still sits below the process environment. A developer overriding LOG_LEVEL in their personal file is unaffected by the defaults; a deploy injecting LOG_LEVEL is unaffected by either. Returning the applied keys gives you the source log for free — print the dictionary at startup and every “which value am I actually using?” question becomes a lookup.

Anchoring root to the module’s location rather than the working directory is what makes this behave the same under pytest, a systemd unit, an IDE run configuration, and a plain python -m app. It is a one-line change that eliminates the most common false report about .env files doing nothing at all.

Deployment parity: local to production

1. Local dev — developer keeps an uncommitted .env; load_env() fills missing keys only.
2. CI — no .env file exists; required keys come from pipeline variables and the secret store.
3. Staging/Production — the orchestrator injects real values; load_env() is a no-op because the file is absent and keys are already set.

The same code path runs everywhere; only the presence of the file differs. That property is what makes local testing meaningful: when a developer reproduces a production issue locally, the resolution logic they are exercising is the identical function, and the only variable is which layer happens to hold the value. If instead the loader were wrapped in if ENV == "development", local runs would exercise a code path that never runs in production, and the two environments could diverge without any test noticing.

CI is the step worth being deliberate about, because it is where the two hazards meet. There is no .env file, so every required key must come from pipeline variables — which is a useful forcing function, since a test suite that only passes with a developer’s local file has an undeclared dependency. Point the pipeline at a minimal set of test values and let the missing-key validation fail the build when someone adds a required setting without adding it to the pipeline. That failure is cheap in CI and expensive in production, which is exactly the trade you want.

There is a useful test hiding in this list. Delete your .env, set the handful of variables the application genuinely needs, and start it. If it runs, the file is doing what it should — filling gaps — and nothing depends on it being present. If it fails on a key you did not know was required, you have just found a value that was living in a developer’s file and nowhere else, which is the same value that will be missing the first time someone deploys to a new environment. Running that test occasionally, or wiring it into CI as a job with a deliberately minimal variable set, keeps the file from quietly becoming load-bearing.

Security boundaries & guardrails

• .env is always in .gitignore; a committed .env.example carries dummy values only.
• Run gitleaks or detect-secrets as a pre-commit hook to block accidental commits.
• Set file permissions to 600 so other local users cannot read it.
• Keep override=False; never let a local file win over an injected secret.
• Wrap any secret read from .env in SecretStr inside the settings model.
• Add .env to .dockerignore as well — .gitignore does not stop a COPY . . from baking the file into an image.

The pre-commit hook is the guardrail that actually changes outcomes, because it is the only one that operates before the mistake becomes permanent. .gitignore protects against git add ., but not against git add -f, not against a file named .env.local that the pattern misses, and not against a credential pasted directly into a source file. A scanner that runs on staged content catches all three, and it catches them on the developer’s machine, where the fix is deleting a line rather than rewriting history and asking every colleague to re-clone the repository.

If a secret does reach a commit, the order of operations matters and the instinct is usually wrong. Rotate first, purge second. Rewriting history with git filter-repo is slow, disruptive for everyone with a clone, and — critically — does not help if the repository has already been fetched, mirrored, or indexed by a CI cache. The credential should be treated as compromised from the moment it was pushed, so revoking it is what actually closes the exposure. Purging history afterwards is worth doing to stop the value being re-leaked, but it is cleanup, not containment. Write the two-step order down somewhere the team will find it during an incident, because under pressure the instinct is always to make the commit disappear first.

SecretStr addresses a quieter leak path. Values that arrive from .env end up inside a settings object that gets logged during debugging, serialised into an error report, or rendered by an exception handler that prints the whole model. Wrapping the field means repr() prints SecretStr('**********') and the real value is only available through an explicit get_secret_value() call, so accidental exposure requires an explicit act rather than an oversight.

The 600 permission bit is the guardrail people dismiss as paranoia, and it is worth one sentence of defence. On a shared build agent, a container image with a world-readable file baked in, or a developer machine running any tool with broad filesystem access, a default 644 means the credential is readable by processes that have no business seeing it. Setting the bit costs nothing and closes a path that no amount of git tooling addresses, because the exposure never involves a commit at all.

Troubleshooting

• Production secret overwritten by .env — you called load_dotenv(override=True); switch to the isolated-merge pattern above.
• .env accidentally committed — rotate every credential it contained immediately, then purge it from history and add the pre-commit scanner.
• Variable not picked up — the key already exists in os.environ; with override=False that is intentional, the existing value wins.
• SyntaxWarning on load in Python 3.12 — unescaped backslashes in values; see Safely Load .env Files in Python 3.12.
• Value truncated at a # — the value needs quoting; an unquoted hash starts a comment.
• File ignored entirely — the loader resolves .env relative to the working directory, not the module; a process started from a different directory finds nothing.

That last one accounts for a surprising share of “the file does nothing” reports. Running python app.py from the repository root works; running the same code from a subdirectory, from a systemd unit with a different WorkingDirectory, or through an IDE run configuration that sets its own working directory does not. Resolving the path relative to a known anchor — Path(__file__).resolve().parent.parent / ".env" — removes the ambiguity, and it makes the file’s location a property of the project layout rather than of however the process happened to be launched.

The override=False entry is worth reading twice, because it is the only line in the list that describes correct behaviour being mistaken for a bug. A developer edits .env, restarts, and sees no change — the natural conclusion is that the loader is broken. In fact a shell export from an earlier session, an IDE run configuration, or a leftover docker compose environment entry is supplying the key, and the file is being skipped exactly as designed. Printing the applied-keys dictionary at startup turns that thirty-minute confusion into a glance.

Key takeaways

The invariant: a .env file may fill missing local values but may never override an injected one, and it never enters version control. Load it into an isolated dict, merge with override=False semantics, and gate it with gitignore plus a pre-commit scanner. Those three moves cover both hazards — the runtime one, where a stale file outranks a real secret, and the repository one, where a credential becomes permanent history.

Everything else on this page is detail around that core. Parse with dotenv_values so the merge policy is explicit and reviewable; quote values that contain # or $ so the parser returns what you wrote; resolve the path from a known anchor so the file is always found regardless of the working directory; and wrap secrets in SecretStr so a debug log or a serialised error report cannot spill them. Get those right once, in a single loader module that every entry point imports without exception, and the file becomes what it was always meant to be — a local convenience with no power to affect production. The measure of success is that nobody thinks about the file: it is present on developer machines, absent everywhere else, and the same startup code runs in both cases without caring which situation it is in.