Pass Secrets to Subprocesses Safely in Python

Shelling out is routine — a database dump, a migration tool, a CLI that has no Python binding — and each of those calls has to receive a credential somehow. The obvious way, putting it in the command line, is the one that publishes it to every process on the host. This page covers the channels available, how to stop a child inheriting credentials it does not need, and the ordering that makes each choice safe, extending the environment variables and os.environ topic.

Problem 1: the credential on the command line

# ANTI-PATTERN: visible to every process on the host while this runs
subprocess.run(["pg_dump", f"postgresql://app:{password}@db/app", "-f", "out.sql"])

A process’s full argument list is readable through the process table: ps aux shows it, /proc/<pid>/cmdline contains it, and any monitoring agent, container runtime or audit subsystem that records process starts records it too. The exposure lasts as long as the command runs, which for a database dump can be a long time, and the recorded copies last as long as their retention policies.

There is no way to make an argument private, so the fix is always to use a different channel. Every widely-used CLI offers at least one: an environment variable it reads, a password prompt it will accept on standard input, or a configuration file path. Choosing among those is the rest of this page.

Problem 2: handing the child everything you hold

# ANTI-PATTERN: the dump tool receives the payment key, the session secret, everything
subprocess.run(["pg_dump", "-d", "app"])       # no env argument: inherits the lot

Without an explicit env, a child process receives a copy of the parent’s entire environment. A backup tool that needs one database password also receives your payment provider key, your signing secret and anything else the service holds — and can log, transmit or crash-dump any of it. That is not a hypothetical: tools do print their environment in debug modes, and crash handlers commonly capture it.

Pass a constructed environment containing only what the child needs. Start from the small set of variables required for the process to function at all — PATH, HOME, LANG, and whatever the tool documents — add the one credential, and pass that. The child then cannot leak what it never received, which is a stronger guarantee than any care taken inside it.

Problem 3: building a command string for a shell

# ANTI-PATTERN: quoting, injection and tracing, all at once
subprocess.run(f"mysqldump -u app -p{password} appdb > dump.sql", shell=True)

shell=True introduces three problems simultaneously. The value must be quoted correctly or a character in the password breaks the command — or worse, changes it. The whole command string passes through a shell that may be traced, logged, or recorded in history. And the credential is still an argument, so the process-table exposure applies as well.

Use a list of arguments with no shell, and handle redirection in Python by passing a file object as stdout. That removes the quoting question entirely, makes injection impossible, and leaves you with exactly one decision: which channel carries the secret.

Secure implementation

# runner.py — a minimal environment, the secret out of the argument list
import os
import subprocess
from pathlib import Path
from config import settings

# 1. the only variables the child genuinely needs to function
BASE_ENV = {k: os.environ[k] for k in ("PATH", "HOME", "LANG", "TZ") if k in os.environ}


def dump_database(target: Path) -> None:
    child_env = {
        **BASE_ENV,
        # 2. exactly one credential, named as the tool documents it
        "PGPASSWORD": settings.db_password.get_secret_value(),
        "PGHOST": settings.db_host,
        "PGUSER": settings.db_user,
    }
    with target.open("wb") as out:
        subprocess.run(
            ["pg_dump", "--dbname", "app"],   # 3. a list, no shell, no credential in the arguments
            env=child_env,                    # 4. an explicit environment, not an inherited one
            stdout=out,                       # 5. redirection in Python rather than through a shell
            check=True,
        )

# stdin_runner.py — for a tool that reads a credential from standard input
def restore_with_stdin_password(dump: Path) -> None:
    process = subprocess.Popen(
        ["some-cli", "restore", "--password-stdin", "--file", str(dump)],
        stdin=subprocess.PIPE,
        env=BASE_ENV,                        # 6. no credential in the environment either
        text=True,
    )
    # 7. the value exists only between these two lines, in a pipe nothing else can read
    process.communicate(input=settings.db_password.get_secret_value())
    if process.returncode != 0:
        raise RuntimeError("restore failed")  # 8. no output echoed: it may contain the input

The BASE_ENV dictionary is worth building once and importing everywhere a subprocess is launched. It makes the default behaviour explicit — this child gets these variables — and it turns “which credentials can this tool see?” into a question the code answers directly. When a tool turns out to need one more variable, adding it is a visible, reviewable change rather than something that was always silently available.

Note the error handling in the stdin example: the return code is checked, and the process output is not echoed. A tool that receives a password on standard input sometimes includes it in an error message on failure, and reflexively printing process.stderr after a failure is how that reaches your log. Log the return code and a description you control, exactly as you would at any other client boundary.

Choosing a channel, in order

There is a preference order that resolves nearly every case. Standard input first, where the tool supports it: the value never enters the process table or the environment, exists only while the pipe is open, and is not inherited by anything the child itself spawns. Many CLIs have a --password-stdin flag or read a prompt, and using it costs a couple of extra lines around Popen.

An explicitly-scoped environment variable second. This is the most widely supported channel and is safe enough when the environment is constructed rather than inherited: the value is readable by the child and by anything the child spawns, which is exactly the population you have chosen. It is the right default for tools with no stdin option.

A temporary file third, with care. Create it with restrictive permissions before writing — open with mode 0o600 rather than creating and then chmod-ing, since the gap between the two is a window — put it on a memory-backed filesystem if the platform has one, and delete it in a finally block so a crash does not leave it behind. It is the fallback for tools that only accept a credentials file, and it is genuinely acceptable when done in that order.

The command line never. If a tool offers no channel but its argument list, that is a defect worth reporting and worth working around — usually by wrapping it in a small script that reads from the environment and constructs the arguments itself, keeping the exposure inside a process you control.

One organisational note: the number of places a codebase launches subprocesses is usually small and worth centralising. A single module exposing run_tool(name, args, secrets=...) gives you one place where the base environment is built, one place where the shell is refused, and one place a reviewer checks when the question is “can this leak a credential?”. Scattered subprocess.run calls each make the same three decisions independently, and one of them eventually gets it wrong.

Gotchas and version-specific behaviour

• Command-line arguments are world-readable through the process table; there is no way to hide them.
• Without an explicit env, subprocess copies the parent’s whole environment to the child.
• shell=True adds quoting and injection risk and puts the credential through a shell that may be traced.
• Redirect with a Python file object as stdout rather than a shell redirection.
• A tool’s error output may echo a credential it received on stdin, so do not reflexively log child output.
• Create temporary credential files with the restrictive mode at open time, not with a later chmod.
• A child that spawns its own children passes on whatever environment it received, so scoping compounds.

Production parity checklist

• No credential appears in any argument list, in any code path, including error and retry paths.
• Every subprocess call passes an explicit env built from a small base plus what that child needs.
• shell=True is not used; commands are lists and redirection is handled in Python.
• Tools that support a stdin credential use it in preference to an environment variable.
• Temporary credential files are created with mode 0o600 and removed in a finally block.
• Child process output is not logged verbatim when the child received a credential.

Key takeaways

A credential in an argument list is public to the host and to everything that records process starts, and no amount of care elsewhere changes that. Work down a fixed preference order instead: standard input where the tool supports it, an explicitly-constructed environment where it does not, a restrictive temporary file for tools that insist on one, and never the command line.

The second half is scoping. subprocess hands a child the parent’s entire environment by default, which means a backup tool receives your payment key along with the database password it needed. Building a small base environment and adding exactly one credential converts an implicit grant into a reviewable line of code, and a child cannot disclose what it never received. Combine that with a list-form command, no shell, and not echoing child output after a failure, and shelling out stops being a hole in an otherwise careful design. The surrounding practices are on the environment variables topic page.