ProbableOdyssey | Blake Cook

TIL: How to parse config from env vars in Python

· 2 min read · 418 words

Table of Contents

In an effort to collate all my runtime options and parameters into a single source of truth, I initially created a src/globals.py file with the values and their defaults:

import os

FOO: str = os.get("FOO", "my_value")
BAR: int = int(os.get("BAR", "55"))
BAZ: bool = os.get("BAZ", "False").lower().startswith("t")

A simple solution, these symbols can then be imported by using from src.globals import FOO, ..., but this solution has a couple of drawbacks:

For example, let’s say we have a function floob that uses a dict containing foo, bar, and baz:

from src.globals import FOO, BAR, BAZ


def floob(data: dict[str, str | int | bool] | None = None):
    data = data or {}
    foo = data.get("foo", FOO)
    bar = data.get("bar", BAR)
    baz = data.get("baz", BAZ)
    ...

Admittedly this is a contrived example, but it can start to get unwieldy as the number of get statements increases and becomes more scattered about. Not the mention there is the possibility of a bug in globals.py when casting items to their correct types.

I came across pydantic_settings as a solution to this by providing a class to configure this.

from pydantic import Field
from pydantic_settings import BaseSettings


# An env prefix can optionally be specified as well
class MyConfig(BaseSettings, env_prefix="APP_"):
    foo: str = Field(
        description="foo",
        default="my_value",
    )
    bar: int = Field(
        description="bar",
        default=55,
    )
    baz: bool = Field(
        description="baz",
        default=False,
    )

When this class is instantiated, the values are looked up from the given kwargs, then dynamically looked up from the environment, and then falls back to the defaults:

# $ export APP_FOO="env_value"
config = Config(baz=True)

print(config.foo)
# "env_value"
print(config.bar)
# 55
print(config.baz)
# True

Where this really shines is in the integration with the rest of pydantic, such as value validation:

from typing import Annotated

from pydantic import Field, AfterValidator
from pydantic_settings import BaseSettings


def is_allowed(value: str) -> str:
    if value is in ["not_allowed", "wumbo"]:
        raise ValueError(f"Invalid string value: '{value}'")
    return value

CheckedStr = Annotated[str, AfterValidator(is_allowed)]

# An env prefix can optionally be specified as well
class MyConfig(BaseSettings, env_prefix="APP_"):
    foo: Checked = Field(
        description="foo",
        default="my_value",
    )
    bar: int = Field(
        description="bar",
        default=55,
    )
    baz: bool = Field(
        description="baz",
        default=False,
    )

# $ export APP_FOO=wumbo
_ = Config()  # Raises a validation error

The end result is a more flexible config class, cleaner definition of defaults, robust parsing of env vars without the standard boilerplate, on top of the validation and type-checking from pydantic. …

#data-validation #environment-variables #pydantic #validation #project-configuration #configuration-management #python #type-checking #best-practices #software-engineering #configuration

Reply to this post by email blZake@proZbableodyssey.blog (remove Z characters) ↪

Comments

Leave a comment

Markdown is supported. Your email is private and only used if you'd like a reply.