I’m Blake. I write about probability, stochastic processes, Python tooling, and the practical side of building things with code. More about me →

  • TIL: Dynamically Installing Python Wheels and spaCy Models at Runtime

    A common practice when serving models is to dynamically download model weights at runtime to keep Docker image sizes small. Recently, with spaCy, this proved more complex than anticipated. spaCy model weights are distributed as Python wheels, designed for installation with Pip.

    I wanted to avoid uv adding these weights directly to my project, as this could significantly increase image size and potentially impact CI/CD build times.

    Before embarking on solving this problem, we should measure the performance impact first. Another team member did this, since I still struggle to follow this advice sometimes! While the size of the image increased, the total build time concern was totally mitigated by image caching in out CI/CD pipeline.

    The best code is no code, so this was without doubt the best approach for this particular problem. But, I was still curious about how to dynamically load python wheels in a way that was agnostic to pip or uv. Here’s the code that solved this niche problem:

    import site
    import tempfile
    import urllib.request
    import zipfile
    from pathlib import Path
    
    import spacy
    
    
    def install_spacy_model(model_url: str) -> None:
        """Installs a spaCy model by downloading and extracting its wheel from a URL."""
        last_reported_percent = -10
    
        def progress_hook(
            block_num: int,
            block_size: int,
            total_size: int,
        ) -> None:
            """Progress callback for urllib.request.urlretrieve."""
            nonlocal last_reported_percent
            downloaded = block_num * block_size
            if total_size > 0:
                percent = min(100, (downloaded * 100) // total_size)
                # Report progress every 10%
                rounded_percent = (percent // 10) * 10
                if rounded_percent > last_reported_percent:
                    mb_downloaded = downloaded / (1024 * 1024)
                    mb_total = total_size / (1024 * 1024)
                    print(
                        f"Progress: {rounded_percent}% ({mb_downloaded:.1f}/{mb_total:.1f} MB)"
                    )
                    last_reported_percent = rounded_percent
    
        try:
            print(
                f"Downloading model from {model_url}"
            )  # Changed 'url' to 'model_url' for consistency
    
            # Download the wheel file to a temporary location
            with tempfile.NamedTemporaryFile(suffix=".whl", delete=False) as temp_file:
                urllib.request.urlretrieve(  # nosec
                    model_url, temp_file.name, reporthook=progress_hook
                )
                wheel_path = temp_file.name
    
            # Extract the wheel to site-packages
            site_packages = site.getsitepackages()[0]
    
            with zipfile.ZipFile(wheel_path, "r") as wheel_zip:
                wheel_zip.extractall(site_packages)
    
            # Clean up temporary file
            Path(wheel_path).unlink()
    
            print("Successfully installed model")
    
        except Exception as e:
            print(f"Failed to install model: {e}")
            raise e
    
    
    model_name = "en_core_web_sm"  # Example model name
    model_url = "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.0/en_core_web_sm-3.7.0-py3-none-any.whl"  # Example URL
    
    try:
        model = spacy.load(model_name)
    except OSError:
        print(
            f"Model '{model_name}' not found. Installing..."
        )  # Added single quotes for clarity
        install_spacy_model(model_url)
        model = spacy.load(model_name)
  • WSGI vs ASGI: Serving Python Models with Gunicorn or Uvicorn

    You’ve trained a brilliant model and/or built a nice web app with FastAPI or Flask in Python. Now comes the less glamorous step: getting predictions to users in real time. How do we translate web requests into something that our Python backend can process? We’ll need a web server to run our web app so people or other systems can send HTTP requests like “here’s some data, give me a prediction”.

    There’s two main choices for types of servers we can use: WSGI (Web server gateway interface) or ASGI (Asynchronous Server Gateway Interface). This is often something I’ve glosses over in my experience, but I’ve learned more about the differences and started to understand where one might be better suited than the other.

    A quick metaphor that helped me:

    • WSGI is like a sit-down restaurant: each table (request) gets its own waiter until the meal is served
    • ASGI is like a food truck with a smart queue: customers place orders, and the kitchen keeps cooking while people wait.

    Gunicorn: The WSGI Workhorse

    Gunicorn (“Green Unicorn”) speaks WSGI, the classic Python web standard. It’s the go-to choice for traditional frameworks like Flask or Django.

    Key traits:

    • Multiple workers: Gunicorn launches several independent worker processes. Each worker handles one request at a time (and if one crashes, the others keep running)

    • CPU-friendly: Great when each request needs real CPU time — for example, a big scikit-learn model doing heavy feature engineering or a PyTorch model making predictions.

    • Battle-tested: If your API doesn’t use async/await, Gunicorn is the safe, well-worn path.

    For example (using Flask):

    # main.py
    from flask import Flask, request, jsonify
    import joblib
    
    app = Flask(__name__)
    model = joblib.load("model.pkl")  # CPU-heavy model
    
    
    @app.route("/predict", methods=["POST"])
    def predict():
        data = request.json["features"]
        yhat = model.predict([data])
        return jsonify(prediction=yhat[0])

    Launch with:

    gunicorn -w 4 main:app   # 4 workers, good for multi-core CPUs

    Uvicorn: The Async Speedster

    Uvicorn speaks ASGI, the newer standard built for asynchronous (non-blocking) code. It shines when your app spends most of its time waiting — for example, making API calls or database queries.

    Key traits:

    • Event loop: By default Uvicorn runs a single-threaded “event loop”, juggling many requests without blocking.

    • I/O-friendly: Perfect for apps built with FastAPI or Starlette, where you can await network calls and keep serving others while one request is waiting.

    • Workers optional: You can run multiple event-loop workers if you want more CPU coverage.

    For example (using FastAPI):

    # main.py
    from fastapi import FastAPI
    import joblib
    
    app = FastAPI()
    model = joblib.load("model.pkl")  # could also call async APIs inside
    
    
    @app.post("/predict")
    async def predict(features: list[float]):
        yhat = model.predict([features])
        return {"prediction": yhat[0]}

    Launch with:

    uvicorn main:app --reload --workers 1   # add --workers N if CPU-heavy

    But there’s a way to combine these approaches if you’re using FastAPI but need both async I/O and CPU scaling: use Gunicorn workers for the Unicorn event loops:

    gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app

    This gives you Gunicorns multi-process power plus Uvicorns async support — handy for ML APIs that fetch remote data and crunch numbers.

    Which Should You Use for Models?

    • CPU-heavy inference (e.g., scikit-learn, PyTorch, TensorFlow): Gunicorn (multiple processes spread the CPU load)

    • I/O-heavy tasks (fetching data, chaining APIs, streaming): Uvicorn (async shines when waiting is the bottleneck)

    • Hybrid: Run Uvicorn inside Gunicorn to get both multiprocessing and async support

    For many data scientists, the choice of server is an afterthought. But the right one can mean the difference between a sluggish API and one that scales smoothly as requests pile up.

  • TIL: How to Address Stray Pandas `SettingWithCopy` Warnings

    If you’ve used Pandas, you’ve probably seen:

    SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame

    It means Pandas isn’t sure if you’re editing the original DataFrame or a temporary view. Sometimes the change sticks, sometimes it doesn’t — which is a recipe for silent bugs.

    To fix it:

    1. Find the warnings

    Run scripts/tests and force warnings to be raised as errors:

    # If using pytest, run your tests
    pytest -W error::pandas.errors.SettingWithCopyWarning
    # Otherwise, run your script and make all warnings raise errors:
    python -W always your_script.py 2>&1 | grep SettingWithCopy

    If you’re in a notebook environment, we can force all these warning to be errors, to find where they are and address them. At the top of the notebook:

    import warnings
    import pandas as pd
    
    warnings.simplefilter("error", category=pd.errors.SettingWithCopyWarning)
    1. Inspect each case

    Look for slices like df[df["age"] > 30] or df.loc[df["y"] == "a"] followed by mutations.

    Decide whether you want a copy or intend to mutate the original.

    1. Be explicit
    • If you want a copy:

      df2 = df[df["age"] > 30].copy()
    • If you want to modify the original:

      df.loc[df["age"] > 30, "category"] = "senior"

    Always make your intent clear with .copy() or .loc. The warning goes away, and your code becomes predictable.

    Long term fix: Polars instead of Pandas

    polars sidesteps this entire issue. Every operation is explicit: you either create a new DataFrame or you don’t. No ambiguous views, no surprise warnings, and you also get fast performance out of the box.

    Here’s the same operation in Polars:

    import polars as pl
    
    df = pl.DataFrame({"age": [25, 35, 50]})
    df2 = df.filter(pl.col("age") > 30).with_columns(pl.lit("senior").alias("category"))
    print(df2)

    No warnings, no ambiguity — just the damn result you asked for.

  • Extreme Programming for ML Teams: Faster Delivery, Reliable Results

    Despite their overlap, the best practices of software engineering in one field often falter when applied directly to data science. Agile rituals that work for feature-driven software projects can feel clumsy when applied to ML. In software, progress can be sliced into neat features shipped every one or two weeks. In data science, progress is murkier: projects take longer, discovery work is essential, and failed experiments aren’t setbacks — they’re the process itself.

    Agile frameworks were developed for software because traditional “waterfall” workflows from engineering could not keep pace in the uncertain world of software. Data science teams face the same problem, but with different constraints. The lesson is the same: collaboration and knowledge sharing are the countermeasures to uncertainty — enabling teams to learn quickly, embrace creativity, and move toward faster, more reliable progress.

    That’s why a recent post by Jacob Clark from Hyperact (Should we revisit XP in the age of AI?) caught my attention. Jacob makes a compelling argument for bringing “extreme programming” (XP) back to the forefront for agile software development today, and I noticed that the tenets of XP are exactly the same as what I’ve observed makes ML teams successful.

    What is XP?

    XP is an agile software development methodology focused on improving software quality and responsiveness to changing requirements. Introduced by Kent Beck in 1996, the core idea is to take good development practices to the “extreme”. It’s defined by 5 core values:

    1. Communication: constant, clear communication between team members and stakeholders.

    2. Simplicity: build the simplest solution that works; avoid over-engineering.

    3. Feedback: use frequent testing and customer involvement to get rapid feedback.

    4. Courage: be willing to change direction, refactor, or throw away code when needed.

    5. Respect: everyone’s contributions are valued; sustainable teamwork.

    XP prescribes a set of engineering practices, many of which are now mainstream agile techniques:

    1. Collaboration & Team Practices

      • Pair Programming – two developers work together at one workstation.
      • Collective Code Ownership – anyone can change any part of the code.
      • Coding Standards – shared style to keep codebase consistent.
    2. Code Quality & Engineering Discipline

      • Test-Driven Development (TDD) – write automated tests before writing code.
      • Refactoring – improve code structure continuously without changing behavior.
      • Simple Design – design for current needs, not speculative future requirements.
    3. Process & Delivery Practices

      • Continuous Integration (CI) – integrate and test code frequently (multiple times a day).
      • Small Releases – deliver working software frequently (days to weeks).
      • Sustainable Pace (“40-hour week”) – avoid burnout; maintain long-term productivity.
      • On-Site Customer – customer representative works closely with the team to clarify requirements.

    In practice, development is done in short 1-2 week iterations, tasks are derived from user stores — which keep the requirements and needs in full focus. Importantly, planning for each task is adaptive: the scope can change, but time boxes are fixed.

    This workflow is very responsive to changing requirements, increases collaboration and knowledge sharing, and encourages high code quality through TDD, CI, and refactoring.

    However, it’s not a practice suited for large distributed teams. It makes a strong assumption on the ease of getting regular time with a customer representative, and it also appears to have a high cost to productivity at first. These are valid concerns, but I strongly argue against the perceived buy-in cost. The heart of this workflow is not only sufficient for fast and reliable development — it’s absolutely necessary!

    Adapting XP for Data Science

    Some of these principles work well in data science without modification, but parts of this need re-interpretation to be applied effectively in ML teams.

    Where XP Fits Well:

    • Continuous Integration (CI)

      • Automatically run evaluation suites, ensure reproducibility
      • Automatic re-training of models
      • Combine code and data pipelines into CI/CD
    • Test-Driven Development (TDD)

      • Hard to apply directly, but you can do data-driven TDD:

        • Write evaluation metrics/tests before building the model.
        • Define expected performance thresholds (e.g., “F1 ≥ 0.8 on holdout data”).
    • Refactoring & Simple Design

      • Keep pipelines and feature engineering code clean and modular
        • Use dependency injection patterns when writing code for models and pipelines
        • Simpler code is more interpretable code, thus is much less of a liability!
      • Use comments judiciously to help with knowledge transfer
        • “code tells you how, comments tell you why”
      • Avoid premature scaling
        • (i.e. don’t over-engineer a data pipeline for 1TB when you have 10GB)
    • Pair Programming and Collective Code Ownership

      • Pair on model debugging, feature engineering, or reproducibility reviews
      • Helps avoid “hero research code” that no one else understands
      • Antidote for “silo-ing” of individual team members on long-running projects
      • Significantly de-risks a project by effortlessly distributing knowledge
      • Document datasets and model assumptions, not just code!
    • Sustainable Pace

      • Especially relevant: ML/AI projects often push people into long “experiment binge” cycles
      • XP’s principle of sustainable pace keeps teams fresh and effective

    Where XP Needs Adaptation:

    • Small Releases

      • Software: a new feature every week
      • ML: a new model version every week may not be realistic. Instead:
        • Release dashboards, experiments, prototypes in small increments
        • Treat notebooks, reports, or trained baselines as “releases”
    • On-Site Customer

      • In ML, “the customer” often doesn’t know what’s feasible until we try
      • Adaptation: keep close contact with domain experts / stakeholders, but frame it as joint discovery.

    Where XP Breaks Down:

    • TDD on research code

      • Writing tests before experimentation doesn’t make sense when the science is unknown
      • Better: validate after by writing reproducibility tests and data quality checks.
      • This can be done easily in notebooks after ingesting and transforming data:
        • Verify column names in DataFrames
        • check that the summary statistics of columns match what’s expected
        • Validate data types and schemas match what’s expected
    • Simplicity principle vs. experimental complexity

      • Sometimes you do need to try a more complex model just to test a hypothesis
      • Simplicity can’t always be enforced upfront — it comes later, after pruning ideas

    An XP Workflow for data science

    Adapting XP to data science means treating experiments, pipelines, and models as the deliverables of your workflow, rather than traditional software features. A typical workflow might look like this:

    Iteration Cycle (1–2 weeks)

    • Define experiment stories: Each iteration begins with a clear objective, e.g., “train model X with features Y and Z” or “clean and validate dataset A.”
    • Run experiments and document results: Capture metrics, observations, and insights. Sharing results early ensures the team learns collectively and avoids duplicated effort.
    • Add engineering stories: Include tasks like automating data pipelines, writing tests, or deploying a model service.
    • Deliver a “release”: In ML, a release isn’t always a production model. It could be a prototype, an evaluation report, a cleaned dataset, or a baseline model ready for further experimentation.

    Daily Rhythm

    • Quick standups: Share what worked, what failed, and what the next step is.
    • Pair programming and reviews: Collaborate on experiments, code, and reproducibility checks.
    • Continuous integration (CI): Run automated tests on both code and data pipelines. Validate unit tests, schema checks, and model evaluation scripts frequently.

    Testing and Validation

    • Code tests: Ensure unit and integration tests catch regressions in preprocessing or pipeline logic.
    • Data tests: Check schemas, missing values, statistical ranges, and data drift.
    • Model tests: Monitor performance thresholds, fairness metrics, and latency for deployed models.

    By combining these practices, the workflow maintains XPs core values: frequent feedback, simplicity, collaboration, and courage to pivot. Experiments are treated as releases, knowledge is shared continuously, and data quality and reproducibility are built into the process. The result is a team that can explore boldly while still delivering reliable, high-quality outcomes.

    Key takeaways

    XP Brings robust development practices to ML Teams by taking the elements of good collaboration to the extreme, resulting in

    • Cleaner pipelines, fewer hidden assumptions.
    • Faster feedback, fewer wasted experiments.
    • Reproducibility by design.
    • A sustainable way to balance research curiosity with engineering rigor.

    Something I think is worth highlighting: frequent pairing may feel like a bottleneck, but I’ve experienced the complete opposite effect whenever I engage with it. Simply put, it’s more fun. Some of the most insightful tips I’ve learned didn’t come from textbooks, they come from people I paired with over the years. The shared enjoyment of learning motivates sustainable progress and keeps momentum consistently high over a long stretch of time.

    XP wasn’t written with ML in mind, but its core philosophy — rapid feedback, simplicity, discipline — fits the challenges of ML/AI engineering. By treating experiments as releases, data as code, and evaluation as testing, XP gives ML teams a methodology that supports both exploration and reliable delivery.