I’m Blake. I write about probability, stochastic processes, Python tooling, and the practical side of building things with code. More about me →

  • Dotfile management with Chezmoi and Vim

    I have three computers I use on a regular basis, and my Vim-based shell workflow is heavily dependent on my configuration. Dropbox was my first port of call, but after some conflicts caused by automatic syncing, I learned that manual pulling and pushing with Git is much better suited to this problem. Like other developers, I created a dotfiles repo and used stow to symlink all the files and folders in my configuration. I thought this was a pretty good setup, but unresolved bugs would still sometimes appear on my systems.

    I trialed chezmoi and found it to be an improvement over my old stow-based setup. It takes a slightly more deliberate approach to managing dotfiles.

    Instead of directly symlinking files from your Git repo into your home directory, chezmoi manages that repo under the hood and gives you a controlled “staging area” for changes. This staging area lives in your .local/share/chezmoi directory and acts as an intermediary between your live config files and version control. Any time you make changes to your dotfiles, you explicitly add them to chezmoi — no more accidental commits of stray files or awkward .gitignore rules sprinkled throughout $HOME.

    Once you track your files with chezmoi add, syncing between machines is a straightforward two-step process. Make local edits, re-add your files with chezmoi re-add, go to the chezmoi directory with chezmoi cd and commit and push like you would with any other Git project. Pull remote changes and apply them with chezmoi update. Even through the “push” step feels a bit verbose, this workflow makes it harder to overwrite good configs with bad ones and gives you a cleaner separation between what’s on your machine and what’s stored in the repo.

    chezmoi has made my setup feel more predictable and less error-prone. The final piece of the puzzle was integrating it with my Vim workflow. I wanted to be able to edit tracked files, keep syntax highlighting consistent, and even handle syncing without leaving the editor. With a simple Git alias (git sync) and a handful of Vim commands, I’ve made chezmoi fit naturally into how I already work.

    In my ~/.gitconfig, I have a handy little alias for committing and pushing whatever unstaged changes I have:

    [alias]
        sync = "!f() { \
            msg=\"sync: $(whoami)@$(hostname) $(date '+%Y-%m-%d %H:%M:%S')\"; \
            git add . && \
            git commit -m \"$msg\" && \
            git push; \
        }; f"

    Now I can at least push my chezmoi changes with chezmoi git sync, nice!

    But we can do better than that. How can we integrate this into Vim? Here’s what I’ve written in ~/.vim/plugins/dotfile.vim:

    " Mappings to jump straight into my configurations:
    nnoremap <Leader>; <cmd>edit ~/.vim/vimrc <bar> lcd %:p:h<CR>
    nnoremap <Leader>: <cmd>exec 'edit ' . system("chezmoi source-path") <bar> lcd %:p:h<CR>
    
    " Commands to pull/push changes
    command! DotPull !chezmoi update
    command! DotPush !chezmoi git sync
    
    
    "" If a tracked file is edited locally, automatically re-add it
    "
    function! AddChangesToDotfiles()
      let current_file = expand('%')
      if (system('chezmoi managed ' . current_file) !=? '') && (system('chezmoi diff ' . current_file) != '')
        execute 'silent !chezmoi add ' . current_file
        echo 'Added to dotfiles'
      endif
    endfunction
    
    augroup add_changes_to_dotfiles
      autocmd!
      autocmd BufWritePost * call AddChangesToDotfiles()
    augroup END
    
    
    "" If a tracked file is edited in chezmoi, automatically apply it
    "
    function! ApplyChangesFromDotfiles()
      execute 'silent !chezmoi target-path ' . expand('%') . ' | chezmoi apply'
      echo 'Applied from dotfiles'
    endfunction
    
    augroup apply_changes_from_dotfiles
      autocmd!
      autocmd BufWritePost ~/.local/share/chezmoi/* call ApplyChangesFromDotfiles()
    augroup END
    
    
    "" If opening a file in chezmoi, add proper syntax highlighting
    " (e.g. highlight `dot_bashrc` as `.bashrc`)
    "
    function! DotfilesSetFiletype()
      let l:realpath = trim(system('chezmoi target-path ' . shellescape(expand('%:p'))))
      execute 'setfiletype ' fnameescape(fnamemodify(l:realpath, ':t'))
    endfunction
    
    augroup dotfiles_set_filetype
      autocmd!
      autocmd BufRead,BufNewFile ~/.local/share/chezmoi/* call DotfilesSetFiletype()
    augroup END

    If you haven’t already, I highly encourage you to check out their documentation and quick start

  • TIL: How to generate PR descriptions with Aider

    One of my least favourite things about writing code is writing PR descriptions. After writing the code, annotating all the types, adding docstrings, writing markdown documentation on usage, it feels like I’m repeating myself yet again. Also I can only look at one page of git diffs a ta time, so writing this extra document is a slow process.

    One real-world use case for AI in my workflow addresses this need. I’ve written previously on my AI setup with Aider and Openrouter here. I’ve included this command in my shell to quickly draft an appropriate PR description:

    git diff main..HEAD -- ':(exclude)uv.lock' > PULL_REQUEST.md && aider PULL_REQUEST.md \
      --read .github/PULL_REQUEST_TEMPLATE.md \
      --message-file ~/.aider/message_files/replace_diff_with_pr_description.md

    This uses the message I’ve written at ~/.aider/message_files/replace_diff_with_pr_description.md:

    I have given you a git diff.
    Can you write a pull request description for this change using the template?
    Replace all the content in the file.
    Please keep it as short and direct as possible.

    Aider is a fantastic tool that will add all the files with context that it needs. The end result is a new PULL_REQUEST.md file with a draft of the description which I can easily edit before posting.

    I haven’t done any “prompt engineering” for this, and I’ve only ignored the uv.lock file in my python projects. But this is a nice little template that can be modified easily.

  • TIL: Reliable Pytest Testing with Mocks (by Example)

    Suppose we have the following class in./src/grammar.py:

    # ./src/grammar.py
    import logging
    import language_tool_python
    
    logger = logging.getLogger(__name__)
    
    
    class GrammarMetric:
        """Compute grammar quality scores for text summaries using LanguageTool.
    
        Because the Public API has a rate limit, this metric depends on LanguageTool
        and Java. To install on MacOS, use
    
            brew install openjdk
            sudo ln -sfn /opt/homebrew/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk
    
        On Linux, use
    
            sudo apt update && sudo apt install default-jre
    
        """
    
        default_language = "en-US"
    
        def __init__(self, language: str | None = None):
            self.language = language or self.default_language
            self.server = language_tool_python.LanguageTool(self.language)
    
        def __call__(self, data: list[str]) -> list[float]:
            results = []
            for text in data:
                matches = self.server.check(text)
                results.append(float(max(0, 1 - len(matches) / len(text.split()))))
            return results

    This works — but running tests on it requires downloading LanguageTool resources and Java. That’s slow and flaky in CI. Let’s mock it.

    We’ll:

    • Create shared fixtures and a skipifci mark.
    • Write a mock for LanguageTool.
    • Test the mock to avoid false confidence.
    • Test the actual GrammarMetric using the mock.

    Definitions:

    Mock and MagicMock:

    A Mock is a fake object used in place of a real one during testing. It lets you simulate behavior and test code in isolation. A MagicMock: A powerful kind of mock from Python’s unittest.mock module that automatically creates fake methods and tracks how they’re used.

    The MagicMock object creates attributes and methods dynamically when you access them. When you access an attribute (e.g., mock.some_method()), MagicMock automatically returns another MagicMock instance.

    When you pass keyword arguments to the MagicMock initialiser, like MagicMock(foo=123), it sets those as attributes on the mock. So mock = MagicMock(foo=123) means mock.foo will return 123 instead of a new MagicMock. This lets you control or override parts of the mocks behavior up front

    MagicMock objects have two special attributes that control what’s returned:

    • return_value: Sets what a mock should return when called. For example, mock.method.return_value = 42 makes mock.method() return 42

    • side_effect: Lets you control more complex behavior — like raising exceptions, returning different values each time, or calling a function. For example, mock.method.side_effect = ValueError("oops") makes mock.method() raise an error

    Other Terms:

    • patch: A function from unittest.mock that temporarily replaces a real object (like a class or function) with a mock during a test. It’s often used with a with block or as a decorator.

    • Fixture: A reusable setup function in pytest that provides test data or objects to tests. Defined using @pytest.fixture.

    • @pytest.mark.skipif: A decorator to conditionally skip tests — for example, skipping slow or flaky tests when running in CI (for example, "CI" in os.environ).

    Step 1: Shared Fixtures and Marks

    In ./tests/conftest.py:

    # ./tests/conftest.py
    import os
    import pytest
    from functools import partial
    
    
    def n_samples() -> int:
        return 5
    
    
    def mock_data(n_samples) -> list[str]:
        return ["This is some mock data." for _ in range(n_samples)]
    
    
    skipifci = partial(pytest.mark.skipif, "CI" in os.environ)

    Step 2: Decide What to Mock (and What to Ignore)

    We want to avoid calling LanguageTool.check() in CI. But how does it behave?

    In a Python shell:

    >>> import language_tool_python
    >>> lt = language_tool_python.LanguageTool("en-US")
    >>> matches = lt.check("I can grammar good")
    >>> type(matches), type(matches[0])
    # (list, language_tool_python.Match)

    You can inspect a Match object with:

    >>> dir(matches[0])
    # ... many attributes, but we don’t use them in our code

    If we want to inspect this object more, we could use the following for-loop:

    >>> for k in dir(result): print(f"{k}({type(k).__name__}): {repr(getattr(result, k)):.32}...")
    # ... many attributes, but we don’t use them in our code

    Since our code only checks the number of matches, not their content, returning a list of dummy MagicMock instances is sufficient

    Step 3: Write the Mocks

    In ./tests/test_grammar.py:

    # ./tests/test_grammar.py
    import random
    from unittest.mock import MagicMock, patch
    
    import language_tool_python
    import pytest
    from conftest import skipifci
    
    from src.grammar import GrammarMetric
    
    ## Mock language_tool_python.LanguageTool
    #
    
    
    @pytest.fixture
    def mock_check():
        def _mock_check(text, *args, **kwargs):
            return [
                MagicMock(message="mock", sentence=text, ruleId="MOCK_RULE")
                for _ in range(random.randint(1, 3))
            ]
    
        return _mock_check
    
    
    @pytest.fixture
    def mock_language_tool(mock_check):
        with patch("src.grammar.language_tool_python.LanguageTool") as MockLT:
            instance = MockLT.return_value
            instance.check.side_effect = mock_check
            yield instance

    Why this works:

    • patch targets the location where LanguageTool is used (src.grammar).
    • The mock object has the minimal interface needed to test downstream code.

    Step 4: Test the Mock

    We want to make sure our mock returns the same kind of object as the real one, so downstream code doesn’t break.

    # ./tests/test_grammar.py
    # ...
    
    ## Mock language_tool_python.LanguageTool
    #
    # ...
    
    
    @skipifci(reason="Requires real LanguageTool")
    def test_mock_check_matches_real(mock_check):
        real = language_tool_python.LanguageTool("en-US")
        text = "Me fail English? That's unpossible!"
    
        for result in [real.check(text), mock_check(text)]:
            assert isinstance(result, list)
            for r in result:
                assert hasattr(r, "message")

    Step 5: Test the Metric Object

    Write tests that work with and without the mock:

    # ./tests/test_grammar.py
    # ...
    
    ## Mock language_tool_python.LanguageTool
    #
    # ...
    
    ## Test GrammarMetric
    #
    
    
    def _run_metric(metric, data):
        scores = metric(data)
        assert isinstance(scores, list)
        assert all(isinstance(s, float) for s in scores)
        assert len(scores) == len(data)
    
    
    def test_grammar_metric_mocked(mock_data, mock_language_tool):
        metric = GrammarMetric()
        _run_metric(metric, mock_data)
    
    
    @skipifci(reason="Requires LanguageTool download")
    def test_grammar_metric_real(mock_data):
        metric = GrammarMetric()
        _run_metric(metric, mock_data)

    Summary

    This structure lets you:

    • Use real LanguageTool locally to verify your mock
    • Skip flaky or expensive tests in CI
    • Keep tests fast and deterministic without compromising coverage

    Use this pattern any time you need to test around a costly external dependency

  • AI Coding Assistants: What a Month of Testing Taught Me

    The coding ability of LLMs fascinated me the first time I saw it in action with Github Copilot towards the end of 2021. Then ChatGPT amazed us all, and the developments just kept coming. In 2025, “agentic coding” came to the forefront to terrify and excite software engineers across all calibres. Each new iteration comes with a shiny presentation or blog post, often illustrating a toy example or citing some impressive (but not very useful) benchmark numbers. It sounds like a dream — “All I have to do is type what I want, then the computer will write the code for me”. I think it’s too good to be true

    We’ve seen the shortfalls of AI and the risks associated with blindly trusting the output. The output of an LLM is engineered to look correct, it’s all just statistics under the hood. Fundamentally these models match statistical distributions of sequences of text data in a terabyte-scale dataset. Whether the code it produces is correct is not part of the objective function. This makes LLMs the perfect source of some of the hardest bugs to fix. Most developers by 2025 have experienced something like that 1st or 2nd hand several times.

    And yet despite that, people are endorsing agentic coding. Letting these bug machines write the code in an unsupervised manner?

    And then I came across this article “Field Notes From Shipping Real Code With Claude” by Diwank Tomer. This part of the article convinced me that agentic coding might be worthwhile exploring:

    Your tests are your specification. They’re your safety net. They’re the encoded wisdom of every bug you’ve fixed and every edge case you’ve discovered. Guard them zealously.

    With so much hype around this topic, it’s exhausting separating the signal from the noise. So I took this chance to challenge my criticisms, and try and learn more about this over a month of regular usage. Was this tool really worth the endorsements? Am I just holding the hammer wrong? Does it actually work in realistic scenarios? Has it made me more productive as claimed?

    I don’t think my average productivity has changed with these tools, and it is definitely overhyped. If you’re already a proficient coder, you probably wont see much benefit. However I learned a few spots where it genuinely feels like the right tool for the job. This experience has taught me more about how to write better test code and improved by code reviewing skills, which was unexpected. But claims of massively increased productivity don’t check out.

    It hasn’t put me out of a job yet, but it has earned a spot on my toolbelt for now.

    Past attempts at using AI coding

    I don’t like AI completions.

    It was really amazing watching demos of this magical autocomplete, but when I used it in practice I could not stand it.

    I felt like I was spending more time fighting off bad completions then I was spending on actually doing the task I was supposed to be doing. It interrupted my train of thought constantly, and ground my progress to a snails pace. This is probably the main reason I’m skeptical of agentic coding.

    After that didn’t work out, I went back to chatting with ChatGPT in a browser window — copying and pasting snippets of code to and from my editor, perfectly crafting each prompt to give it the context it needs to generate a good answer

    But again, I still feel like I’m not saving any time. In fact, I feel like I’m spending more time since I’m going back and forth with the LLM chat. Bringing this experience closer to the editor might address this speed bottleneck? Agentic coding might be a step in that direction.

    Setup for agentic coding

    I used OpenRouter as my LLM provider. It’s incredibly impressive how this service is able to switch between different LLMs with no subscription. For my 1 month experiment, I topped up my account with $20 (AUD) and I primarily used claude-sonnet-4 via Aider. Over the past month, I’m guessing I used AI coding about 4-6 times per week (each session was about an hour or 2). This cost me about $10 in credit

    I didn’t tweak the configuration of aider too much, but here’s what I put in my ~/.aider.conf.yaml:

    model: openrouter/anthropic/claude-sonnet-4
    
    # Tweaked the colors to match my terminal
    # Note that you shouldn't use `code-theme` and
    # `dark-mode` at the same time. See
    # https://pygments.org/styles for available themes
    code-theme: gruvbox-dark
    user-input-color: "#B8BB26"
    assistant-output-color: "#83A598"
    
    # Streaming responses were too distracting.
    # I prefer to do something else while it's generating
    # a response, then reading through the output before I continue
    stream: false
    
    # I prefer writing my own commits after I've reviewed the diff
    # of the file changes generated by aider
    auto-commits: false
    
    # Aider will be automatically triggered when you write a
    # comment with the token 'AI!'
    # See https://aider.chat/docs/usage/watch.html
    watch-files: true
    
    # When Aider wants to pull a new file into context, this
    # auto-approves it
    yes-always: true
    
    # This file contains some of my personal coding conventions across
    # all of my projects, it's loaded in read-only mode in every session
    read: ~/CONVENTIONS.md

    I adopted a workflow where I would use Aider one commit at a time so that

    1. I deliberately kept the scope of the problem small so the LLM performs better
    2. I could review and test all the AI output in one place (and I could easily revert it if needed)

    So my general setup consists of launching vim as usual to edit code, opening another shell in the same project directory (via a tmux pane) to launch aider, then sending requests to aider by inserting comments in my code in vim such as

    def processor(
        data: str,
        client: Client,
    ) -> ProcessResult:
        """Process input data using the provided client.
    
        Args:
            data: The input data string to be processed
            client: Client instance used for processing operations
    
        Returns:
            ProcessResult: The result of the processing operation
        """
        ...

    After a few iterations with Aider (making my own edits along the way), I would then review the diff using the greatest vim plugin of all time: vim-fugitive. The command :Gdiffsplit allows me to view the current unstaged changes side by side. Once I was happy with the diff, I would create a commit and proceed.

    Review of agentic coding performance

    Let me be blunt: agentic coding sucks at logic-heavy tasks. Even when the code is well-tested, even when the prompt is specific and constrained, it stumbles. Complex control flow, non-obvious edge cases, or any situation that demands conceptual insight — these are where the models tend to hallucinate, make wrong assumptions, or miss subtle but important details. And because the output looks plausible, these errors are harder to detect.

    But that’s not the whole story.

    There are specific areas where agentic coding shines, and they’re not the ones most of the hype is about:

    • It’s great at drafting tests: I found it very helpful to get that first round of unit tests written, especially for tedious or repetitive cases. The AI doesn’t catch every edge case, but it often thinks of things I’ve missed — and the mechanical nature of test writing is a good match for a probabilistic pattern matcher.

    • It helps clarify vague intentions: If I have a rough idea of what I want to do — “I need a function that converts these three formats into a common schema” — Aider helps me quickly sketch out a solution space. Sometimes it gives me what I wanted. More often, it gives me something slightly wrong, and fixing it helps crystallize my own understanding.

    • It’s good for knowledge transfer: When I’m diving into unfamiliar territory, Aider can summarize the purpose of a function, help guess what a cryptic config file is doing, or give me a working example of an obscure library usage.

    • It’s brilliant for boilerplate and communication: Writing docstrings, filling out __init__.py files, generating markdown summaries, or drafting a PR description — this is where agentic coding impresses me most.

    These benefits aren’t flashy, and they certainly don’t support the narrative of “10x productivity” or “AI writes your code so you don’t have to”. That story falls apart in practice.

    As I’ve written about previously: writing code isn’t the bottleneck. Reading code is. Understanding what it does, what it’s supposed to do, and communicating that clearly to others — that’s where time and attention go. That’s where real engineering happens. And that’s exactly where LLMs can provide real leverage.

    In that sense, using Aider regularly has improved my coding skills — not because it writes correct code, but because it forces me to articulate my intentions more clearly. It’s helped me think more about test coverage and failure modes. More importantly, it’s forced me to read code more deliberately, because I can’t trust the machine — and that has a surprisingly positive side effect: it sharpens my code review reflexes.

    Agentic coding isn’t magic. It’s not replacing us. But if you’re deliberate about its use, and honest about its limitations, it can be a genuinely useful tool.