ProbableOdyssey | Blake Cook

Live audio transcription in Python

· 3 min read · 452 words

Table of Contents

I’ve wanted to explore the family of Whisper models for speech-to-text problems. I finally found a bit of time to experiment, and used a bit of Python to perform live audio transcription from my microphone. Unexpectedly, I found my self enjoying sharpening my multithreading skills along the way.

The core idea was straightforward: capture audio, feed it to a Whisper model, and display the transcribed text. I ended up using the dependencies uv add sounddevice soundfile faster_whisper for this project.

My solution involved using a separate thread for audio recording, which pushes incoming audio blocks into a queue. The main thread (or another consumer) then pulls data from this queue, records it to a temporary file, and ultimately, passes it to the Faster Whisper model for transcription. This queue mechanism was crucial for decoupling the audio input from the transcription process, preventing bottlenecks and ensuring a smooth user experience.

I refactored my code into a clean class to contain all the audio recording logic:

class AudioRecorder:
    def __init__(
        self,
        filename: Path,
        record_device: int,
    ):
        self.filename = filename
        self.record_device = record_device

        device_info = sd.query_devices(self.record_device, "input")
        self.sample_rate = int(device_info["default_samplerate"])
        self.channels = device_info["max_input_channels"]

        self.record_queue = queue.Queue()

    def record_callback(self, indata, _frames, _time, status):
        """This is called (from a separate thread) for each audio block."""
        if status:
            print(status, file=sys.stderr)
        self.record_queue.put(indata.copy())

    def record(self, overwrite: bool = False):
        if self.data:
            if not overwrite:
                raise ValueError("Data loaded, use `overwite=True` or clear the data")
            self.data = None

        try:
            with (
                sf.SoundFile(
                    self.filename,
                    mode="x",
                    samplerate=self.record_sample_rate,
                    channels=self.record_channels,
                ) as file,
                sd.InputStream(
                    samplerate=self.record_sample_rate,
                    device=self.record_device,
                    channels=self.record_channels,
                    callback=self.record_callback,
                ),
            ):
                for msg in [
                    "#" * 80,
                    "press Ctrl+C to stop the recording",
                    "#" * 80,
                ]:
                    print(msg, file=sys.stderr)

                while True:
                    file.write(self.record_queue.get())

        except KeyboardInterrupt:
            pass

The record_callback function, powered by sounddevice, executes in a separate thread. It continuously shoves audio data into self.record_queue. Meanwhile, the main record loop get()s this data from the queue, effectively writing the audio without explicitly managing threads myself.

After the recording is complete, the temporary WAV file is passed to the FasterWhisper model for transcription:

def transcribe(filename, model):
    segments, _info = model.transcribe(filename, language="en", beam_size=5)
    output = []
    for segment in segments:
        line = "[%.2fs -> %.2fs] %s" % (
            segment.start,
            segment.end,
            segment.text,
        )
        output.append(line)
        print(line)
    return output

Putting this all together in a central main() function:

import queue
import sys
import tempfile
from pathlib import Path

import sounddevice as sd
import soundfile as sf
from faster_whisper import WhisperModel


class AudioRecorder: ...


def transcribe(filename, model): ...


def main():
    device = 1  # see `sd.query_devices()`

    model_size = "small"
    model = WhisperModel(model_size, compute_type="float32")

    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmpfile:
        filename = tmpfile.name

        player = AudioRecorder(filename, record_device=device)
        player.record()

        _segments = transcribe(filename, model)


if __name__ == "__main__":
    main()

The results from this were pretty impressive, even with a small model!

#deep-learning #speech-to-text #multithreading #transcription #real-time-transcription #programming #machine-learning #python #whisper #audio-processing

Reply to this post by email blZake@proZbableodyssey.blog (remove Z characters) ↪