Live audio transcription in Python
Table of Contents
I’ve wanted to explore the family of Whisper models for speech-to-text problems. I finally found a bit of time to experiment, and used a bit of Python to perform live audio transcription from my microphone. Unexpectedly, I found my self enjoying sharpening my multithreading skills along the way.
The core idea was straightforward: capture audio, feed it to a Whisper model,
and display the transcribed text. I ended up using the dependencies
uv add sounddevice soundfile faster_whisper for this project.
My solution involved using a separate thread for audio recording, which pushes
incoming audio blocks into a queue. The main thread (or another consumer)
then pulls data from this queue, records it to a temporary file, and
ultimately, passes it to the Faster Whisper model for transcription. This
queue mechanism was crucial for decoupling the audio input from the
transcription process, preventing bottlenecks and ensuring a smooth user
experience.
I refactored my code into a clean class to contain all the audio recording logic:
class AudioRecorder:
def __init__(
self,
filename: Path,
record_device: int,
):
self.filename = filename
self.record_device = record_device
device_info = sd.query_devices(self.record_device, "input")
self.sample_rate = int(device_info["default_samplerate"])
self.channels = device_info["max_input_channels"]
self.record_queue = queue.Queue()
def record_callback(self, indata, _frames, _time, status):
"""This is called (from a separate thread) for each audio block."""
if status:
print(status, file=sys.stderr)
self.record_queue.put(indata.copy())
def record(self, overwrite: bool = False):
if self.data:
if not overwrite:
raise ValueError("Data loaded, use `overwite=True` or clear the data")
self.data = None
try:
with (
sf.SoundFile(
self.filename,
mode="x",
samplerate=self.record_sample_rate,
channels=self.record_channels,
) as file,
sd.InputStream(
samplerate=self.record_sample_rate,
device=self.record_device,
channels=self.record_channels,
callback=self.record_callback,
),
):
for msg in [
"#" * 80,
"press Ctrl+C to stop the recording",
"#" * 80,
]:
print(msg, file=sys.stderr)
while True:
file.write(self.record_queue.get())
except KeyboardInterrupt:
passThe record_callback function, powered by sounddevice, executes in a
separate thread. It continuously shoves audio data into self.record_queue.
Meanwhile, the main record loop get()s this data from the queue,
effectively writing the audio without explicitly managing threads myself.
After the recording is complete, the temporary WAV file is passed to the
FasterWhisper model for transcription:
def transcribe(filename, model):
segments, _info = model.transcribe(filename, language="en", beam_size=5)
output = []
for segment in segments:
line = "[%.2fs -> %.2fs] %s" % (
segment.start,
segment.end,
segment.text,
)
output.append(line)
print(line)
return outputPutting this all together in a central main() function:
import queue
import sys
import tempfile
from pathlib import Path
import sounddevice as sd
import soundfile as sf
from faster_whisper import WhisperModel
class AudioRecorder: ...
def transcribe(filename, model): ...
def main():
device = 1 # see `sd.query_devices()`
model_size = "small"
model = WhisperModel(model_size, compute_type="float32")
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmpfile:
filename = tmpfile.name
player = AudioRecorder(filename, record_device=device)
player.record()
_segments = transcribe(filename, model)
if __name__ == "__main__":
main()The results from this were pretty impressive, even with a small model!
#deep-learning #speech-to-text #multithreading #transcription #real-time-transcription #programming #machine-learning #python #whisper #audio-processing
Reply to this post by email blZake@proZbableodyssey.blog (remove Z characters) ↪