Audio AI

Speech-to-Text (STT)

Transcribe spoken audio into text with high accuracy. Explore state-of-the-art models like open-source Whisper, or commercial APIs for real-time transcription and speaker diarization.

ASR Landscape

Automatic Speech Recognition (ASR) has been revolutionized by Transformer-based models. OpenAI's Whisper set a new standard for robust, multi-lingual transcription, while services like Deepgram push the boundaries of speed and affordability.

Commercial APIs

OpenAI Whisper API: Industry standard quality.
Deepgram: Extremely fast, specialized for real-time.
Google Speech-to-Text: Strong enterprise integration.
AssemblyAI: Excellent features like summarization/topics.

Open Source / Local

OpenAI Whisper: The base model for most modern ASR.
Faster-Whisper: Optimized inference implementation (CTranslate2).
Distil-Whisper: Smaller, faster version of Whisper.

OpenAI Whisper API

Simple usage for high-quality transcription.

from openai import OpenAI
client = OpenAI()

audio_file = open("speech.mp3", "rb")
transcript = client.audio.transcriptions.create(
  model="whisper-1",
  file=audio_file,
  response_format="text" # or json, verbose_json (for timestamps)
)

print(transcript)

Local: Faster-Whisper

Run Whisper efficiently on your own hardware using CTranslate2 optimization. 4x faster than original implementation.

1. Installation

pip install faster-whisper

2. Python Usage

from faster_whisper import WhisperModel

model_size = "large-v3"

# Run on GPU with FP16
model = WhisperModel(model_size, device="cuda", compute_type="float16")

# or run on CPU with INT8
# model = WhisperModel(model_size, device="cpu", compute_type="int8")

segments, info = model.transcribe("audio.mp3", beam_size=5)

print("Detected language '%s' with probability %f" % (info.language, info.language_probability))

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Deepgram (Real-time speed)

Leading choice for live streaming audio transcription due to extremely low latency and low cost.

from deepgram import DeepgramClient, PrerecordedOptions

deepgram = DeepgramClient("DEEPGRAM_API_KEY")

with open("audio.wav", "rb") as file:
    buffer_data = file.read()

payload = {
    "buffer": buffer_data,
}

options = PrerecordedOptions(
    model="nova-2",
    smart_format=True,
)

response = deepgram.listen.prerecorded.v("1").transcribe_file(payload, options)
print(response.results.channels[0].alternatives[0].transcript)

Comparison Matrix

Tool	Type	Accuracy	Speed	Cost
OpenAI Whisper API	API	Excellent	Medium	$0.006 / min
Deepgram Nova-2	API	Excellent	Very Fast	$0.0043 / min
Faster-Whisper	Local	Excellent (Large-v3)	Hardware Dependent	Free
AssemblyAI	API	Great	Medium	Mid-range