Back to projects

Voice Chat Assistant

Speak into your mic, get a live transcript, an LLM reply, and a spoken response back, no local models or GPU required.

SWE (AI/MLOps)Voice AILLMReal-time SystemsServerlessDeepgramGroq

Voice Chat Assistant is a full-duplex voice conversation app, speech-to-text, LLM reply, text-to-speech, shipped as two frontends over related backend logic: a Streamlit app for local use (streaming mic audio over a websocket to Deepgram, Groq for chat completion with full conversational memory, Deepgram TTS for playback), and a React and Vite frontend deployed on Vercel that records audio in the browser and talks to a self-contained Python serverless function. It started as a Jupyter notebook wired to a local Ollama model, undeployable without a GPU; the rewrite replaced Ollama with Groq's hosted inference, added retry-with-backoff around every external API call, structured logging, a mocked pytest suite, and GitHub Actions CI. Getting the Vercel deployment right meant real production debugging: a Python file named index.py was silently shadowing the static site's own index.html, requiring an explicit builds/routes configuration once Vercel's zero-config detection proved unreliable for the hybrid static-plus-serverless shape.

What I built

  • Built a full-duplex voice pipeline (Deepgram STT, Groq LLM with full conversational memory, Deepgram TTS) shipped as two frontends over related backend logic: a Streamlit local app and a React and Vite app deployed on Vercel.
  • Migrated the LLM backend from a local Ollama model to Groq's hosted inference to remove the GPU dependency and make the app deployable to a public URL.
  • Moved microphone capture from server-side (sounddevice, only works when server and laptop are the same machine) to the browser (MediaRecorder), the change required to make voice input work for a deployed, multi-user web app.
  • Diagnosed a Vercel routing collision where a Python file named index.py silently shadowed the static frontend's own index.html, root-caused by reading Vercel's function and route inventory directly rather than trusting error messages.
  • Landed on an explicit builds and routes configuration after zero-config auto-detection proved unreliable for the hybrid static-site-plus-serverless-function shape, and scoped the deployment's Root Directory correctly for the monorepo.
  • Added retry-with-backoff around every external API call (STT, LLM, TTS), structured logging, and a pytest suite mocking all external calls, wired into GitHub Actions CI on every push.
  • Removed a hardcoded, plaintext API key from the legacy prototype and replaced it with environment-variable-based secrets management ahead of making the repo public.

Tech stack

SWE (AI/MLOps)Voice AILLMReal-time SystemsServerlessDeepgramGroq

Key results

156ms

STT latency

892ms

LLM latency

142ms

TTS latency

1.19s

Total round-trip

Back to all projects