I Have Built Realtime Conversation Assistant at a Hackathon
I thought the hardest part would be latency. We were building a realtime conversation assistant: a system that listens to speech, understands conversational context, and generates suggestions fast enough for someone to actually use them during a live conversation. On paper, the pipeline sounded straightforward:speech-to-text → context processing → LLM generation → suggestions. In reality, everything broke once humans started talking. It went even worse when tested in the middle of crowded ope