Flagship system

KOTA

Conversation as infrastructure.

Turns restaurant phone calls into structured, actionable orders without requiring the restaurant to replace or deeply integrate its POS.

Built for environments where missed calls mean lost revenue, and speed matters more than perfect inputs.

Voice Agent · LLM Orchestration · Menu Intelligence · Real-time Streaming · Order Sync

Visit live site → (opens in new tab)
Live demodeployed
Real-timespeech to order
POS-independentno replacement

What Actually Happens

Customer says

“uh yeah can I get like two orange chickens and... wait do you guys have chow mein?”
hesitationfiller wordsmenu ambiguitymid-sentence correction

Structured result

itemOrange Chicken ×2
itemChow Mein ×1 (confirmed)
confidencehigh
missing infonone

The Problem

Restaurants miss calls during peak hours.

Lost orders. Lost revenue. No record of what was missed.

Staff are forced to choose between phone and counter.

Inconsistent handling. Customers repeat themselves. Errors compound.

Existing solutions require POS lock-in or workflow rewrites.

Adoption fails because the cure is more invasive than the disease.

System Flow

01Call
inIncoming phone call
transformTelephony capture via streaming
outRaw audio stream
02Speech
inRaw audio stream
transformDeepgram Nova-2 real-time transcription
outUnstructured text with timestamps
03Intent
inUnstructured transcript
transformLLM extracts items, quantities, modifiers
outStructured intent object
04Menu Grounding
inIntent object
transformValidated against live menu data, resolves ambiguity
outMenu-verified order items
05Resolution
inVerified items
transformConfidence check, missing-info detection, confirmation loop
outComplete order ticket
06Output
inOrder ticket
transformDirect integration to kitchen workflow
outActive order, no staff intervention

Where Systems Break

Unclear input“give me the usual”: no context, no history
Overlapping items“chicken fried rice... no wait, shrimp”: item swap mid-sentence
Missing modifiers“large lo mein”: size not on menu, only one size exists
Background noiseKitchen clatter, multiple speakers, phone static

How KOTA Handles It

Clarification loopSystem asks targeted follow-ups instead of failing silently
Fallback promptsWhen confidence drops below threshold, rephrase and re-confirm
Confidence thresholdsEach item scored independently, so partial orders can proceed
Structured retriesFailed extractions re-enter the pipeline with accumulated context

Interface Principles

Trust

Operators see what the system heard, what it interpreted, and what it decided at every step. No black boxes.

Visibility

Live transcription, confidence indicators, and order state are surfaced in real time. The system is legible while running.

Operator clarity

Staff don't need to learn the system. They see completed orders arrive. Intervention is optional, not required.

System Impact

Missed callsreducedCalls answered autonomously during peak hours when staff can't pick up
Manual transcriptioneliminatedNo more writing orders on paper or repeating back to confirm
Order cycleshortenedStructured ticket hits kitchen workflow in seconds, not minutes
Workflow disruptionzeroNo new hardware, no POS migration, no staff retraining required

Stack

Voice

Twilio Streaming · Deepgram Nova-2 STT · WebSocket audio pipeline

Intelligence

GPT-4 Realtime · Structured JSON extraction · Menu grounding engine

Execution

Order resolution pipeline · Confidence scoring · Kitchen workflow sync

Infrastructure

GKE · Cloud SQL · S3 logging · Real-time monitoring

agent protocol