KummaFollow on X

Build · a write-up

KOTA

Conversation as infrastructure.

Turns restaurant phone calls into structured, actionable orders without requiring the restaurant to replace or deeply integrate its POS.

Underneath it is one rule: clarify before you commit. This page documents how the system works, and where it does not.

Messy speech becomes a menu-verified ticket with confidence checks.

What actually happens

Customer says

“uh yeah can I get like two orange chickens and... wait do you guys have chow mein?”
hesitationfiller wordsmenu ambiguitymid-sentence correction

Structured result

itemOrange Chicken ×2
itemChow Mein ×1 (confirmed)
confidencehigh
missing infonone

Constraints

The design had to fit into a working kitchen without asking it to change. Every constraint below is a line the system does not cross.

No POS replacementStructured tickets reach the kitchen workflow directly, without replacing or deeply integrating the point-of-sale system.
No new hardwareThe system answers the existing phone line. There is nothing new to install on site.
No staff retrainingStaff see completed orders arrive. Intervention is optional, not required.
Peak-hour firstDesigned for the moments when phone volume exceeds staff capacity and calls would otherwise go unanswered.

Architecture and system flow

01Call
inIncoming phone call
transformTelephony capture via streaming
outRaw audio stream
02Speech
inRaw audio stream
transformDeepgram Nova-2 real-time transcription
outUnstructured text with timestamps
03Intent
inUnstructured transcript
transformLLM extracts items, quantities, modifiers
outStructured intent object
04Menu Grounding
inIntent object
transformValidated against live menu data, resolves ambiguity
outMenu-verified order items
05Resolution
inVerified items
transformConfidence check, missing-info detection, confirmation loop
outComplete order ticket
06Output
inOrder ticket
transformDirect integration to kitchen workflow
outActive order, no staff intervention

Guardrail design

The core rule is clarify before commit. Each item is scored independently, missing information is detected, and the system runs a confirmation loop before a ticket is committed. Operators can see what it heard, what it interpreted, and what it decided at every step.

Trust

Operators see what the system heard, what it interpreted, and what it decided at every step. No black boxes.

Visibility

Live transcription, confidence indicators, and order state are surfaced in real time. The system is legible while running.

Operator clarity

Staff don't need to learn the system. They see completed orders arrive. Intervention is optional, not required.

What broke and how I fixed it

Unclear input“give me the usual”: no context, no history
Overlapping items“chicken fried rice... no wait, shrimp”: item swap mid-sentence
Missing modifiers“large lo mein”: size not on menu, only one size exists
Background noiseKitchen clatter, multiple speakers, phone static

How KOTA handles it

Clarification loopSystem asks targeted follow-ups instead of failing silently
Fallback promptsWhen confidence drops below threshold, rephrase and re-confirm
Confidence thresholdsEach item scored independently, so partial orders can proceed
Structured retriesFailed extractions re-enter the pipeline with accumulated context