Menu and domain grounding

Two jobs, not one

Transcription and resolution are different problems and should be kept apart. Transcription turns audio into a spoken phrase: "lemme get a large latay". Resolution takes that phrase and decides which real catalog entry, if any, it refers to. A model that does both in one step tends to smooth over the gap — "latay" becomes a confident latte with no record that it was a guess. Separating the steps gives you a place to attach a confidence score and a place to decline.

The catalog is the ground truth. Every accepted order line points at a real id; the model's job is to map onto that structure, not to extend it. Enumerating aliases and modifier options up front is what lets a resolver match a spoken phrase instead of trusting the model's free-form output.

Illustrative: the catalog as ground truth

// The catalog is the ground truth. Every accepted order line must point
// at a real entry here — the model does not get to invent a new one.
type MenuItem = {
  id: string;
  name: string;
  aliases: string[];          // synonyms, nicknames, common mishearings
  modifiers: ModifierGroup[]; // what can legally attach to this item
};

type ModifierGroup = {
  id: string;
  name: string;               // e.g. "milk", "size", "spice"
  options: string[];          // enumerated — not free text
  required: boolean;
};

const menu: MenuItem[] = [
  {
    id: "latte",
    name: "Caffe latte",
    aliases: ["latte", "cafe latte", "latay", "flat latte"],
    modifiers: [
      { id: "size", name: "size", options: ["small", "medium", "large"], required: true },
      { id: "milk", name: "milk", options: ["whole", "oat", "soy", "almond"], required: false },
    ],
  },
  // ...
];

Failure modes

• Hallucinated item. The caller says something the menu does not cover and the model returns a fluent, reasonable-sounding item anyway. The order line looks valid but points at nothing the kitchen can make, and the mistake surfaces at fulfilment rather than on the call.
• Wrong modifier binding. Two items and two modifiers in one utterance — 'a large latte and an oat cortado' — and 'oat' attaches to the latte, or 'large' drifts onto the cortado. Modifiers were parsed as a flat list instead of bound to the item that can legally take them.
• Synonym or nickname miss. A regular orders by a name the staff use but the catalog does not list, so the resolver scores every real item low and either declines a valid order or forces a clarify on something the caller considers obvious. The alias table is thinner than how people actually speak.
• Silent substitution. The requested item is genuinely off-menu, and instead of saying so the agent maps it to the nearest thing that does exist and reads it back as if it were what the caller asked for. The caller only finds out when the wrong thing arrives.
• Quantity ambiguity. 'A couple of croissants' or 'a few' is resolved to a specific number without confirmation, or dropped to a quantity of one. The item resolved cleanly but the count is a guess dressed up as a fact.

Implementation notes

Put a resolver between the transcript and the order. It scores the spoken phrase against every item's canonical name and its aliases, keeps the best match, and returns a confidence with it. The alias list is where synonyms, nicknames, and common mishearings live, so the match is against how people actually say the item, not just its printed name.

Illustrative: resolving a phrase to a real item

// Two jobs, kept apart: transcription hands us a spoken phrase; resolution
// maps that phrase onto a real catalog entry (or refuses to).
type Match = { item: MenuItem; confidence: number };

function resolveItem(spoken: string, menu: MenuItem[]): Match | null {
  const phrase = normalize(spoken); // lowercase, strip filler, expand "a"/"an"
  let best: Match | null = null;

  for (const item of menu) {
    // Score the phrase against the canonical name and every known alias.
    // similarity() is illustrative — token overlap plus edit distance,
    // NOT a benchmarked accuracy figure.
    const candidates = [item.name, ...item.aliases];
    const score = Math.max(...candidates.map((c) => similarity(phrase, normalize(c))));

    if (!best || score > best.confidence) {
      best = { item, confidence: score };
    }
  }

  // Below the floor, we do not guess. A low top score means "did not hear a
  // real item", which is a clarify-or-decline signal, not a match.
  const FLOOR = 0.6; // tune against real calls, not a measured constant
  return best && best.confidence >= FLOOR ? best : null;
}

The confidence floor is what turns "always produce something" into "produce something or admit you did not hear a real item." Below the floor is not a weak match to accept quietly; it is a signal to clarify or to say the item is not on the menu. The value is a starting point to tune against real calls, not a measured constant — raise it when the agent accepts near-misses, lower it when it clarifies things a caller finds obvious.

Modifiers bind to items, not to the utterance. Once an item resolves, match each spoken modifier against that item's own enumerated options, so "oat" can only land on something that offers a milk choice. A modifier that fits no group on the item is not silently dropped — it is flagged, because it usually means the wrong item resolved or the caller is describing something you do not offer. Required groups with nothing bound become an ask-back, not a default.

Illustrative: binding modifiers to the resolved item

// A modifier only counts if the item it lands on can legally take it.
// "Oat" is meaningless on a pastry; "large" is meaningless on a fixed-size
// item. Bind against the item's own enumerated options, then flag leftovers.
function bindModifiers(item: MenuItem, spokenMods: string[]) {
  const applied: Record<string, string> = {};
  const unresolved: string[] = [];

  for (const raw of spokenMods) {
    const token = normalize(raw);
    const group = item.modifiers.find((g) =>
      g.options.some((opt) => similarity(token, opt) >= 0.7),
    );

    if (group) {
      const opt = group.options.find((o) => similarity(token, o) >= 0.7)!;
      applied[group.id] = opt; // last one wins if the caller restated it
    } else {
      unresolved.push(raw); // heard a modifier that this item cannot take
    }
  }

  // required groups with nothing bound are an ask-back, not a default.
  const missing = item.modifiers.filter((g) => g.required && !(g.id in applied));
  return { applied, missing, unresolved };
}

Quantity gets the same treatment as everything else: a vague count is an unresolved field, not a value to invent. "A couple" and "a few" should confirm rather than commit. And when the top match sits below the floor, the honest move is to name it — "we do not have that, did you mean one of these" over an enumerated set — rather than substitute the nearest real item and hope.

Related notes

• Clarify before commit — confirming an ambiguous order before it becomes a ticket
• How a real-time voice pipeline turns a phone call into a kitchen ticket