Few-Shot Prompting and Structured Output in Prompt Engineering

Suppose a parcel-locker operator wants every message left at its kiosks sorted into labels such as door_fault or wrong_code. With only an instruction, a common outcome is label drift: the model returns door_fault on one run, Door Fault on another and a full sentence on a third. Nothing is broken, but your program cannot count labels that keep changing spelling. Examples fix that better than more adjectives.

When an instruction is not enough

Instructions describe what you want. Examples show it. If a label has a fuzzy boundary (is "screen is hard to read" a fault or just feedback?), a sentence of explanation leaves room for interpretation, while two worked cases remove most of it. The same goes for format: showing the exact reply you expect is more reliable than describing it.

A few-shot prompt for kiosk feedback

The prompt below states the labels and the output shape once, then shows four examples in the same format, then ends with the new message and leaves the reply open. Ending on an open slot, in the same layout as the examples, is what nudges the model to continue the pattern.

You label feedback left at a parcel-locker kiosk.

Labels: door_fault, wrong_code, late_delivery, other
Reply with JSON only: {"category": <label>, "urgent": <true|false>}
urgent is true only if the customer cannot collect a parcel right now.

Message: "Door 14 will not open and my medicine is inside"
{"category": "door_fault", "urgent": true}

Message: "The code in your email says invalid"
{"category": "wrong_code", "urgent": true}

Message: "Parcel was meant to arrive Tuesday, still nothing"
{"category": "late_delivery", "urgent": false}

Message: "Lovely kiosk, but the screen is hard to read in sunlight"
{"category": "other", "urgent": false}

Message: "{{message}}"

Notice what the examples quietly teach beyond the labels. The urgent flag follows the rule stated above them: the medicine and the invalid code both block a pickup right now, while the late parcel does not. If you wrote an example where urgent contradicted that rule, the model would have two conflicting teachers, and in practice the examples usually win. Treat the examples as part of the specification and review them as carefully as the instructions.

Choosing examples that earn their place

• Cover every label at least once, or the model will under-use the missing ones.
• Include the awkward case. The sunlight-screen example exists to show that mild complaints are other, not door_fault.
• Vary the wording so the model learns the idea, not a phrase. Three examples that all start with "Door" teach a keyword.
• Keep the format identical in every example, including punctuation and key order.
• Mix the order. If the last example before the new message has the same label as many examples earlier, some models lean towards repeating it. Shuffle and compare on your test set.
• Never reuse test messages as examples. If you do, your evaluation will look better than reality.

Structured output you can trust

When a program reads the answer, ask for JSON with named fields and a closed set of allowed values, as above. Then treat the reply as untrusted input. Models sometimes add a sentence before the JSON, wrap it in a code fence, misspell a label or omit a field. Some providers offer features that constrain output to a schema; whether and how depends on the provider, so check its documentation, but validate in your own code either way.

import json

ALLOWED = {"door_fault", "wrong_code", "late_delivery", "other"}

def parse_label(raw):
    """Turn a model's reply into a validated dict, or raise ValueError with the reason."""
    text = raw.strip()
    if text.startswith("```"):                       # models sometimes wrap JSON in a code fence
        text = text.strip("`")
        text = text.removeprefix("json").strip()
    try:
        data = json.loads(text)
    except json.JSONDecodeError as exc:
        raise ValueError(f"not valid JSON: {exc.msg}") from exc
    if not isinstance(data, dict):
        raise ValueError("expected a JSON object")
    if data.get("category") not in ALLOWED:
        raise ValueError(f"category {data.get('category')!r} is not one of {sorted(ALLOWED)}")
    if not isinstance(data.get("urgent"), bool):
        raise ValueError("urgent must be true or false")
    return data

replies = [
    '{"category": "door_fault", "urgent": true}',
    '```json\n{"category": "wrong_code", "urgent": false}\n```',
    'Sure! Here is the label: {"category": "door_fault"}',
    '{"category": "broken door", "urgent": true}',
]
for raw in replies:
    try:
        print("OK   ", parse_label(raw))
    except ValueError as err:
        print("RETRY", err)

OK    {'category': 'door_fault', 'urgent': True}
OK    {'category': 'wrong_code', 'urgent': False}
RETRY not valid JSON: Expecting value
RETRY category 'broken door' is not one of ['door_fault', 'late_delivery', 'other', 'wrong_code']

The two RETRY lines are the interesting ones. The chatty reply fails at json.loads, and the invented category fails the allowed-list check. In production a failure like that triggers one retry in which you send the error message back ("your reply was not valid JSON: ..."), and after that a safe fallback such as routing the message to a human.

Common mistakes and the symptom each one causes

• Every example has the same label. Symptom: the model labels almost everything with that label. Fix: balance the labels.
• Examples contradict the instructions. Symptom: the model follows the examples and ignores the rule (urgent marked true when the rule says otherwise). Fix: check each example against your own rules.
• Examples in slightly different formats. Symptom: the reply format wobbles between runs, sometimes losing a key. Fix: make them byte-for-byte consistent in structure.
• No list of allowed values. Symptom: categories such as "broken door" appear and a downstream lookup fails with a KeyError. Fix: enumerate the labels and validate.
• Asking for JSON but allowing chatter. Symptom: json.JSONDecodeError ("Expecting value") when the reply starts with "Sure! Here is...". Fix: say "JSON only", strip fences defensively, retry once.
• Too many examples. Symptom: slower, costlier calls with no gain. Fix: add examples only while the test set keeps improving.

How this shows up in real projects and interviews

Classification, extraction and routing are among the most common uses of models in business software, and almost all of them are a few-shot prompt plus a validator exactly like this. In an interview you may be asked how you would stop a model from returning free text when your code expects an enum. A good answer covers examples, an explicit allowed list, provider-side schema features if available, validation in code, a bounded retry and a fallback. Mentioning that you keep the examples under version control and rerun your test set when you change one shows engineering maturity. A cheaper alternative worth naming: for a stable, high-volume classification task, a small purpose-trained classifier may beat any prompt; see machine learning fundamentals.