🧠 Prompt Engineering Mastery Pack — 30 Structure-First Prompts for Production LLM Use

Your Prompt Engineering Session — Structure-First Analysis

Diagnosis

Your current prompt fails predictably in four specific ways:

1. No input schema — the prompt doesn't tell the LLM what 'issue type' categories are valid. 'Billing,' 'technical support,' 'return request' is one categorization; 'urgent/complaint/inquiry' is another. LLM picks randomly depending on the email.

2. No output schema — 'Return as JSON' is not a schema. The LLM guesses field names, types, and structure. Some responses return `urgency_rating` as a number (correct), some as string ('3'), some as ordinal ('high'), some as scale ('3/5').

3. No urgency definition — 'urgency 1-5' means different things to different LLMs without calibration. A 'wait 3 weeks for response' issue might be rated 1 or 3 depending on run.

4. No examples — zero-shot extraction on a 6-field structured output is notoriously unreliable. Few-shot examples (just 2-3) typically move accuracy from ~80% to ~95% on extraction tasks.

5. No validation step — the LLM isn't asked to verify it extracted all fields. Partial extractions slip through as '80% correct.'

This isn't a 'GPT-5 is inconsistent' problem — it's a prompt design problem. The fix is structural.

The Structural Fix

Replace the current prompt with this XML-structured version:

<role>
You are a customer support email extraction system. You extract structured data from customer emails and return it in strict JSON format matching the provided schema. You do not generate prose, commentary, or explanations — only the JSON object.
</role>

<task>
Extract six fields from the customer email below. Return a JSON object exactly matching the schema provided. If a field is unclear or missing from the email, use 'unknown' (for strings) or null (for numbers) — do not guess.
</task>

<output_schema>
{
  "customer_name": "string — as found in email signature or 'unknown'",
  "product_mentioned": "string — specific product name or SKU, or 'unknown' if none mentioned",
  "issue_type": "one of: 'billing_issue' | 'technical_support' | 'return_request' | 'product_inquiry' | 'account_access' | 'other'",
  "sentiment": "one of: 'positive' | 'neutral' | 'negative' | 'very_negative'",
  "urgency": "integer 1-5 based on calibration below",
  "requested_resolution": "string — concise description of what customer wants, or 'unspecified'"
}
</output_schema>

<urgency_calibration>
1 = Informational inquiry, no urgency. Example: 'Wondering about feature X.'
2 = Standard request, within normal SLA. Example: 'Please update my shipping address.'
3 = Time-sensitive but not blocking. Example: 'Order confirmation would help before weekend.'
4 = Urgent — active impact on customer. Example: 'Your service is down, my team can't work.'
5 = Critical — escalation or compliance concern. Example: 'I'm reporting this to my bank/BBB/attorney.'
</urgency_calibration>

<examples>
<example_1>
<email>
Hi team, this is John Martinez (order #A4821). I bought the Pro subscription last week and was charged $99 but the features aren't showing up in my dashboard. Can someone fix this by Friday? I have a client demo Monday.
- John
</email>
<output>
{
  "customer_name": "John Martinez",
  "product_mentioned": "Pro subscription",
  "issue_type": "billing_issue",
  "sentiment": "negative",
  "urgency": 3,
  "requested_resolution": "Activate Pro subscription features by Friday for Monday client demo"
}
</output>
</example_1>

<example_2>
<email>
Your product is absolutely broken and has been for 3 days. My entire sales team can't access their accounts. We pay $2000/month for this and I'm about to cancel and report this to my credit card. Respond immediately.
</email>
<output>
{
  "customer_name": "unknown",
  "product_mentioned": "unknown",
  "issue_type": "account_access",
  "sentiment": "very_negative",
  "urgency": 5,
  "requested_resolution": "Immediate restoration of team account access; retention at risk"
}
</output>
</example_2>
</examples>

<email>
[insert customer email here]
</email>

<output>

Key structural changes:

• Explicit role as an extraction system (not a 'helpful analyst')
• Output schema with enum constraints on categorical fields
• Urgency calibration with examples (anchors rating to meaning)
• Two few-shot examples covering easy and hard cases
• Explicit 'unknown/null for missing fields' rule (prevents hallucination)
• Asks for JSON object only (no prose)

Few-Shot Examples (why these specifically)

The two examples I included handle different failure modes:

• Example 1 (John Martinez): standard case with name + product + moderate urgency. Teaches the model the 'happy path' extraction.
• Example 2 (anonymous rage email): no name, vague product, extreme sentiment, max urgency. Teaches the model to handle missing-field cases and calibrate urgency 5 correctly (mentions of 'reporting,' 'cancel,' 'immediate' are the signals).

For production, add 2-3 more examples covering: (a) neutral inquiry with no product mentioned, (b) billing error with dollar amount specified, (c) positive feedback with feature request (edge case: 'positive' sentiment doesn't mean 'issue_type = other' automatically).

Platform Calibration

You're on GPT-5-mini. Specific adjustments:

• GPT-5 models generally respect JSON Mode (set response_format: {type: 'json_object'} in the API call). Use this in addition to prompt instruction — it enforces valid JSON output at the API level.
• For structured extraction, GPT-5 Thinking > GPT-5 base > GPT-5-mini on accuracy. GPT-5-mini is 4x cheaper but may require more few-shot examples (5-7 instead of 2-3) to hit 95%.
• Temperature 0 for extraction — you want deterministic outputs, not creative ones. Current GPT-5 API default is usually 1.0; explicitly set to 0.
• If you need to migrate to Claude: switch the XML tags to Claude's preferred <...> style (you're already using them — good). Claude 3.5 Sonnet and Opus 4.7 both excel at XML-structured extraction. Often Claude + XML outperforms GPT-5 on extraction tasks for equivalent cost.

Evaluation Plan

Before shipping this to production, build an eval set:

1. Collect 30-50 real customer emails from your support queue covering:

- 5 easy cases (clear product, clear issue, normal urgency)

- 15 medium cases (some ambiguity in one or two fields)

- 10 hard cases (missing info, ambiguous sentiment, borderline urgency)

- 5 adversarial cases (trying to trick the system — prompt injection attempts, emails in multiple languages, unusual formatting)

2. Hand-label the correct output for each. This is your ground truth. 2-3 hours of work.

3. Run your prompt V1 against all 50 cases. Measure:

- Exact match accuracy per field (customer_name exact match, issue_type exact match, etc.)

- Overall full-record accuracy (all 6 fields correct)

- Your target: 95%+ full-record accuracy

4. Make V2 based on failure patterns. Is sentiment consistently wrong? Add sentiment examples. Is urgency over-inflating? Tighten calibration rules.

5. A/B test V1 vs V2 on the eval set. Pick V2 only if it improves accuracy without degrading on any dimension.

Iteration Path

V1 (above): ship with JSON Mode + temperature 0. Expect ~90-93% full-record accuracy on the eval set.

V2 (add): chain-of-thought reasoning for urgency + sentiment.

<reasoning>
Before outputting the JSON, think through:
1. What is the customer's tone?
2. What signals of urgency are present (time pressure, escalation language, consequences)?
3. What's the specific product/feature mentioned?
Then output the JSON.
</reasoning>

Trades ~200 extra output tokens for typically 3-5% accuracy lift on reasoning-heavy fields.

V3 (if still needed): split into two-call architecture. Call 1 extracts name/product/issue_type (factual). Call 2 assesses sentiment/urgency (judgment) with call 1's output as context. 2x cost for usually 2-3% accuracy lift. Only if V2 isn't hitting 95%.

Red Flags

1. 'It works 80% of the time' without an eval set is 'I've looked at 10 cases and they seemed fine.' Build the eval set FIRST — you'll be shocked at how wrong your intuition is.

2. The phrase 'Act as a helpful expert' in your original prompt. This kind of role-adjective stack doesn't improve accuracy; it just burns tokens.

3. Missing enum constraints on issue_type. 'Issue type' as free text means the LLM invents categories ('Customer Feedback', 'Urgent Issue', 'Support Request'). Enum constrains it.

4. Missing urgency calibration. '1-5' is ambiguous. Anchoring to specific example scenarios makes it consistent.

5. Shipping on subjective impression. 'Looks good to me' is not an eval. Structured accuracy measurement is.

Key Takeaways

• Your 80% → 95% gap is a prompt design problem, not a model problem. Specifically: no schema, no calibration, no examples, no validation.
• XML-structured prompts with explicit input/output schemas outperform adjective-heavy prose. Switch the structure.
• Few-shot examples (2-3 for start, 5-7 for GPT-5-mini) are the biggest single improvement for extraction tasks. Add them.
• Enum constraints on categorical fields prevent the LLM from inventing categories. issue_type as 'billing_issue | technical_support | ...' beats free-text 'issue_type'.
• JSON Mode + temperature 0 for GPT-5 extraction. Enforces structured output at the API level.
• Build an eval set of 30-50 real cases with hand-labeled ground truth. Measure full-record accuracy, target 95%+.
• Iterate V1 → V2 based on failure patterns, not intuition. Usually 2-3 iteration cycles hit 95%+ from a solid V1.
• Platform-calibrate: if GPT-5-mini can't hit 95% after 3 iterations, upgrade to GPT-5 base or try Claude 3.5 Sonnet. Sometimes the model is the limit.