Test support-agent updates on simulated customers first: run multi-turn chats with synthetic buyers, keep only the versions that cut unnecessary handoffs, then A/B the winner with real people: AI agent prompt
Nubank (Nov 5, 2026) reports that its AI Research team cut average AI support-agent update cycles from 21.2 days to 4.4 days by testing new versions against synthetic customers in large-scale simulation before any live rollout. The paper “Screen Before You Serve: Simulation for Production Customer Experience AI Agents at 140M Scale” describes multi-turn conversations with synthetic customers that have different profiles and needs. Simulation guided changes to a Card Management agent (address updates and customer-requested reissues); offline, one version cut unnecessary human handoffs from 22.4% to 16.0%. In a live A/B test against a Card Delivery agent, the simulation-built agent showed a 4.9-percentage-point higher self-service rate and a 36.7-point higher transactional NPS. The team also screened 29 open-weight model configurations across more than 16,000 simulated conversations before refining a Qwen setup that later raised self-service 8.8 points and cut p95 latency 25% in a live A/B. Nubank notes self-service alone does not prove every issue was resolved, and that simulators must be validated before teams rely on them. The prompt below gives any team a smaller version of that screen-before-you-serve loop.
The prompt
Help me screen a support-agent change on simulated customers before real ones see it. Agent job: [e.g. card delivery status, address updates, returns]. Change I'm testing: [prompt edit, new model, new tool, new handoff rule]. Success criteria set in advance: [e.g. fewer unnecessary human handoffs, no drop in satisfaction, latency under X]. 1) Invent 20 synthetic customers with different profiles and needs (frustrated, first-time, policy edge case, unclear ask, escalation-seeker). For each, write a multi-turn chat script the agent must handle. 2) Run (or simulate) both the current agent and the new version on every script. Score each: resolved without human, wrong or unsafe answer, unnecessary handoff, missing clarification. 3) Rank the change: ship, revise, or kill. Only recommend ship if it clearly beats the current version on the success criteria and does not invent policy. 4) List the 5 hardest failure chats to turn into permanent regression tests. 5) Draft a short live A/B plan: who sees the new agent, what we measure, and when a human pulls it. Mark anything I still need to verify with real customers as [CHECK]. Do not invent product policy; ask me for the real FAQ or policy text.
Fill in the blanks, then copy
This prompt has 3 [bracket]s to fill in. Type your details below and the prompt updates as you go. Your answers stay in this browser and are reused on other prompts with the same blanks. Nothing is sent to us.
Example values are fictional, just to show how a filled-in prompt reads. Replace them with your real details.How to use it
- Copy the prompt above.
- Replace every [bracket] with your own details (use the fill-in-the-blanks box above, which remembers your answers for the next prompt).
- Paste it into ChatGPT, Claude, Gemini and review the answer before you use it.
Works with: ChatGPT, Claude, Gemini, your support-agent platform, spreadsheets for scoring
Real-world example: nu.com/en/newsroom/company/nubank-cuts-ai-agent-update-cycles-from-thr
Featured in: AI prompts to summarize meetings, notes, documents and reports · AI prompts for writing SOPs, reports, policies and checklists
Before you use it: what to check
Already built into this prompt
- fewer unnecessary human handoffs, no drop in satisfaction, latency under X].
- Score each: resolved without human, wrong or unsafe answer, unnecessary handoff, missing clarification.
- Do not invent product policy; ask me for the real FAQ or policy text.
Check yourself
- Remove names, contact details, account numbers and other personal data the task doesn't need before you paste, and check your AI tool's data and training settings.
- Check names, dates, numbers and any claim against your own source material before you use the answer. AI can misread or invent details.
Using an AI agent? Download this prompt as a SKILL.md agent skill with these checks built in, or get the .zip for Claude.ai. How to install skills.
Want to make this prompt even better? Grade it with the free prompt grader or build your own with the AI prompt generator.
32 copy-paste agent prompts (PDF) plus 5 new prompts every week.
Get it freeMore Customer Support prompts
See all Customer Support prompts →