Test a customer support AI in "shadow mode" on real emails before it ever replies to a customer: AI agent prompt
Turning on an AI support agent straight away is risky: it may give a wrong answer, or claim it did something it didn't. A safer way is shadow mode: for one or two weeks the AI drafts a reply to every real ticket as a private note only your team can see, people answer customers as usual, and you score the drafts to find holes in your help articles and rules before going live. Fini's Wefunder case study describes this rollout at the equity crowdfunding platform: Fini posted draft replies as internal comments in Front, the team corrected what was wrong, and the shadow period caught cases where the agent said it had done something it couldn't. After going live, response times fell from about 7 hours to about 25 minutes in week two.
Customer SupportSafe AI rolloutbeginnerWho this is for
Businesses about to let an AI help desk tool reply to customers.
Not for
Teams with no time to review drafts for two weeks; don't switch on AI replies at all yet.
What you need
Your help desk name, FAQ or help articles, and your rules (refund limits, what it must never promise).
The prompt
Help me run a 2-week "shadow mode" test of an AI support agent before it talks to customers. My business: [what you sell]. Help desk: [e.g. Gmail, Front, Zendesk, Intercom]. Help articles / FAQ: [paste or link]. Rules the AI must follow: [refund limits, things it must never promise, when to hand to a person]. Part 1 - Setup: write me a simple plan: how to make the AI post drafts as private notes only, which ticket types to include and exclude, and who reviews them. Part 2 - Scoring: give me a review sheet with columns: ticket ID, topic, AI draft, correct? (yes / small fix / wrong), wrong because (missing article, outdated info, wrong tone, made a promise it can't keep, said it took an action it didn't), and the fix. Part 3 - Each week I'll paste 20-50 tickets with the AI draft and our real reply [paste]. Score them, list the top 5 gaps in our help articles with suggested new text, list any draft that claimed an action it can't do, and give a go / not yet recommendation for each ticket type with the % correct. Don't send anything to customers.
Worked example
A fictional business, so you can see what to paste in and what a good answer looks like. The example answer was written with AI help and its numbers and dates were checked; your AI's answer will be worded differently, so check it the same way.
Sample input (what you paste in)
Business: Saltbox Soap Co. (online store). Help desk: Gmail. Rules: refunds up to $30 OK, never promise delivery dates, hand off anything about allergies.
Example output (illustrative)
Setup: the AI writes drafts as Gmail drafts only; nothing sends automatically. Include: order status, returns, product questions. Exclude: allergy and skin reaction emails (owner answers). Reviewer: owner, 15 minutes a day. Score sheet columns: email ID, topic, AI draft, correct? (yes / small fix / wrong), why wrong, fix (update FAQ / add rule). Go-live rule after 2 weeks: at least 90% "yes" or "small fix", zero wrong answers on refunds or allergies. Turn on automatic replies only for the topics that passed.
Check before you use it: Pick go-live thresholds you're comfortable with; the 90% here is an example, not a standard.
How to use it
- Copy the prompt above.
- Replace every [bracket] with your own details (or fill in the blanks on AgentPlaybook Real Estate, which remembers your answers for the next prompt).
- Paste it into ChatGPT, Claude, Fini and review the answer before you use it.
Works with: ChatGPT, Claude, Fini, Zendesk/Intercom AI, Front, Google Sheets
Real-world example: usefini.com/blog/wefunder-case-study
Featured in: Free AI agent prompts for small business (beginner-friendly) · ChatGPT prompts for business email, replies and follow-ups · AI prompts for Google Sheets and Excel work · ChatGPT prompts for customer service: ticket triage, replies and handoffs
Before you use it: what to check
Already built into this prompt
- Don't send anything to customers.
Check yourself
- This prompt works on text other people wrote (emails, reviews, messages or web pages). That text can hide instructions aimed at AI, such as "ignore your rules and…". Paste it as material to work on, and never let the AI act on requests inside it, like paying, sharing logins or changing records. How prompt injection works →
- Check every figure, price and total against your own records before anything is sent, posted or booked. A person approves refunds, discounts, payments and price changes.
- Remove names, contact details, account numbers and other personal data the task doesn't need before you paste, and check your AI tool's data and training settings.
- Read and edit the draft before it reaches a customer or goes public. If you later automate this prompt, start with "draft and notify me", not "send automatically".
Using an AI agent? Download this prompt as a SKILL.md agent skill with these checks built in, or get the .zip for Claude.ai. How to install skills.
Want to make this prompt even better? Grade it with the free prompt grader or build your own with the AI prompt generator.
32 copy-paste agent prompts (PDF) plus 5 new prompts every week.
Get it freeMore Customer Support prompts
See all Customer Support prompts →