The AI Testing Prompt Pack: 18 Prompts to Validate

Act as a user who has never used a product like [your product] before. 
You are not technical. You don't know what [core feature] is supposed to do. 
You've just landed on the product for the first time with a vague goal: [describe user goal in plain language].

Walk me through what you try first, what confuses you, what you assume incorrectly, 
and where you give up or ask for help. Be specific. Don't be charitable to the product.

Product context: [paste your onboarding copy or core feature description here]

I'm going to describe an AI product. Your job is to identify the three most likely 
ways users will try to use it for something it wasn't designed for.

For each misuse, describe:
- What the user was actually trying to accomplish
- Why this product looked like the right tool to them
- What happens when they try it and where it breaks

Product description: [paste your one-paragraph product description]
Core intended use case: [describe the primary job-to-be-done]

You are a senior product manager at a company that tried an AI product similar to [your product] 
and had a bad experience. You're evaluating this product with high skepticism.

Review the following product description and identify:
1. The three claims that feel unsubstantiated or vague
2. The two scenarios where this product would likely fail you
3. The one question you'd ask in a demo that would reveal whether the product is real or vaporware

Product description: [paste landing page or core feature copy]

I'm going to give you a system prompt or product description. 
Your job is to find the five places where the instructions assume context 
that a real user won't have.

For each gap, explain:
- What context the product assumes the user has
- What a realistic user actually knows at this point in the flow
- What goes wrong when the assumed context is missing

System prompt / product description: [paste here]

You are a QA engineer who specializes in breaking AI products. 
Your goal is to generate 10 adversarial inputs for the following product feature.

For each input, explain:
- Why a real user might reasonably send this
- What the ideal product response would be
- What a poorly-designed product would do instead

Feature description: [describe the feature and its intended input/output]
Intended user type: [describe who uses this feature]

Generate 8 test inputs for the following AI feature that represent edge cases 
in language, locale, and communication style:

- 2 inputs from non-native English speakers using common grammatical patterns from [language 1] and [language 2]
- 2 inputs with heavy use of industry jargon from a domain your product wasn't designed for
- 2 inputs that mix languages mid-sentence
- 2 inputs that use formal register where the product expects casual, or vice versa

For each input, note what the product needs to handle gracefully 
and what a failure looks like.

Feature: [describe the feature]

For the following AI product feature, generate three batches of test inputs:

Batch 1 - Empty or near-empty inputs (5 examples):
Inputs where the user gives almost no information but still expects a useful response.

Batch 2 - Ambiguous inputs (5 examples):
Inputs that could reasonably be interpreted in three or more different ways.

Batch 3 - Contradictory inputs (5 examples):
Inputs where the user's stated goal conflicts with another stated constraint or preference.

For each batch, describe what the product should do and what it typically does wrong.

Feature: [describe the feature and its expected input format]

Simulate a user who has been using [your product] for 45 minutes across a complex task. 
They've changed their mind twice, given conflicting instructions, and are now on step 7 
of a 10-step workflow.

Generate a sequence of 12 inputs that represent this session realistically, 
including the mid-session pivots. Then evaluate:
- Where does context likely get lost or corrupted?
- Where does the product give a response that was correct for an earlier state but wrong now?
- What does the user experience when this happens?

Product: [describe your product]
Task scenario: [describe a realistic complex task your users do]

I'm going to give you 5 semantically similar inputs for the same AI product feature. 
Your job is to predict where the outputs will be inconsistent in a way that would 
confuse or frustrate users.

For each pair of similar inputs, identify:
- What a consistent product should do the same way both times
- What a consistent product might reasonably do differently
- What an inconsistent product does that would feel arbitrary to users

Inputs:
1. [input variant 1]
2. [input variant 2]
3. [input variant 3]
4. [input variant 4]
5. [input variant 5]

Feature context: [describe the feature]

For the following AI product and use case, identify the five specific input types 
most likely to produce hallucinations or fabricated information.

For each risk area:
- Describe the input pattern that triggers it
- Explain why this pattern is particularly likely to produce fabrication
- Suggest a guardrail or response pattern that would reduce harm without 
  making the product useless

Product: [describe your product]
Core use case: [describe the primary task users perform]
Data sources the product has access to: [describe what the product knows]

Review the following sample outputs from an AI product and audit them for 
tone, register, and voice consistency.

For each output, evaluate:
1. Is the tone appropriate for the context and user state?
2. Does the register (formal/casual/technical) match what the user used?
3. Are there any phrases that feel robotic, evasive, or generically AI-generated?
4. What would a human expert in this domain say instead?

Outputs to review:
[paste 3-5 real outputs from your product here]

Intended product voice: [describe the voice and tone guidelines for your product]

I'm testing whether an AI product's outputs are calibrated correctly between 
being too brief (unhelpfully thin) and too verbose (unusable walls of text).

For each of the following inputs, evaluate what the ideal output length and 
structure should be, then compare to what the product actually produces.

Test inputs:
1. A simple factual question about [your domain]
2. A request for a recommendation with three constraints
3. A complex multi-part question with no clear right answer
4. A follow-up question to a previous answer (simulate this)
5. An urgent, time-sensitive request

Product outputs: [paste actual outputs for each scenario]

I'm going to give you a user complaint about an AI product. 
Your job is to work backward and identify the most likely root causes.

For each complaint, generate:
1. Three hypotheses for what caused this behavior (from most to least likely)
2. A reproduction path: the specific input sequence most likely to trigger this
3. The fix category: is this a prompt issue, a context issue, a model limitation, 
   or a product design issue?

User complaint: [paste actual complaint or support ticket]
Product context: [describe what the product is supposed to do in this scenario]

For the following AI product feature, identify the five failure modes that 
would be hardest for users to detect, report, or attribute to the product.

These are failures where:
- The output looks plausible but is wrong in a non-obvious way
- The user blames themselves rather than the product
- The error compounds over multiple steps before becoming visible

Feature: [describe the feature]
User task: [describe what users are trying to accomplish]

For each silent failure, describe: the mechanism, a realistic example, 
and how you'd instrument the product to catch it.

Test how an AI product behaves when it doesn't know something, can't do something, 
or encounters a situation outside its design.

Generate 6 test scenarios where the product should gracefully decline or redirect:
- 2 scenarios where the request is outside the product's scope
- 2 scenarios where the product lacks the information to answer reliably
- 2 scenarios where completing the request would likely harm the user

For each scenario, describe:
- What a graceful response looks like
- What a typical AI product does instead (deflects generically, hallucinates, 
  completes anyway)
- How to evaluate whether your product handles this well

Product: [describe your product and its defined scope]

I'm going to show you an AI product. After reviewing it, complete these tasks:

1. Write one sentence describing what this product does, from the perspective of a user 
   who just successfully completed their first task.
2. Write one sentence describing what this product does, from the perspective of a user 
   who just failed at their first task.
3. Write one sentence describing what this product does, from the perspective of a user 
   who's been using it for three months.

Then: identify the gaps between these three sentences and what your intended 
product positioning says.

Product description and sample outputs: [paste here]
Intended positioning: [paste your core value prop]

We launch [product or feature] in [timeframe]. Before we do, I want to predict the
support load instead of discovering it.

Product: [describe what it does and who it's for]
What's changing at launch: [new product, new feature, changed behavior]
How users will find it: [onboarding, announcement, in-product surface]
Known rough edges we're shipping with: [be honest, list them]

Generate:
1. The ten questions users are most likely to ask in the first week, ordered by volume.
2. For each, whether we currently have an answer a support person could send today.
   Mark each: documented / answerable but undocumented / no good answer exists.
3. The three complaints most likely to be framed as "it's broken" when the product
   is working as designed. These are design problems wearing a support costume.
4. The one question that, if it shows up in volume, means we should pause the rollout.

Don't give me generic support advice. Base every prediction on the specific
product described above.

We're about to launch [product or feature]. Before it goes live, I want to define
what "this is going badly" looks like in numbers, while I can still think clearly
about it.

The launch: [describe what's shipping and to whom]
What we're measuring today: [list your current metrics and instrumentation]
What we believe success looks like: [describe the expected outcome]

Help me define the kill criteria:
1. For each of the following, give me a specific threshold that should trigger a pause
   or rollback, not a discussion: error and failure rate, task completion rate,
   support ticket volume, and any quality signal specific to this product.
2. For each threshold, tell me whether we can currently measure it. Flag the ones
   where we'd be flying blind.
3. What's the failure mode that our current instrumentation would not catch at all?
4. How long do we wait before acting on a bad signal, and who decides? Write it as
   a rule someone can follow at 2am without calling a meeting.
5. What does rolling this back actually cost, and does that cost quietly make the
   rollback impossible? If so, say that plainly now.