Why model changes need a review card
An AI workflow can behave differently after a vendor silently updates a model, a team switches providers, a prompt is rewritten, retrieval sources change, or temperature/settings are adjusted. The change may look small in the tool UI but still alter tone, facts, formatting, escalation behavior, privacy handling, or customer promises.
STOP AUTOMATION: do not let AI approve, deploy, expand, or hide a model/provider/settings change. A named human owner must compare before/after outputs, verify source facts, review customer-impact risks, and keep a rollback path before the workflow runs on real customer or business records.
Model-change review card
| Field | What to capture | Owner review rule |
|---|---|---|
| Workflow affected | Workflow name, owner, tools connected, customer/internal use, launch date, and where outputs are used. | Review only one workflow at a time; do not approve a blanket model switch across the business. |
| Change details | Old model/provider/version/settings, new model/provider/version/settings, prompt version, retrieval/source changes, and date changed. | Record the exact change before testing; vague “new AI version” notes are not enough. |
| Before/after test set | At least 5 real or realistic cases: normal, edge, missing-data, sensitive-data, and customer-facing examples. | Use approved sample data only. Do not paste private customer records unless the owner has approved that use. |
| Output comparison | Facts, tone, formatting, required disclaimers, escalation behavior, CRM labels, and missing-data handling. | Any invented price, deadline, policy, approval, diagnosis, refund, guarantee, or customer history triggers STOP AUTOMATION. |
| Customer/business impact | Who could see the output, what could go wrong, whether review workload changes, and whether staff need updated instructions. | Customer-facing changes need human review and a rollback message before going live. |
| Decision and rollback | Keep, fix, pause, revert, or pilot-only decision; rollback owner; correction notes; next review date. | No production rollout without a named owner and rollback path. |
Copy/paste test cases
Normal case: Use approved source facts to draft [workflow output]. Confirm the new model preserves required tone, fields, disclaimers, and review labels.Missing-data case: Source facts are incomplete for [price/date/policy/approval/customer detail]. The AI must ask for owner review and must not invent the missing fact.Sensitive-data case: The input includes [customer/private/internal detail]. The AI must avoid exposing unnecessary private data, avoid new assumptions, and route to [owner/reviewer] before customer-facing use.Rollback note: We paused/reverted [workflow/model change] because [issue]. Use the previous approved prompt/model until [owner] verifies corrected outputs on [test set].AI review prompt
Act as a cautious small-business AI workflow QA reviewer. Compare the old-output notes and new-output notes below. Use only verified source facts. Do not invent model capabilities, vendor claims, customer history, prices, policies, approvals, legal advice, or performance results. Return: 1) behavior changes, 2) fact/source risks, 3) customer-impact risks, 4) review-workload changes, 5) missing tests, and 6) a keep/fix/pause/revert recommendation with STOP AUTOMATION if the model change is not ready.
Workflow:
Old model/provider/settings:
New model/provider/settings:
Prompt/retrieval/source changes:
Approved source facts:
Before-output notes:
After-output notes:
Known incidents/corrections:
Rollback owner:Fast QA before rollout
- Run the same test cases through old and new settings before trusting the new output.
- Check missing-data behavior; the model should ask for review instead of filling gaps.
- Verify customer-facing wording for tone, disclaimers, fees, policies, timing, warranties, and escalation paths.
- Keep a correction log for any output that changed in a risky way.
- Do a pilot with human review before expanding the model change to every user or workflow.
Related free assets: AI Workflow Test Case Checklist, AI Workflow Post-Launch Review Checklist, and AI Output Correction Log.
Disclosure: Horizon Flow is Andrew Burton's digital product catalog. This worksheet is useful without purchase; product links are labeled and UTM-tagged.