Example page made by Ghostlaunch. This product doesn't exist yet.
Back to the exampleRegression checks for prompts and skills
Quietdiff replays your past cases on every pull request and flags changed tool calls, skipped steps and tone drift, so leads can approve consultants' changes with evidence.
14-day free trial. Your cases stay in your repo.
Prompts, skills and agent workflows change every week, often written by consultants who aren't engineers. One lead signs off, and the regression shows up later, when a client hits it.
"I'm basically eyeballing diffs of English text and hoping."
"I shipped a tone change and have no idea if I made it worse."
"Our golden set doesn't catch the agent calling a different tool."
"Skill vs no skill tells me nothing about the content."
"Our consultants can't write eval code, so changes go out untested."
No eval code to write. Quietdiff works where changes already get reviewed.
Add Quietdiff to the repo where your prompts, skills and agent workflows live. It runs on every pull request.
Keep real past conversations and tasks as cases in your repo. Consultants add new ones by saving examples, not writing code.
Quietdiff replays the cases on the old and new version and posts what changed in behavior. Approve or bounce with evidence.
Quietdiff compares how the agent behaves, not just what it says.
Each replay records tool calls, steps taken and tone, then compares old and new versions case by case. Reworded but equivalent answers don't trigger false alarms.
Quietdiff also runs a control with neutral filler text of the same length, so you see whether the instructions help or just their presence does.
The GitHub app runs on each pull request and posts a report as a comment. Nothing depends on someone remembering to test.
Cases are saved examples from real work. Anyone who writes prompts can add one, and the lead stops owning a test codebase.
Every flagged case shows the before and after side by side: which tool changed, which step was skipped, where tone drifted.
Cases never leave your repository for storage. Quietdiff keeps only the reports, and deletes them after 30 days.
Built for
Tech leads at AI consultancies
You review prompt and skill changes from consultants before they reach client deployments, and you're the only check in between.
Internal AI teams with domain experts
Ops staff and subject experts write the prompts. You need to know a change is safe without reading every line.
Maintainers of shared skill libraries
Skills get reused across agents and clients. One tone tweak can quietly change behavior in places you don't watch.
Probably not for
Most teams rely on a lead reading diffs or a golden set of expected outputs. Both miss behavior changes.
| Criteria | Quietdiff | Manual diff review | Text-only golden set |
|---|---|---|---|
| Catches changed tool calls and skipped steps | Yes | No | No |
| Flags tone drift across many cases | Yes | Partly | No |
| Shows if skill content matters vs its length | Yes | No | No |
| Runs on every pull request | Yes | No | Partly |
| Tolerates legitimate rewording | Yes | Yes | No |
| Scales past a few changes a week | Yes | No | Yes |
Why I'm building Quietdiff
The people writing our prompts and skills are mostly technical consultants, not full-time engineers, and I'm the review layer before anything gets merged and used in the field. I recently shipped a tone change to a skill I maintain and wanted to know whether I'd made it worse. I couldn't. Our text-only golden set wouldn't catch it, and 'skill vs no skill' can't tell you whether the content does anything. The thing nobody warns you about isn't price or rate limits. It's the quiet regression. Quietdiff is the check I wanted on every pull request.
— The founder of Quietdiff
Start with a 14-day trial on your own repo, then continue on the Team plan.
14 days on your own repo, to see what your recent changes really did.
Free
For leads who approve prompt and skill changes they didn't write.
$99/ month
Priced per team, not per seat. Cancel anytime.
Cases stay in your repository. Quietdiff reads them when a pull request runs and stores only the resulting reports, which are deleted after 30 days.
Those are eval frameworks for engineers who write assertions and test code. Quietdiff needs no eval code: it replays saved cases on the old and new version and reports what changed in behavior, right in the pull request.
A text-only golden set compares final strings. It misses a different tool call or a skipped step, and it fails whenever wording legitimately changes. Quietdiff compares tool calls, steps and tone instead.
Alongside your change, Quietdiff runs a version with neutral filler text of the same length. If behavior shifts the same way, the skill's content isn't what's doing the work.
No new tools. They open pull requests as they do today and can add a case by saving a real example to the cases folder. The report appears on their PR.
Install the GitHub app on your repo and point it at a folder of past cases. If you already have example conversations, the first report can run on your next pull request.
$99 per month per team after a 14-day free trial. No per-seat pricing.
Get a behavior report on every prompt, skill and agent change before you hit merge.