Acceptance Criteria for AI Agents: 3,375 Criteria Checked

On this page
  1. Acceptance criteria are growing faster than ever
  2. What 3,375 criteria look like
  3. From a label to something an agent can prove
  4. Rules for criteria an agent can prove
  5. Make the agent show the proof
  6. Check criteria before the agent starts
  7. Sources

“Review form.” That is a whole acceptance criterion, copied from a public GitHub issue this month. A person on the team knows which form, what “review” means and when it is done. An AI coding agent will build a form, call it reviewed, and write a test that proves its own guess.

Acceptance criteria are suddenly everywhere. GitHub’s search counts about 41,000 new issues a day that mention them, 22 times the rate of a year before. Many are written for agents, some by agents. We checked 3,375 of them to see how many an agent could actually prove.

Short answer: an AI agent can only prove a criterion that names something a test can observe: a result, a number, a state, a “must”. In 3,375 criteria from 969 public GitHub issues, 39.6% had none of that, 0.9% used Given/When/Then and 16.7% had a number. Write each criterion as a check: “Given …, when …, then the API returns 400 with reason refund_window_expired”. Keep to about eight per work item, say what is out of scope, and ask the agent to finish with one line per criterion: the test that proves it, or “not proven”.

Acceptance criteria are growing faster than ever

Bar chart of public GitHub issues whose text contains acceptance criteria, per day, by the quarter they were created, from GitHub's issue search counts read on 18 November 2026: about 1,900 a day in July to September 2025, 3,000 in October to December 2025, 7,600 in January to March 2026, 15,100 in April to June 2026, 25,600 in July to September 2026, and 41,200 a day from 1 October to 17 November 2026, 22 times the rate of a year before. In 969 of those issues with a list of criteria, 3,375 criteria in all: a median of 3 per issue and 8 words each; 0.9 percent Given, When, Then; 16.7 percent with a number; 39.6 percent with no observable result, number or must.

GitHub’s issue search counts public issues whose text contains “acceptance criteria”. Per day, by the quarter they were created: about 1,900 a day in July to September 2025, 15,100 in April to June 2026, 41,200 from 1 October to 17 November 2026. The counts are GitHub’s own estimates and include issues created by tools; the shape is what matters. Since coding agents started taking issues, teams write criteria for them, and agents write criteria for each other.

What 3,375 criteria look like

How we sampled. On 18 November 2026 we took up to 200 issues from GitHub’s search for each of six periods since July 2025, 1,199 in all, found the acceptance criteria section (a heading, a bold line or “Acceptance criteria:”), and read its list items: 969 issues had a list, 3,375 criteria. We checked each with Agent Ready’s own wording rules: a vague-word list (“fast”, “user-friendly”, “etc.”, “appropriate” and 18 more) and a lenient test for “something an agent can check” (a verb of an observable result such as shows, returns, saves or rejects; a state such as is or stays; a number; “must”; or Given/When/Then).

What we measuredResult
Criteria per issue, median (90th percentile)3 (6)
Words per criterion, median8
Issues with checkboxes - [ ]67.1%
Criteria with no observable result, number or must39.6%
Issues with at least one such criterion64.2%
Criteria with a number16.7%
Criteria in Given/When/Then form0.9%
Criteria with a vague word1.2%
Issues with more than 8 criteria2.6% (25)
Placeholder criteria (“TBD”, “n/a”, ”…”)36, in 31 issues

Three things stand out. Criteria are short: eight words, three per issue. They are rarely vague in the classic sense: “fast” and “user-friendly” are almost gone. And the real problem is noun phrases: “Review form”, “Leader election pattern explained”, “Clickable tracking link”, “Required resources/approvals listed”. Each names a thing, not a result. A person fills the gap from context; an agent fills it with a guess.

Our pattern is a heuristic and it errs both ways: “Builds successfully” counts as untestable to it because “builds” is not on its verb list, and some criteria were in German or Indonesian. Read 39.6% as “a lot”, not as a precise figure.

From a label to something an agent can prove

Real criteria from the sample, and a version a test can check:

From a public issueProvable version
Review formGiven a submitted expense, when a manager opens it, then the form shows the amount, the receipt and Approve and Reject buttons
Leader election pattern explainedThe README section “Leader election” has a sequence diagram and covers what happens when the leader stops
Clickable tracking linkThe order e-mail contains a link that opens the carrier’s tracking page for that parcel
Required resources/approvals listedThe ticket lists every approver by role; the workflow blocks Start until all of them approved
Builds successfullymake check-all passes on the pull request

The pattern: name who or what starts it, what happens, and what you can see at the end, with the number if there is one. If you cannot write the last part, the work is not ready, for a person or an agent.

Rules for criteria an agent can prove

  1. One result per line. “Saves the draft and e-mails the reviewer” is two criteria, and an agent can prove one and claim both.
  2. Observable, not internal. “Uses the cache” is hard to test from outside; “the second request returns in under 100 ms” is not.
  3. Numbers everywhere they exist. Limits, rounding, retries, time zones, counts. 16.7% of public criteria had one.
  4. Boundaries. Day 30 and day 31; zero items and one; the empty string.
  5. Out of scope, written down. It stops the agent from “helpfully” doing the next ticket.
  6. About eight at most. More is several agent runs. Agent Ready’s Strict for agents preset flags a work item with more than 8 criteria or 800 words and drafts one sub-task per criterion.
  7. No placeholders. “TBD” under the heading is not a criterion; Agent Ready does not count it.

Given/When/Then is a good default when the situation matters. More examples for people, and the Jira side: acceptance criteria: 18 examples. The full readiness list, one story rewritten four times: Definition of Ready for agents.

Make the agent show the proof

A pull request that “meets the acceptance criteria” is a claim. Ask for the evidence instead, one line per criterion:

AC1 Given a payment older than 30 days ... returns 400
    proven by: refunds.test.ts › "rejects a refund on day 31"
AC2 Payments of 30 days or less refund as today
    proven by: refunds.test.ts › "refunds on day 30"
AC3 A test covers day 30 and day 31
    not proven: day 31 covered, day 30 only through AC2

Agent Ready makes this the agent’s last step. Through its MCP tools, an agent finishes a work item with finish-issue and a receipt: for each criterion the test that proves it, or “not proven”, plus what changed and the risks. The work item shows “2 of 3 criteria proven by tests” with a bar of one segment per criterion. The receipt is then checked against the pull request and builds Jira knows about: Verified, Unverified (no pull request yet) or Contradicted (a red build), checked again for 14 days. In our Claude Code runs, two of three work items finished with 4 of 4 criteria proven; the third stopped honestly because its criteria were placeholders.

Check criteria before the agent starts

The cheapest fix is before the handoff. Agent Ready checks every Jira work item for an acceptance criteria section with something under it, and, with Clear wording on, for vague words and criteria with nothing to test, using the same rules as this study. Write a criterion step by step turns three short answers into one “Given …, when …, then …” line, with no AI. On sites with Rovo, its agent drafts missing criteria from the ticket’s own text and marks what it does not know as questions. The Issue is agent-ready validator keeps a work item without criteria out of Ready for agent.

Sources

  • GitHub: issue search counts for “acceptance criteria” by creation date, read 18 November 2026 (GitHub’s estimates).
  • Our sample: 1,199 public GitHub issues from the same search, 969 with a list of criteria, 3,375 criteria, checked with Agent Ready’s wording rules (vague words and its testable-criterion pattern) on 18 November 2026.
  • Related: Rovo agents in Jira, tested (2 of 7 runs invented facts), Definition of Ready.

Frequently asked questions

How do you write acceptance criteria for an AI coding agent?

Write each one as something a test can check: a situation, an action and a result you can observe, such as "Given a payment older than 30 days, when a refund is requested, then the API returns 400 with reason refund_window_expired". Add the numbers (limits, counts, formats), keep to about eight criteria per work item, and say what is out of scope.

Can AI write acceptance criteria?

It can draft them from what the ticket already says, and you must check every number and name. In our test of Rovo agents on Jira, 2 of 7 runs invented facts when asked to fill criteria the ticket had no facts for. Let an agent draft, then a person who knows the work decides.

Do acceptance criteria have to be Given/When/Then?

No. A rule ("Totals round to two decimals") is fine when it names an observable result. Given/When/Then helps most when the situation matters. In our sample only 0.9% of criteria used it; far more common was a noun phrase like "Review form", which a test cannot check.

How do I know an agent met the acceptance criteria?

Ask it to prove each one: for every criterion, the test that checks it, or "not proven". Then check the claim against the pull request and its build. Agent Ready makes this the agent's last step through MCP, a receipt with one line per criterion, marked Verified or Contradicted against Jira's development information.

Keep reading