AI / Agentic
Prediction: Clear Acceptance Criteria Will Outlast Faster Code Generation
By Vivek Jalondhara · 5 min read
Published
A conditional forecast about agentic development, tested through a retry-rejected-rows feature and explicit counterexamples.
Prediction: if producing candidate implementations continues to require less effort, clear acceptance criteria will become a more valuable part of software work. The claim concerns the relative value of deciding and verifying behaviour. It does not assume that generation costs always fall or that every task benefits from an agent.
Consider a proposed Sheet Parser feature called Retry rejected rows. Two generated implementations could both look reasonable while disagreeing about which rows qualify and what happens to previously accepted data. The example below is a thought experiment, not a measured productivity study.
Name the decision that additional code cannot settle
Rejected might mean extraction failed, validation failed, or a reviewer deliberately rejected the proposed value. Retrying all three states is a product choice. So is deciding whether the retry uses the original source document or a replacement upload. Generating more implementation variants cannot establish the intended answer.
For this exercise, retry only validation-failed drafts from a named import revision, leave accepted rows unchanged, and create new candidates for review. Human-rejected rows require an explicit separate action. These are proposed requirements, chosen to make the feature's consequences reviewable rather than presented as universal rules.
Explain the mechanism behind the forecast
When many plausible implementations are available, the bottleneck may become comparing them against the same intent. A precise scenario can eliminate a variant that looks elegant but silently reprocesses accepted prices. That comparison remains useful regardless of which model or editor produced the code.
Anthropic's agent-evaluation guidance distinguishes an agent's account of its actions from the resulting state in the environment. Applied to this example, the relevant result is which drafts were created and which accepted records changed, not whether the assistant reports that the retry feature works.
Make the criteria concrete enough to disagree with
Build a fictional import containing an accepted row, a validation-failed row, and a human-rejected row. Request the retry for its original revision. The expected result is one new draft candidate for the validation-failed row, with the other two outcomes untouched.
Gherkin's context, action, and outcome structure is one useful expression of such cases. Add a second scenario where the source revision is unavailable and the operation returns an explainable failure. These examples make disagreement visible without specifying the exact queue, framework, or database layout.
Test the prediction instead of treating it as inevitable
Compare two task briefs for the same bounded exercise: one names only the feature, while the other includes those semantics and fixtures. Keep the available repository context and verification budget comparable. Record incorrect assumptions, review questions, and whether the final behaviour matches the reviewed requirement.
Do not interpret one successful run as a general productivity result. Repeated attempts, reviewer familiarity, task difficulty, and tool behaviour can change the outcome. The useful observation is whether explicit criteria reduce meaningful disagreement in this setting, including the effort required to write and maintain them.
Include situations where the forecast may not hold
Exploratory work may lack stable acceptance criteria because discovering the right interaction is the task. A rigid brief can lock the team into a poor solution. An existing mature suite may already encode enough intent that another written layer contributes little. Some defects require specialist investigation beyond example-based specifications.
Agents may also improve at recovering implicit requirements from repositories, while code generation itself may remain a substantial cost for difficult changes. These are countercases to the forecast. They suggest measuring clarity and review outcomes rather than assuming specification work must expand for every change.
Adopt the useful practice without waiting for the future
Write down the few decisions whose alternatives would materially change the feature. Use representative scenarios, preserve room for discovery, and update the criteria when the intended behaviour changes. This helps today's reviewers even if the broader prediction proves too strong.
The tradeoff is specification effort against ambiguity and rework. More pages do not necessarily mean more clarity. A short, maintained set of cases can remain valuable as tools change because it explains what the product must do and gives the next implementation a fair way to demonstrate it.
A criterion that makes retry semantics observable
This practical acceptance brief describes the thought experiment. It gives a reviewer concrete state changes to verify without choosing a job framework or claiming a current implementation.
Illustrative example · Acceptance brief · Simplified for explanation
Starting import revision: R1
Row A: accepted
Row B: validation_failed
Row C: rejected_by_reviewer
Action: retry validation-failed rows from R1.
Expected: create a new pending candidate for B.
Expected: leave A's accepted record unchanged.
Expected: leave C rejected.
Unavailable R1: report failure; do not substitute another source.
Completion evidence: compare row identities and states before/after.- Identify competing interpretations
- Write discriminating scenarios
- Compare candidate implementations
- Observe outcomes and review effort
Conceptual flow for the example above.
Tradeoff to keep in mind
The brief is one proposed policy. A different product may intentionally retry human-rejected rows or use a newer document, but should specify that behaviour before treating an implementation as correct.