AI / ML
Valid JSON Is Not Enough to Approve an Extracted Vendor Price
By Vivek Jalondhara · 5 min read
Published
Separate response shape, business rules, and source agreement when reviewing AI-extracted prices, using a valid but incorrect row.
A response can parse successfully, pass a schema, and still describe the wrong price. This is easiest to see with an example whose fields look ordinary: a product code, an amount, a currency, and a selling unit.
The following educational review exercise uses a fictional vendor sheet in the Sheet Parser domain. It challenges the proposed shortcut of treating schema-valid extraction as accepted catalogue data. It does not report a defect or an extraction result from the actual project.
Construct a response that passes the easy checks
The source row says INR 900.00 per pack of ten connectors. The candidate response contains amount 900.00, currency INR, and unit each. Every field has the expected JSON type. The currency is supported and the amount is non-negative. None of those checks reveals that the unit changed the meaning of the price.
This example gives a reviewer a concrete counterexample to automatic approval. It does not require a malformed response or an obviously absurd number. The difficult errors often preserve a convincing shape while changing the relationship between a value and its surrounding document context.
Assign a limited job to the schema
Use a schema to define required fields, accepted value types, and the boundaries of a response. If missing information is allowed, represent it deliberately. A required string cannot distinguish a correct product description from a plausible description copied from the adjacent row.
OpenRouter supports schema-constrained responses for compatible models. That helps the application consume a predictable structure, but the schema cannot inspect a private source unless the surrounding application provides and checks relevant evidence. Treat a successful shape check as one completed stage with a known purpose.
Check business rules without confusing them with evidence
Next apply rules the application actually owns. For example, a positive pack quantity might be required whenever the selling unit is pack. Supported currencies and maximum field lengths are other contract decisions. These checks can reject inconsistent proposals even when the source itself is difficult to interpret.
Passing business rules still leaves a source-agreement question. A candidate can name a valid pack quantity that the document never supplied. The application's internal consistency and the vendor's stated meaning are different properties, so give them separate outcomes in a review result rather than a single undifferentiated valid flag.
Preserve evidence that can be checked
Store a source location associated with the candidate and show the relevant price text, unit heading, and any qualifying note. A model-supplied quotation or row identifier needs verification too; generating an explanation does not guarantee that the referenced evidence exists. Prefer locations grounded by the file-reading stage where the format permits it.
For the connector example, a reviewer needs to see per pack of ten next to the amount. If the source preview shows only the numeric cell, the interface has removed the very context that would reveal the mistake. Evidence selection is part of the product design.
Make acceptance a separate operation
A review outcome can record shape passed, business rules passed, and source agreement unresolved. The row remains a draft until the required conditions are satisfied. Server-owned acceptance should identify the authorised reviewer or documented acceptance policy and the exact draft revision being accepted.
Automatic acceptance may be appropriate for a narrowly supported document contract with demonstrated behaviour. Define that boundary explicitly and preserve an exception path. A new vendor layout or an ambiguous unit should leave the supported path rather than inherit approval from a model's confidence statement.
Turn the counterexample into a reusable exercise
Keep the source phrase and the deliberately wrong candidate together as a fixture. Ask a developer to identify which layer should catch the unit mismatch and what the user should see. The answer should connect the stored evidence, review status, and accepted catalogue write.
Then vary only one property: remove the pack count, change the currency heading, or duplicate the product code under another vendor. Small variations expose whether the design follows an explicit contract. The goal is a system that can explain why a row is accepted, rather than a response that merely looks easy to store.
A valid shape with an incorrect selling unit
This JSON is intentionally wrong relative to the source phrase. It is a review fixture, not an approved price. A shape validator alone has no rule that connects each to the source wording.
Illustrative example · JSON · Simplified for explanation
{
"sourceText": "Connector C-10: INR 900.00 per pack of ten",
"candidate": {
"sku": "C-10",
"amount": "900.00",
"currency": "INR",
"unit": "each"
},
"expectedReview": {
"shape": "passes",
"sourceAgreement": "fails",
"reason": "The amount is for a pack, not one connector."
}
}- Validate response shape
- Check application rules
- Compare with source context
- Accept or return a review reason
Conceptual flow for the example above.
Tradeoff to keep in mind
More validation layers require clear ownership to avoid duplicate or contradictory errors. Each layer should state what it establishes; none should claim a stronger guarantee than the evidence supports.