Architecture
Prediction: Background AI Jobs Will Make Recovery Part of API Design
By Vivek Jalondhara · 5 min read
Published
A prediction about AI application APIs: durable operation status and explicit recovery actions will matter as much as the initial submit response.
My prediction is that applications with substantial background AI work will increasingly treat recovery as part of their public API contract. This is a design forecast, not a measured adoption trend or a claim that every AI request belongs in a queue.
The reasoning starts with a user who submits a vendor sheet, closes the page, and returns later. In an educational Sheet Parser scenario, that person needs to recover a meaningful answer about the submitted work. A spinner tied to the original browser request cannot provide that answer after the page disappears.
Separate present HTTP behaviour from the prediction
MDN documents that HTTP 202 means processing was accepted, not that it completed or will necessarily succeed. Google’s long-running-operation guidance describes returning an operation that a caller can inspect later for progress and results. These are established interface patterns, independent of the forecast.
The prediction concerns where product design effort will go. If an application asks users to wait for extraction, validation, and review preparation, the experience after submission becomes a larger part of the feature. I expect more API discussions to start with what a returning user can understand and safely do next.
Give submitted work a stable identity
An operation identifier should lead back to the input version and the accepted options for that attempt. Otherwise, a user returning after another upload may be unable to tell which source sheet produced the displayed result. Preserve enough non-sensitive context to distinguish those attempts without exposing the source document to unauthorised readers.
For the exercise, define one operation as processing one immutable upload revision with one accepted configuration. If the user changes the source, create a new operation rather than silently changing what an existing identifier means. An internal execution retry can remain attached to the same logical operation if its meaning is unchanged.
Expose a state that explains the next action
A generic error leaves the user guessing whether to resubmit, correct the sheet, wait, or contact support. Define failure categories around useful actions. Invalid source structure could lead to replacing the upload; an unavailable dependency could lead to a supported retry; an uncertain external outcome may require reconciliation first.
Do not promise that every failure is retryable. In the proposed contract, the server returns actions appropriate to the current state, and rechecks their validity when the user invokes them. A stale page showing a Retry button cannot authorize a retry after the operation has already moved into review.
Describe cancellation and review honestly
A cancellation request is an intention to stop future work. It need not mean that an already-issued provider call has been reversed. Represent cancellation requested separately from cancellation confirmed if the implementation cannot make them the same event. Keep any unresolved effect visible to the component responsible for reconciliation.
Likewise, successful extraction need not mean approved catalogue data. In this scenario, awaiting-review describes a useful intermediate outcome: a draft exists, but a person must resolve questionable rows before publication. Keeping that distinction in the operation resource prevents a technically successful model call from being presented as completed business work.
Design the return visit and the end of retention
A returning user should be able to find their operations from an authorised list, not only from a URL copied before navigation. Show the source label, submission time, and current state with the same access restrictions used for the underlying vendor data. Treat browser storage as convenience, not the sole index of work.
Choose an explicit retention policy for operation metadata and results. When an operation has expired, make the documented client behaviour clear. Do not instruct users to blindly resubmit after every missing response, because the original submission may already have produced a result that is retained under a different lifecycle.
Look for counterexamples and evidence
This prediction is weaker for small, bounded synchronous tasks where a normal request consistently meets the product’s latency requirements. It is also less relevant to an internal batch process whose consumers already use a separate job system. Adding operation resources there may create complexity without improving anyone’s ability to recover.
To evaluate the forecast in a real product, inspect actual support questions and usability sessions for confusion after refresh, duplicate submissions, and uncertain retries. Those observations would justify investment more convincingly than adding a queue because the feature uses AI. The desired outcome is a clear recovery conversation between the client and server.
Describe an import waiting for review
Illustrative application-specific status payload, not an implementation of the Google Operation schema. The attempt count describes internal execution attempts for one unchanged input.
Illustrative example · JSON · Simplified for explanation
{
"operationId": "import-demo-17",
"sourceRevision": "upload-demo-3",
"revision": 6,
"attempt": 2,
"state": "awaiting-review",
"draftId": "draft-demo-9",
"allowedActions": ["open-review"]
}- Accept an identified input revision
- Track the durable operation
- Recover status after leaving the page
- Offer a valid next action
Conceptual flow for the example above.
Tradeoff to keep in mind
The payload describes visibility, not worker reliability. Durable state changes, bounded retries, tenant authorisation, and reconciliation of uncertain effects still need implementation.