Handle slow and failed AI responses in a normal web application

A user asks an AI assistant to summarize a note. The spinner remains for a minute, the user navigates away, and the response later overwrites a newer summary. Another request fails and the page discards the note. These are ordinary application lifecycle problems amplified by a slow external dependency.

The fix begins with a clear request contract and state model. This article uses an illustrative note-summary screen. It treats provider-specific streaming and cancellation as capabilities to verify, while keeping the application's user-facing behavior explicit and independent.

Define the states users can encounter

Represent idle, submitting, waiting, succeeded, failed, cancelled, and timed-out outcomes as appropriate for the feature. Avoid one loading boolean plus several unrelated flags that can produce contradictory combinations.

Include the user's draft separately from the generated result. A provider failure should not erase input. Decide whether the previous result remains visible during a new request and label it so users do not mistake it for the current response.

Distinguish accepted work from finished work. If the server creates a background operation, return its identity and expose a status path. If it holds a synchronous request open, make the expected wait and timeout behavior coherent with hosting limits. The interface should reflect the actual lifecycle rather than an imagined instant response.

Separate client and server time budgets

A browser timeout limits how long the user waits, but it may not stop provider work. A server deadline limits the application's request lifecycle, while the provider may have its own limits. Document these layers and verify how cancellation propagates.

Set budgets based on the product journey and deployment constraints rather than an arbitrary large number. A request that exceeds the platform's execution limit needs a different design, not just a longer frontend spinner. Background work can help, but it adds status tracking and durable ownership requirements.

Handle timeout as a defined outcome. The page should tell the user whether work may still complete, whether retry is safe, and how to recover. Do not claim cancellation stopped all remote processing unless the provider and application actually guarantee it.

Give every request an owner

Assign a request identity or generation number and allow only the current request to update the current result, error, and pending state. Starting a new summary makes prior responses obsolete for that screen.

This protects against late success and late failure. An older provider error should not replace a newer successful result. An old completion should not turn off the loading indicator for work still in progress. The ownership rule applies to every state transition, not just result assignment.

Cancel obsolete work where supported to reduce resource use. Still retain the ownership check, because cancellation can race with completion or fail to reach every layer. The guide on search response ordering develops the same browser pattern for ordinary API calls.

Design streaming as a partial-result contract

If the feature streams text, decide whether partial output is usable when the connection fails. A half-finished summary should not silently look like a complete one. Label interrupted output and provide a retry path.

Do not treat each chunk as a final validated structured record. If downstream code requires complete JSON, collect and validate the finished output before committing it. Streaming format and application semantics are different layers.

Consider navigation and reconnection. A transient network interruption may require retrieving the completed operation by identity rather than starting another one. If operations are not durable, explain that limitation and preserve the input so the user can retry deliberately.

Make retries safe and bounded

Classify failures before retrying. A temporary provider or network error may be retryable. Invalid input, missing authorization, or a policy rejection usually requires a different action. Repeating every failure can waste money and confuse users.

Use bounded attempts and account for total time. A sequence of long retries can exceed the product's wait budget even when each individual request has a timeout. Honor provider retry guidance where available, and avoid synchronized retry storms under a shared outage.

If the AI request also triggers a side effect, retry semantics require more care. Generating a draft twice differs from sending two emails. Separate generation from action execution and use idempotent operation contracts for consequential writes where needed.

Use AI to inspect the lifecycle

Give a coding assistant the state transitions and request path. Ask it to find execution orders that produce stale results, lost input, or conflicting states. Require concrete sequences rather than generic advice to add error handling.

An illustrative request is: “Trace what happens when request A times out, request B succeeds, then A completes. Identify every state write. Propose a narrow ownership rule and tests for success, failure, cancellation, and unmount.” This uses the assistant to reason about timing while keeping the contract visible.

Verify the proposed implementation with deferred responses or controlled server doubles. Test late success, late rejection, user cancellation, navigation, and timeout. A fast happy-path provider mock cannot reveal these lifecycle defects.

Verify deployment and communicate limits

Check the real hosting timeout, network path, and provider behavior in an approved isolated environment. Record what local doubles cannot establish. If the server cannot propagate cancellation, the UI can still avoid stale updates but should not promise that billing stops immediately.

Measure useful operational outcomes: request duration distributions, timeout categories, retry counts, and completed versus abandoned operations. Keep logs free of full sensitive prompts unless approved handling requires them. Request identifiers often provide enough correlation.

A resilient AI screen behaves predictably when the model is slow or unavailable. It preserves user work, makes state understandable, and keeps late responses from taking ownership of a newer interaction. These guarantees belong in application code, while provider capabilities remain explicit dependencies to verify.

Check accessibility during long waits. Keep the cancel control reachable by keyboard, avoid moving focus when streaming chunks arrive, and announce meaningful state changes without reading every token. A loading indicator should describe which operation is pending when the page contains several tasks. If the user returns after navigation, show whether the operation was abandoned or can be recovered. These interaction details help users understand the same lifecycle guarantees that the server and request ownership checks enforce.

Keep cancellation and abandonment distinct in operational records. A user leaving the page may invalidate UI ownership without a confirmed provider cancellation. Record whether work stopped locally, whether a cancellation request was sent, and whether completion later arrived. That distinction helps explain usage and prevents support messages from promising an outcome the system cannot establish. It also gives engineers a way to identify expensive abandoned work and decide whether durable background status or stronger propagation would improve the journey.

Sources

MDN: Using Fetch documents request and response handling.

MDN: AbortController documents cancellation controls.

Anthropic: Building effective agents discusses latency tradeoffs in model workflows.