Set a performance budget that can fail in CI

A performance budget is useful when it changes an engineering decision. A report nobody reads is observability, not a release gate. Conversely, a gate that fails randomly teaches the team to ignore it. The objective is to catch meaningful regressions in a reproducible environment while leaving a clear path for investigating and accepting justified changes.

Imagine a product dashboard that gradually accumulates new scripts and widgets. Each feature works, but the first usable screen gets slower. A budget should make that accumulated cost visible before another release. It should also distinguish a real regression from a noisy runner or a unavailable test dependency. This guide describes a practical starting point, not a universal set of numbers.

Choose a journey before choosing a metric

Name the screen and what the user must accomplish there. Loading a public article, opening an authenticated dashboard, and filtering a table are different journeys. Choose one or two important ones first. A broad collection of metrics without a journey often produces lots of warnings and little agreement about which failures matter.

Define the condition you want to preserve. It might be a bounded JavaScript payload, a page-load measurement under controlled conditions, or the time needed for a synthetic interaction. Payload size is relatively easy to measure consistently, but it is not the same as responsiveness. A smaller bundle can still run expensive code during every click.

Do not copy another team's thresholds without understanding its hardware, network, application and users. Establish a baseline from your own application and decide what deterioration would be unacceptable. Document whether a number is a temporary baseline, a product requirement, or a longer-term target. These labels keep a gate from pretending to represent evidence it does not have.

Control the environment that produces the measurement

Pin the relevant tooling and browser version through a deliberate update process. Use a production build, representative fixtures, and a consistent route. Keep external services from introducing uncontrolled variability where possible. If a page requires authentication, establish a repeatable test session rather than measuring a redirect to the login screen by accident.

Record runner characteristics and throttling configuration. Comparing a fast local laptop with a shared CI machine can create misleading conclusions. The budget does not require an unrealistic laboratory, but it does require enough consistency to interpret changes. If the environment changes, revisit the baseline instead of silently moving thresholds.

Separate application performance from unavailable dependencies. A failed API connection should produce a clear setup or availability failure. It should not become a low performance score that developers try to fix through bundle optimization. Verify that the intended page and content actually loaded before accepting the measurement.

Use several measurements without hiding the distribution

Run more than one sample when the chosen measurement is noisy. Preserve individual results and define how the gate summarizes them. A median can reduce the effect of a single outlier, while another percentile may fit your goal better. The key is a documented rule, not whichever summary makes the current pull request pass.

Avoid averaging unrelated journeys together. A fast login page does not compensate for a slow inventory page. Likewise, a good overall score can hide one metric that regressed in a way relevant to your users. Store the raw report so a reviewer can inspect the cause rather than arguing only about the final color.

Lighthouse scoring has its own interpretation and weighting. If you use Lighthouse CI, understand its assertions and reports rather than treating a score as an absolute description of every real visit. A laboratory check is a useful regression signal, but it does not replace field observations on devices and networks used by actual readers or customers.

Start with one enforceable gate

A small budget that the team can explain is better than twenty settings introduced together. The following configuration is illustrative and its thresholds are project choices, not recommended universal limits:

{
  "ci": {
    "collect": {
      "url": ["http://localhost:3000/blog/example"],
      "numberOfRuns": 3
    },
    "assert": {
      "assertions": {
        "categories:performance": ["error", { "minScore": 0.8 }]
      }
    }
  }
}

Before using it, adapt the route, server startup, fixtures, tooling installation and report storage to your project. A configuration file alone does not create a reliable pipeline. Verify that the check examines a real article and fails when you intentionally introduce a relevant slowdown in an isolated branch.

If your first priority is payload growth, use a dedicated size check with a clear comparison basis. If your concern is interaction, add a scripted interaction measurement. Keep the budget connected to the journey rather than adding tools because they are fashionable or because an AI assistant listed them.

Make failure explainable to a reviewer

When a check fails, report the journey, baseline, observed result, threshold and link to the artifact. Include enough context to distinguish a changed application from a changed environment. A bare “performance failed” message forces the reviewer to reproduce infrastructure before investigating code.

Establish a policy for intentional increases. An accessibility improvement or valuable feature may justify a cost, but the decision should be recorded with evidence and alternatives. Do not make threshold changes invisible housekeeping in the same pull request. Explain why the previous limit no longer represents the intended requirement.

For an unexpected failure, inspect the waterfall and main-thread work before changing the limit. Budget violations should initiate diagnosis. Raising the threshold because a report is inconvenient removes the mechanism that was supposed to protect the user journey.

Use AI to organize evidence and counterexamples

An assistant can compare reports, identify changed resources, or suggest hypotheses tied to code paths. Provide redacted reports and the relevant diff. Ask it which change could explain the observed metric and which experiment would disprove that explanation. Do not let a model invent measurements or replace a failing assertion with a less relevant one.

Require it to preserve the test conditions and identify unexecuted checks. A proposed fix that reduces transfer bytes but moves work into an interaction needs another verification path. This is a useful place to combine the budget with browser timing diagnosis.

Verify the gate before relying on it

Test a passing baseline, an intentional regression, a missing dependency, and a permitted change. Inspect the reports for each outcome. Confirm that the check cannot pass against an empty page, a redirect, or a cached response that bypasses the important content. Document how engineers run the same check locally.

After rollout, compare CI signals with real-user observations and incidents. If the gate repeatedly misses meaningful problems, adjust the journey or metric rather than accumulating unrelated assertions. If it is noisy, improve the environment or statistical rule before weakening enforcement. A performance budget earns trust through clear evidence and predictable behavior, not through strict numbers alone.