Detect when an AI assistant invented an API or dependency

An assistant suggests importing createTransactionalSession from a database package. The name fits the library's vocabulary and the surrounding code looks sensible. Your editor cannot resolve it because the export does not exist. A more subtle failure occurs when a method exists in a different package version but not the one deployed by your application.

Treat generated API references as claims to verify. This article uses an illustrative library integration rather than a specific database SDK so the method applies to package imports, framework options, and browser APIs alike.

Identify the dependency actually in use

Read the package manifest and lockfile. The declared version range is not always the installed version. A workspace may override a package or resolve several versions. Confirm which package instance the relevant module imports.

Do not assume the assistant knows the repository's dependency versions. Its example may reflect an older release, a newer release, or a similarly named library. Ask it to state the package name, version assumption, and source documentation for each unfamiliar API.

Check whether the integration runs on the server, browser, worker, or edge runtime. A valid Node API can still be unusable in the browser. An export may be supported only in a specific entry point. Version and runtime context are part of the contract, not incidental setup details.

Inspect exports and type declarations

Look at the installed package's exports and declaration files. Search for the proposed symbol with a tool such as ripgrep. If it appears only in tests or internal source, determine whether it is public and reachable through the supported entry point.

A successful text search is not enough. A method may belong to a class instance rather than the package namespace. It may accept different arguments or return a promise rather than a direct value. Inspect the signature and the surrounding type.

Some packages have incomplete types, and some applications use untyped JavaScript. In those cases, inspect official documentation and the supported runtime export. Avoid reaching into private package paths merely to make the assistant's suggestion work. Private internals can change independently of the documented API contract.

Match documentation to the installed version

Use primary documentation and versioned references where available. A current documentation page may describe an API introduced after your locked version. Release notes can establish when the feature appeared or changed.

Compare the method's required arguments, error behavior, and return value with the generated code. A correctly named method can still be used incorrectly. For example, the assistant might pass an options object in the argument position that expects a callback, producing a runtime failure despite a plausible snippet.

Record what is verified and what remains ambiguous. If documentation does not clearly cover the installed version, inspect the package source or a minimal runtime example. Do not convert ambiguity into certainty by selecting the first search result that uses the desired name.

Build the smallest executable probe

Create an isolated example that imports the proposed API and exercises only the claimed behavior. Use synthetic data and a local or disposable environment. The probe should fail clearly if the symbol, argument shape, or result assumption is wrong.

For an ordinary package export, importing it and checking its type may establish availability. For transactional behavior, a real isolated storage test may be necessary because a method name does not prove rollback semantics. Choose evidence that matches the property being claimed.

Do not run a probe that writes to production merely to verify an assistant's example. If the API needs credentials or external infrastructure, document that boundary and use the approved test environment. A mock can verify your wrapper's behavior but cannot establish the provider's actual semantics.

Ask the assistant to repair from evidence

Give the assistant the installed version, relevant declaration, and failing probe. Ask it to propose an implementation using the verified public API. Require an explanation of how the corrected code maps to the actual signature.

An illustrative prompt is: “This package version exports openSession, not createTransactionalSession. Here is the public declaration and the failing import. Adapt the bounded change without adding dependencies or using private paths. Identify any behavior the available API cannot provide.” This directs the assistant toward reality instead of another invented name.

If the needed feature is unavailable, compare alternatives explicitly. The choices might be implementing a small wrapper, changing the design, or upgrading the package. An upgrade is a separate compatibility decision with its own tests, not an automatic rescue for a generated snippet.

Verify integration beyond the probe

After the minimal example works, check the actual call path. Error translation, cleanup, concurrency, and serialization can still be wrong. A library method that requires disposal may leak resources when integrated into a long-lived server.

Run the relevant tests and build for the deployment runtime. A local Node probe does not prove compatibility with an edge build or browser bundle. Inspect whether the new import brings server-only code into a client path or changes bundle behavior.

Add a focused regression test for the property the integration needs. Avoid a test that only asserts the wrapper called the same invented method through a mock. The guide on provider doubles explains why a mock's invented interface can make incorrect code appear valid.

Prevent the same mistake without banning assistance

Keep dependency versions and approved integrations visible in task context. Include a neighboring verified implementation when it is applicable. Ask the assistant to cite unfamiliar APIs and to flag uncertain compatibility rather than confidently filling gaps.

Use type checking, linting, and dependency policies as early feedback. These tools catch many import and signature errors cheaply. They cannot prove every semantic property, so retain targeted execution for behavior such as retry, rollback, or cancellation.

The useful habit is neither trusting every plausible name nor distrusting all generated code. Treat a suggestion as a hypothesis. Confirm that the dependency exists, the API is public in the installed version, the runtime supports it, and the behavior matches the application's need before allowing it into a wider change.

Watch for generated types that describe a different service revision. An import can resolve and the call can type-check while the deployed endpoint expects another field name. Compare the generated artifact’s source and version with the actual API contract, then regenerate through the documented process if needed. Avoid editing the generated declaration to make the assistant’s code compile. That can conceal the mismatch and will likely be overwritten later. The correct evidence connects the installed client, its generation source, and the service it calls.

Sources

GitHub Copilot best practices emphasizes validating generated suggestions.

MDN: JavaScript modules explains import and export behavior.