Test an MCP server at four layers: protocol conformance, capability contracts, transport lifecycle, and security boundaries. Use the MCP Inspector interactively while developing, its CLI for repeatable smoke tests, the official conformance suite where applicable, and application-specific tests for every consequential tool.
Listing tools successfully is connectivity—not production readiness.
Build a test matrix before commands
Record the deployed protocol version and transports first. MCP behavior evolves, so a test without version context can pass against the wrong lifecycle.
| Layer | What must hold | Evidence |
|---|---|---|
| Initialize | Version and capabilities negotiate as expected | Initialize result and server version |
| Discovery | Tools, resources, and prompts match the approved catalog | Sorted schema snapshot |
| Invocation | Valid inputs return typed, bounded results | Request, result, timing, trace ID |
| Invalid input | Missing, extra, malformed, and oversized arguments fail safely | Structured error and no side effect |
| Lifecycle | Cancellation, timeout, reconnect, and shutdown behave correctly | State transitions and cleanup |
| Authorization | Principal, audience, resource, and tool policy are enforced | Policy decision and receipt |
| Adversarial content | Untrusted resources/results cannot expand authority | Blocked action at enforcement point |
The MCP security boundary guide supplies the negative authorization cases. This guide turns the whole server contract into an executable release plan.
Use Inspector for discovery, CLI for automation
The official MCP Inspector provides web, CLI, and terminal clients. During development, launch a local stdio server in the web client:
npx @modelcontextprotocol/inspector node build/server.js
Then reproduce checks in CLI mode:
npx @modelcontextprotocol/inspector --cli node build/server.js \--method initialize --format jsonnpx @modelcontextprotocol/inspector --cli node build/server.js \--method tools/list --format jsonnpx @modelcontextprotocol/inspector --cli node build/server.js \--method tools/call --tool-name order_get \--tool-arg order_id=ord_test_123 --format json
Check the current Inspector help before copying flags into CI. The tool and protocol are versioned; pin the package and record that version with test artifacts rather than executing an unpinned latest release.
For a remote Streamable HTTP server, use an isolated test environment and a limited test credential. Never put production bearer tokens in workflow arguments or logs.
Snapshot capability contracts
Discovery responses are an API surface. Normalize and snapshot:
- tool names, descriptions, input schemas, and annotations;
- resource URIs, templates, MIME types, and subscriptions;
- prompt names, descriptions, arguments, and generated message shape;
- negotiated protocol version and capabilities.
Sort keys before comparison so harmless ordering does not fail the build. Require explicit review when a tool gains a field, a description changes its implied use, or a resource becomes discoverable. A schema can remain valid while permissions or model selection behavior change materially.
Test every tool as a side-effect contract
For each tool, keep a table like this:
| Case | Fixture | Expected result | Expected effect |
|---|---|---|---|
| Valid read | Owned test record | Typed content | No write |
| Missing required field | Omit identifier | Invalid-argument error | None |
| Extra field | Add user_id | Rejected by strict schema | None |
| Not found | Unknown test ID | Stable not-found error | None |
| Unauthorized | Other tenant’s ID | Policy denial | None |
| Duplicate write | Same operation key twice | Same receipt or conflict | One effect |
| Dependency timeout | Stub delays response | Timeout/cancel status | No ambiguous retry |
Model-facing descriptions and schemas are only the selection contract. Runtime validation and authorization remain authoritative; see agent tool schema and error design.
Exercise transport and lifecycle failures
Run the same contract suite over every supported transport. Include:
- client disconnect during an in-flight read;
- cancellation before and during a tool effect;
- server restart between discovery and invocation;
- duplicate delivery or retry after an ambiguous response;
- progress notifications without indefinite timeout extension;
- malformed messages and unsupported protocol versions;
- concurrent calls against shared state;
- resource subscription cleanup after disconnect.
Do not equate client cancellation with rollback. If an external provider accepted a write before the connection closed, reconcile the durable receipt before retrying.
Security tests need an enforcement point
A passing security test names where the request stopped. Add cases for wrong token audience, expired credentials, cross-tenant IDs, consent mismatch, state-handle reuse, redirect manipulation, SSRF targets, poisoned tool results, and unapproved tool changes.
A model saying “I cannot do that” is not enforcement if the server already executed the operation. Assert on the server policy event and downstream fixture state.
The Inspector proxy can launch local processes. Its own documentation warns against exposing it to untrusted networks. Keep default local binding and authentication; do not disable those controls for CI convenience.
Minimal CI gate
[ ] Pin Inspector, SDK, server, and protocol versions[ ] Start isolated dependencies and seed deterministic fixtures[ ] Run initialize and normalized discovery snapshots[ ] Run valid, invalid, authorization, and side-effect cases[ ] Run transport cancellation and timeout cases[ ] Save sanitized JSON results and server logs as artifacts[ ] Tear down fixtures and verify no unexpected writes remain
Separate fast pull-request tests from destructive or slow nightly suites. The pull-request job should use read-only repository permissions and test-only credentials. Never use pull_request_target to execute untrusted contributor code with secrets.
Turn every production MCP failure into a named regression case. The AI-agent CI quality-gate guide shows how to split deterministic invariants, semantic scoring, and release artifacts.
