How to Scope an AI or LLM Application Penetration Test

A useful AI penetration test scope names what the application can read, reveal, and change, then records permission for the paths that exercise those capabilities. A chatbot URL alone does not describe retrieval permissions, tool identities, or a downstream action. This guide turns an architecture discussion into a practical handoff for an authorized test.

The PenTesting Company provides AI and LLM application penetration testing. Use the worksheet below to prepare the scope discussion. It is a planning aid, not a substitute for the signed statement of work or rules of engagement.

Start with a decision, not a list of jailbreak prompts

Write a concrete objective such as: Can a member of Tenant A cause the support assistant to retrieve Tenant B's synthetic ticket, or draft a ticket under Tenant B's identity? Identify the server-side control expected to prevent each action. A changed answer or a proposed tool call is not evidence that protected data was accessed or an API accepted an action.

Record the environment, release identifier, model-integration configuration, prompt revision, tool schemas, and enabled features. If staging uses a mock tool while production uses a live API, say so. A test of the mock can examine orchestration decisions, but does not establish how the live API enforces authorization.

A layer-by-layer scoping worksheet

For each layer that exists, record its hostname or identifier, controller, authorization evidence, allowed techniques, allowed data, and exclusions. Mark missing components as absent. Mark unresolved permission as pending, not implicitly included. Include output destinations such as browser rendering, ticket drafts, exports, and callbacks when they exist.

Separate the application integration from the provider

Obtain written authorization from whoever controls each asset. Permission from an application owner does not by itself authorize testing a model provider's systems. A permitted request through an in-scope application is different from probing a provider endpoint directly, attacking shared infrastructure, or trying to access another provider customer. Document applicable provider terms and approved use of the integration. Provider-system testing is not included by default.

The same distinction applies to hosted retrieval, remote tool servers, and downstream software services. Application credentials do not establish permission to attack their hosts. Where control or permission is uncertain, exclude that path pending written authorization and record the resulting coverage limitation. Review of configuration or supplied documentation may be agreed separately.

Make the account and data comparison testable

List the human and service identities involved: who submits the request, who retrieves documents, and which identity calls each tool. For tenant-isolation objectives, provide separate synthetic records and accounts in two test tenants. For same-tenant ownership checks, include paired users. State the expected allow or deny decision for each role, object, and action.

Agree whether testers can seed documents, edit metadata, change permissions, or inspect retrieval and tool traces. Those are distinct permissions. If only the chat UI is available, record that a missing disclosure in the final answer does not establish that protected content stayed out of model context.

Budget the whole execution path

Specify request rates, concurrency, input sizes, token limits, retries, tool-call counts, and an overall usage ceiling in the rules of engagement. Assign a person to watch the usage meter and define what happens if metering is delayed. A single chat request may trigger retrieval, several model calls, retries, and downstream actions; a browser request count alone may miss those costs.

Agree an early pause threshold below the resource ceiling rather than planning to reach it. Do not assume load, denial-of-service, or deliberate resource-exhaustion testing is included. Name who can pause the test and who can approve resumption after a scope or budget question.

Filled worksheet: Example Corp's support assistant

What to send before kickoff

• Architecture and inventory: A data-flow sketch, entry points, environment URLs, retrieval sources, model-integration details, tool schemas, output destinations, and current configuration revisions.
• Authorization pack: Asset controllers, written permissions, applicable third-party terms, explicitly excluded systems, permitted techniques, and the agreed testing window.
• Account matrix: Roles, paired tenants or users, service identities, expected permissions, MFA arrangements, and synthetic fixtures. Transfer credentials through the agreed secure channel, not a public form.
• Visibility and evidence: Agreed access to retrieval, orchestration, API, and usage logs; correlation identifiers; redaction requirements; retention and deletion terms; and a walkthrough of expected behavior.
• Operational limits: Completed resource budgets, allowed writes and cleanup steps, pause triggers, rollback owner, primary and backup contacts, and who may approve resumption.
• Data and tooling terms: Permitted testing tools and data classes. Client data goes into third-party AI services only if the statement of work allows it.

If we confirm a critical-severity issue during testing, the rules of engagement set how and how quickly your named contact is told, rather than waiting for the report.

Use references to choose scenarios, not promise coverage

The OWASP Top 10 for LLM Applications (2025) helps organize candidate risks such as prompt injection, sensitive information disclosure, excessive agency, and vector and embedding weaknesses. MITRE ATLAS helps structure adversarial scenarios. These are planning references, not claims that every technique or category is covered.

Ask the final report to distinguish exercised paths, unavailable evidence, excluded assets, and configuration changes. Keep the scope worksheet with the report so a later reader can understand which architecture and permissions the evidence describes. For the general access handoff, see how to prepare for a penetration test.

Frequently asked questions