Inside the R&D: Building an AI Pentester from the Ground Up

Daniel Knight, CEO at Vulnetic Inc.

Abstract

This article presents some of the research and development process behind an LLM-driven penetration testing system that went into a provisional patent application we filed in early 2025. The system combines large language models (LLMs), client-server orchestration, iterative exploitation loops, and automated reporting pipelines. We outline some of the architecture, experimental motivations, and technical considerations that motivated our filing.

1. Introduction

Penetration testing has historically been performed by individuals or small teams of specialists. Although effective, this approach does not scale to the current threat environment. Automated vulnerability scanners exist, but these tools generally operate on signatures rather than reasoning and cannot adapt to unexpected conditions and edge cases.

Our research sought to close this gap: could a sufficiently intelligent large language model (LLM) conduct penetration testing in a structured, iterative fashion, with the ability to reason about partial results and adapt strategy dynamically?

2. System Architecture

2.1 Client–Server Model

The system consists of three primary components:

• Client computer — initiates tests via a web interface or shell.
• Server — cloud-based, manages session state, authenticates requests, and relays tasks.
• LLM(s) — selected at runtime from multiple available models (e.g., GPT-5, Claude, Gemini, Llama) depending on task requirements.

2.2 Memory and Session Tracking

All interactions are recorded in a structured database schema. This includes queries, executed commands, outputs, and vulnerability analyses. Maintaining memory enables branch-point testing: one probe can generate multiple exploit paths, and the LLM can revisit branch points based on results.

3. Iterative Exploitation Loop

The system advances through defined stages (see Fig. 4):

1. Authentication and input — user specifies target identifiers (IP, hostnames, ports).
2. Task generation — LLM produces executable instructions (e.g., Nmap scan, curl request).
3. Execution — client runs commands against target system.
4. Response analysis — outputs are returned to LLM for reasoning.
5. Threshold check — system halts once conditions such as Remote Code Execution (RCE) are met.

Example sequence:

• Probe: Nmap reveals open port 80.
• Probe: curl confirms HTTP service is live.
• Probe: HTML analysis reveals vulnerable function.
• Probe: exploit command id returns root permissions.
• Proof marker: system writes “RCE Achieved” to host.

4. Automated Reporting

After a session concludes, results are passed into the reporting pipeline (see Fig. 5):

Findings are serialized as JSON, transformed into structured objects, and rendered into HTML/PDF reports containing:

• Executive summary
• Key findings
• Exploitation path(s)
• Risk assessment
• Mitigation recommendations

Reports can be exported or pushed into enterprise tooling (Jira, ServiceNow, Splunk, SIEMs).

5. Discussion

5.1 Comparison to Existing Tools

A static scanner may discover attack surfaces, but an offensive model can probe and test endpoints to prove vulnerabilities exist.

Manual pentesting is an effective way to assess endpoint security because a human will have a systemic understanding that static tools do not. However a major limitation to manual penetration testing is cost. An un-aided expert is expensive to source, and types and reads slowly.

6. Conclusion

This work represents a step toward agentic penetration testing systems that combine automation with adaptive reasoning.

In the coming weeks, we will be releasing more articles depicting how we attacked the problem set of agentic penetration testing.

Website: https://www.vulnetic.ai

Contact: danielk@vulnetic.ai