loading
Loading content
loading
Quarterly pentests describe a security state that no longer exists by the time the report lands. Continuous validation, scheduled and trigger-based with audit-ready evidence, is the only model that keeps pace with modern infrastructure, modern attackers, and DORA, NIS2, and SEC requirements.
Nenad Zarić · Founder
Quarterly pentests, annual audits, and ad-hoc vulnerability scans produce a snapshot of a moment that no longer exists by the time the report lands. Attack surfaces change daily. AI accelerated the rate of change past the point where periodic testing can keep up. Continuous validation, meaning scheduled, recurring, trigger-based testing with automated evidence generation, is the only model that matches the speed of modern infrastructure and modern attackers. Regulated sectors face an additional reality: DORA, NIS2, and SEC cyber disclosure rules now require provable, repeatable testing. A quarterly pentest report does not satisfy any of them.
Key takeaways
Every quarter, a security consultancy sends a team to your environment. They spend two weeks enumerating, scanning, probing. They produce a report. The report lists findings ranked by severity: 3 critical, 12 high, 47 medium. You read it. You assign remediation tickets. You fix the criticals this quarter, the highs next quarter. The next pentest arrives. The cycle repeats.
This ritual consumes weeks of internal team time for scoping, coordination, and remediation. It produces a document that describes a security state that no longer exists.
Here is what happens between the last day of testing and the day the report lands on your desk:
Your infrastructure team deploys three new microservices. A developer spins up a staging environment with a public-facing endpoint and forgets to restrict access. Marketing launches a campaign microsite on a subdomain that was not in scope. Your cloud team rotates IAM roles for a data migration, inadvertently leaving a permissive trust policy in place. A third-party vendor updates a JavaScript library your checkout flow depends on, introducing a known CVE.
None of this is unusual. It is what happens every week in any organization operating at reasonable velocity. And none of it appears in the pentest report sitting on your desk, because none of it existed during the testing window.
Now multiply this by the number of weeks between tests. Quarterly pentests sample roughly 2 weeks out of 13. That leaves 11 weeks of unvalidated change per quarter. 44 weeks per year in which your attack surface shifts, new vulnerabilities appear, and nobody is checking whether the fixes from last quarter's report still hold.
The remediation regression problem compounds this. You fixed the critical SQL injection the pentest found in Q1. In Q2, a developer refactored the authentication module and inadvertently re-exposed a parameterized query to unsanitized input. Nobody tested it between the fix and the next pentest, because nobody was scheduled to. The Q2 pentest finds a critical SQL injection. It is the same endpoint, regressed silently over 10 weeks of unvalidated change.
This is not a failure of the pentest team. They did exactly what you paid them to do: test what exists during their window and report the findings. The failure is the model. Point-in-time testing, by design, does not see what happens between tests.
Point-in-time testing was defensible when infrastructure moved slowly. Quarterly made sense when quarterly was the rate of change. It no longer is.
CI/CD velocity. Development teams deploy multiple times per day. Every deployment changes the attack surface. A new API endpoint, a config change, a library update. A quarterly pentest samples a two-week window out of 52. The other 50 weeks of change go unvalidated.
Cloud sprawl. Infrastructure that used to be a known set of servers behind a firewall is now a shifting constellation of VMs, containers, serverless functions, and managed services spread across regions and accounts. Shadow IT adds services the security team does not know exist. M&A events dump entire foreign infrastructures into the scope overnight. A CISO at a Fortune 500 financial services company recently told me their attack surface changed by 15% in a single quarter following an acquisition. The quarterly pentest scope was obsolete before the ink-on-paper signature phase concluded.
AI-accelerated attacks. Microsoft and OpenAI published joint threat intelligence in February 2024 documenting state-sponsored actors using LLMs in active operations. Forest Blizzard (Russian GRU) using AI for reconnaissance scripting. Emerald Sleet (North Korean Kimsuky) generating spear-phishing content in fluent, localized language. Crimson Sandstorm (Iranian IRGC) using AI for tool development. These are attributed adversaries with documented AI-augmented workflows.
The numbers on the ground reflect the acceleration. CrowdStrike's 2024 Global Threat Report tracked average breakout time, the window between initial access and lateral movement, at 62 minutes in 2023, down from 84 minutes in 2022. The fastest recorded was 2 minutes and 7 seconds. Rapid7's 2024 Attack Intelligence Report found that the median time from vulnerability disclosure to exploitation has stayed in single-digit days since 2020, and 53% of widely exploited CVEs in 2023 began as zero-day attacks. Google TAG and Mandiant tracked 97 zero-days exploited in the wild in 2023, over 50 percent more than in 2022 and approaching the record of 106 set in 2021.
A quarterly pentest against an adversary moving in minutes is not a defensive strategy. It is a procedural checkbox.
The compliance shift. DORA, the EU Digital Operational Resilience Act, took effect in January 2025. NIS2 followed. The SEC adopted cyber disclosure rules requiring public companies to disclose material cybersecurity incidents within four business days and describe their processes for assessing and managing cyber risk. These regulations share a common thread: they demand documented, repeatable testing that produces evidence. A quarterly pentest report, by itself, satisfies none of them. I will return to this in section 5.
The gap in one number: The average breach lifecycle, detection to containment, is 258 days per IBM's 2024 Cost of a Data Breach report. The average breakout time is 62 minutes. That is roughly eight and a half months of response against one hour of attack. Point-in-time testing shrinks neither number.
Continuous validation means testing runs on a schedule you define, not a calendar event negotiated with a consultancy. It means the same methodology executes weekly, daily, or on trigger: when a new CVE drops in your stack, when a deployment completes, when a new subdomain appears in your asset inventory.
The model has four operating modes:
Scheduled validation. Your testing methodology runs on a fixed cadence. Weekly external recon across all known assets. Daily vulnerability scanning of production infrastructure. The methodology does not change between runs. The output does. You compare this week's results against last week's and see exactly what changed. A new finding is a real change. A persistent finding is a remediation gap with a measurable age.
Recurring checks. The tests that matter most are the ones you run after remediation. You fixed the SQL injection. The question is not whether the pentest team found it. The question is whether the fix was complete, whether it introduced a different vulnerability, and whether the developer who made the change understood the root cause or applied a surface patch that the next pentest will find again. Continuous validation runs the exact same test against the fixed endpoint and verifies the finding is closed. Not a proxy check. The actual test.
Trigger-based workflows. Some events should automatically initiate validation. A new CVE in Apache Struts drops. Your workflow triggers: inventory every asset running Struts, scan for the CVE, report findings. The time between CVE publication and your first validated result shrinks from days of manual spreadsheet work to the execution time of your workflow, typically minutes.
Continuous evidence generation. Every run produces structured output. Findings with CVSS scores, affected assets, discovery timestamps. Remediation records with before/after test results. Audit trails showing exactly what was tested, when, with what tool, and what the result was. This output is not a PDF someone wrote after the fact. It is the machine output of the test itself, timestamped and queryable.
The operational difference between this model and a quarterly pentest is not a matter of degree. It is a different category. Point-in-time testing tells you what was true during a two-week window three months ago. Continuous validation tells you what is true now, what changed since yesterday, and whether the thing you fixed last week is still fixed.
Most security teams already have 80% of what they need. They have the methodology. They have the tools. What they lack is the execution layer that connects them.
The methodology lives in the heads of your senior engineers. They know exactly what to test, in what order, with what tools. They have runbooks in Confluence, playbooks in Sharepoint, shell scripts in a repo nobody maintains. The methodology is sound. The execution is fragile.
An execution layer is the infrastructure that takes a methodology and runs it deterministically. It is not a tool. It is the layer that orchestrates your tools in the sequence your methodology defines, on the schedule your risk profile demands, producing the structured evidence your regulators require.
Here is what that looks like in practice on the Trickest platform, where I have been building this for the last several years.
The methodology becomes a DAG. A directed acyclic graph of nodes. Each node is a concrete operation: enumerate subdomains, scan ports, probe services, run vulnerability checks, triage findings. The graph defines what runs, in what order, with what dependencies. When you change the methodology, you change the graph. The graph is the methodology.
Execution at infrastructure scale. A single workflow fans out across parallel runners. Subdomain enumeration that takes a security engineer four hours on a laptop takes minutes on 60 parallel machines. Port scanning across a class B network that would take two days serially completes in under an hour. The same workflow that tests 100 assets today tests 100,000 tomorrow. The fleet scales horizontally. 500,000 machines, 30-second spin-up, tear-down when the run ends.
Structured output, not tool logs. Results from every node in the graph land in a database, backed by ClickHouse. You do not grep through Nmap XML or Nuclei JSON. "Show me every finding with CVSS at least 7.0 across all assets tagged production that appeared in the last four weeks" is a database query you run in seconds. The output of one test feeds into the next without manual parsing or spreadsheet assembly.
Asset and VMS integration. The execution layer connects to your asset inventory, your CMDB, your vulnerability management system. New assets discovered by recon get added to scope automatically. Findings flow into your VMS with source attribution and verification metadata. The loop from discovery to ticketing closes without a human copy-pasting between tools.
AI as assistant, not operator. The Trickest agent can build workflows from natural language descriptions. You describe the methodology, it composes the graph. Every node it places is visible and inspectable. The human owns the methodology. The agent accelerates the build. Once the workflow exists, it runs deterministically. The same inputs produce the same outputs, every time. This is the distinction that matters for compliance and for sanity: AI helps you build the workflow. It does not autonomously test your production environment and produce findings you cannot reproduce or audit.
Regulation is turning point-in-time testing from a weak practice into a compliance gap.
DORA (EU Digital Operational Resilience Act). In effect since January 2025. Chapter IV, Articles 24 through 27 mandates that financial entities establish a "sound and comprehensive digital operational resilience testing programme." The testing must include a range of methodologies. Tests must be performed by independent parties with documented competence. The scope must be validated by competent authorities. And entities must "establish internal validation methodologies to ascertain that all identified weaknesses, deficiencies or gaps are fully addressed."
That last requirement is the one that breaks the quarterly pentest model. You must prove, with a documented methodology, that every finding was verified and every fix was validated. A pentest report lists findings. It does not demonstrate that verification happened. It does not produce audit trails showing that the critical vulnerability found in Q1 was retested in Q2 and confirmed closed. It does not show that the fix did not regress in Q3 after a refactor. Continuous validation generates that evidence as a property of the system. Every scheduled run, every remediation retest, every trigger-based scan produces structured output with timestamps, tool signatures, and result data. You do not produce compliance evidence after the fact. The system produces it during operation.
NIS2 (EU cybersecurity directive). Article 21(f) mandates "policies and procedures to assess the effectiveness of cybersecurity risk-management measures." Policies and procedures. Documented, repeatable processes. Not tool output from a one-time engagement. A continuous validation workflow running on a defined schedule with structured evidence output is a procedure. A quarterly pentest report is a deliverable. The regulation asks for the former.
SEC cyber disclosure rules. When a US public company discloses a material cybersecurity incident, the board-level scrutiny that follows includes a predictable question: "What testing did you perform, and when?" The answer "we had a pentest in February" is factually true and strategically insufficient. The board, operating under personal liability exposure, wants to know whether your testing program catches incidents at the pace they occur. The answer "we run continuous validation across all production assets weekly, with trigger-based scans on critical CVE publications, and we have structured audit logs showing every test executed and every finding verified" is a different category of answer. It is also the answer continuous validation makes possible.
The three regulations share a common demand: provable security. Not asserted. Not inferred from a consultant's report. Provable, with evidence, on demand. Point-in-time testing cannot provide this. Continuous validation can.
You do not need to replace your team, your tools, or your existing pentest engagements to begin. Continuous validation layers on top of what you already have. Start with one workflow. Run it for two weeks. Compare the results against your last pentest report. The delta tells you whether your current testing cadence is sufficient.
First workflows to automate.
External asset discovery and vulnerability scanning. This is the highest-ROI starting point because the attack surface changes fastest on the external perimeter. Schedule it weekly. The first run establishes a baseline. Every subsequent run produces a diff. New subdomains, new open ports, new vulnerabilities. This is information your quarterly pentest cannot provide because it only sees the perimeter during a two-week window every 13 weeks.
Remediation retesting. Take the top 5 findings from your last pentest report. Build a workflow that reproduces exactly the tests the pentesters performed against those findings. Schedule it weekly. You now have continuous verification that your highest-severity fixes are holding. If one regresses, you know within a week, not three months later when the next pentest arrives.
CVE-triggered scanning. Build a workflow that inventories assets running a specific technology, say Apache Struts or Log4j, and scans for new CVEs in that stack. Trigger it when a critical CVE publishes. The gap between CVE disclosure and your first validated finding goes from days to minutes.
How to measure coverage improvement.
Track three numbers. First, the percentage of your known asset inventory that is tested on a recurring schedule. Most organizations start below 30%. Continuous validation drives this toward 100%. Second, the mean time between finding discovery and remediation verification. This is the regression window. Quarterly pentests produce a window measured in months. Continuous validation shrinks it to the interval between scheduled runs. Third, the number of findings that recur after initial remediation. This is your silent regression rate. Most teams do not track it because point-in-time testing cannot measure it. Continuous validation surfaces it immediately.
What not to do.
Do not start by trying to automate everything. One workflow, one cadence, one set of assets. Prove the value on a narrow scope before expanding. Do not treat continuous validation as a replacement for human-led pentesting. It is a complement. The pentest finds novel vulnerabilities that automated scanning misses. Continuous validation ensures the known vulnerabilities, and the fixes for them, do not degrade between pentests. The two models work together. Point-in-time testing becomes the deep, creative layer. Continuous validation becomes the persistent verification layer. That is the right division of labor.
Point-in-time testing is not broken because pentesters are bad at their jobs. It is broken because the world it was designed for no longer exists. Infrastructure changes daily. Attackers operate in minutes. Regulators demand evidence, not deliverables.
The alternative is continuous validation: scheduled testing that runs on your cadence, trigger-based workflows that fire on events, automated remediation retesting that closes the regression window, and structured evidence output that satisfies DORA, NIS2, and the SEC.
You already have the methodology. Your engineers know what to test and how to test it. What you need is the execution layer that runs that methodology repeatedly, at scale, on schedule, with output you can query and audit.
We built that layer. 15 million jobs across half a million machines, running workflows our users designed, producing structured evidence every run. It works. The quarterly pentest ritual had its run. It is time to move on.
Get a personalized demo
A 30-minute walkthrough. We map the platform to your stack and answer pricing and deployment questions for your environment.