What is a masked 404?

A response that returns HTTP 200 but renders a not-found page. It looks live to a plain status check, so it slips through content-discovery output and inflates the URL list a scanner then has to work through.

Can I not filter 404s by status code in ffuf or gobuster instead?

Status filters like ffuf's -fc or gobuster's status-codes-blacklist drop responses that report a 404 code. A masked 404 answers 200, so those filters keep it. 404checker renders the page to catch the soft failures a status-code rule cannot see.

Where does 404checker fit in a content-discovery workflow?

Run it after a fuzzer or crawler and before probing or scanning. ffuf, feroxbuster, or a crawler produce the candidate URLs; 404checker removes the ones that fake a real page, so only genuine endpoints reach the next node.

Why must input URLs be sorted?

Sorting groups similar URLs together so the browsers reuse context and move through the list faster. The upstream tool calls out sorted input as the way to improve throughput on large sets.

Does 404checker need a real browser to run?

Yes. It launches headless browser processes (Playwright) to render each URL, which is how it tells a real page from a 200 that only shows not-found content. Set --processes to control how many browsers run in parallel; the default matches your CPU count.

What does 404checker produce on Trickest?

A file and a folder holding the URLs that resolve to real pages, with the masked 404s stripped out, ready to feed the next probing or scanning node.

Updated Jul 14, 2026

Discovery

Filter masked 404 pages out of a content-discovery URL list

Check whether a URL redirects to a masked 404 page.

Clean a content-discovery URL set Catch what status-code filters miss Cut false positives before scanning

overview

What 404checker does

404checker reads a list of URLs and separates the ones that resolve to a real page from the ones that only pretend to. A masked 404 answers with HTTP 200 but renders a not-found page, and those false positives quietly poison content-discovery and crawling output. The script drives headless browser processes to render each URL so a soft failure cannot hide behind a 200 status.

It is built for speed on large URL sets. You hand it a sorted input file, set a thread count, and cap how many URLs each pass handles so the overflow spills to the next run instead of stalling. Sorting the input first groups similar responses so the browsers move through them faster.

On Trickest, 404checker is a Discovery node that takes a file of URLs and writes a file and a folder of the ones that survive. Drop it after a content-discovery or crawling stage to clean the list before you probe or scan, so downstream tools never spend time on pages that do not exist.

source github.com/carlospolop/404checker

use cases

Where 404checker fits

Clean a content-discovery URL set

Run 404checker on the output of a fuzzer or crawler to drop URLs that return 200 but render a not-found page, so only real endpoints reach the next stage.

Catch what status-code filters miss

ffuf and gobuster drop hard 404s by status code, but a soft 404 answers 200 and slips through. Run 404checker on the survivors to remove the masked pages those filters leave behind.

Cut false positives before scanning

Filter masked 404s out of a candidate list so a vulnerability scanner spends its time on pages that exist instead of decoy responses.

Validate redirect behavior at scale

Push a large sorted URL list through multiple browser processes to flag soft-404 redirects across a whole estate in one pass.

reference

404checker inputs and flags

6 inputs

Name	Type	Flag	Description
input-file	FILE	--input_file	Input file with URLs, one per line.
threads	STRING	--threads	Number of threads (default 50).
processes	STRING	--processes	Number of browser processes (default number of CPUs).
max-urls	STRING	--max-urls	Max URLs per pass; the rest spill over (default 50000).
user-agent	STRING	--user-agent	User-Agent string to send with each request.
verbose	BOOLEAN	--verbose	Print verbose output.

Showing key inputs. 404checker exposes 6 inputs in total.

example

Run 404checker

404checker · command

# keep only URLs that resolve to a real page, dropping masked 404spython3 404checker.py --input_file urls.txt --output_file real-urls.txt --threads 50 --processes 4

sample output

https://example.com/admin/loginhttps://example.com/api/v1/statushttps://example.com/dashboardhttps://example.com/uploads/https://example.com/.git/confighttps://example.com/backup/db.sqlhttps://203.0.113.24/portal/…  (4,812 URLs in, 2,190 real, 2,622 masked 404s dropped)

guidance

Choosing 404checker

Use 404checker as a filtering stage between content discovery and probing, when a tool has produced a URL list and you want to remove pages that answer 200 but are really not-found. It does not discover or crawl, so feed it the output of ffuf, feroxbuster, or a crawler.

httpx

Probes hosts for status, title, and size. Use it for liveness and classification, not specifically soft-404 detection.

ffuf

Discovers content and filters responses by size or words. 404checker is purpose-built for the masked-404 case.

faq

404checker questions

Point --input_file at a file of URLs (one per line) and --output_file at where the valid URLs should land, e.g. `python3 404checker.py -i urls.txt -o real-urls.txt`. Add --threads and --processes to scale it up. On Trickest, wire a URL file into the node's input instead of passing the flags by hand.

Run 404checker yourself

A URL list feeds 404checker, which renders each one and writes the URLs that resolve to real pages as a queryable output.

Run 404checker in Trickest View the workflow

Facts on this page come from the live Trickest tool library.

Name

Type

Flag

Description

input-file

FILE

--input_file

Input file with URLs, one per line.

threads

STRING

--threads

Number of threads (default 50).

processes

STRING

--processes

Number of browser processes (default number of CPUs).

max-urls

STRING

--max-urls

Max URLs per pass; the rest spill over (default 50000).

user-agent

STRING

--user-agent

User-Agent string to send with each request.

verbose

BOOLEAN

--verbose

Print verbose output.

https://example.com/admin/loginhttps://example.com/api/v1/statushttps://example.com/dashboardhttps://example.com/uploads/https://example.com/.git/confighttps://example.com/backup/db.sqlhttps://203.0.113.24/portal/… (4,812 URLs in, 2,190 real, 2,622 masked 404s dropped)

Filter masked 404 pages out of a content-discovery URL list

What 404checker does

Where 404checker fits

Clean a content-discovery URL set

Catch what status-code filters miss

Cut false positives before scanning

Validate redirect behavior at scale

404checker inputs and flags

Run 404checker

Choosing 404checker

httpx

ffuf

404checker questions

More Discovery tools

anew

apkurlgrep

aquatone

aws-s3-data-finder

bfac

cariddi

Run 404checker yourself

Filter masked 404 pages out of a content-discovery URL list

What 404checker does

Where 404checker fits

Clean a content-discovery URL set

Catch what status-code filters miss

Cut false positives before scanning

Validate redirect behavior at scale

404checker inputs and flags

Run 404checker

Choosing 404checker

httpx

ffuf

404checker questions

More Discovery tools

anew

apkurlgrep

aquatone

aws-s3-data-finder

bfac

cariddi

Run 404checker yourself