Does it fill out forms?

Yes. Provide --form-values and it fills and submits forms during the crawl, surfacing pages behind logins and searches.

How do I limit how far the crawl goes?

Set --max-crawled-count (default 200) and choose --filter-mode simple, smart, or strict so large targets stay manageable.

What does --fuzz-path do?

It fuzzes the target with common paths, and you can point --fuzz-path-dict at a custom dictionary. Use it when you want the browser crawl to also probe known path shapes.

What output should I use on Trickest?

A file and a folder of results. Set --output-mode json for structured requests the next stage can parse, or console for human-readable lines.

Updated Jul 15, 2026

Discovery

Crawl JavaScript apps with a real Chrome browser

Q: Can it feed a vulnerability scanner directly?

Yes. Use --push-to-proxy to send every captured request to a proxy address, so a passive scanner downstream tests the exact traffic the crawl produced.

A powerful browser crawler for web vulnerability scanners.

Crawl JavaScript-heavy applications Capture full requests for a scanner Exercise forms during the crawl

overview

What crawlergo does

crawlergo drives a real Chrome browser to crawl a target, so it sees the same pages a user does. It renders JavaScript, fills forms, and triggers DOM events, which lets it reach routes and requests that a static HTTP crawler never observes. The result is a fuller request map for single-page apps and JavaScript-heavy sites.

Because it collects full requests, not only links, crawlergo is built to feed a scanner. It can push every captured request to a proxy address, so a passive scanner downstream tests the exact traffic the crawl produced. Filtering modes keep the request set focused, and a max-crawled-count bounds how far each task runs.

Wire it as a Discovery node after subdomain discovery when you need the application's real attack surface. Hand the captured endpoints to a prober and a scanner, and set output-mode to json when a downstream stage needs structured requests.

source github.com/Qianlitp/crawlergo

use cases

Where crawlergo fits

Crawl JavaScript-heavy applications

Render the target in Chrome and trigger DOM events to reach routes and API calls that live in bundled JavaScript, not in static anchor tags.

Capture full requests for a scanner

Push every collected request to a proxy address so a passive scanner downstream tests the exact traffic the crawl produced, including POST bodies.

Exercise forms during the crawl

Supply form values so login and search forms get filled and submitted, surfacing post-authentication and result pages the crawler would otherwise miss.

Bound a wide crawl

Set a max-crawled-count and a filter mode to keep large targets manageable, then feed the resulting endpoint set into content discovery or fuzzing.

reference

crawlergo inputs and flags

23 inputs

Name	Type	Flag	Description
url	STRING	·	URLs to scan, separated by spaces (must be same host).
output-mode	STRING	--output-mode	Serialize output: console, json, or none.
filter-mode	STRING	--filter-mode	How collected requests are filtered: simple, smart, or strict (default smart).
max-crawled-count	STRING	--max-crawled-count	Maximum number of URLs visited in this task (default 200).
fuzz-path	BOOLEAN	--fuzz-path	Fuzz the target with common paths (default false).
form-values	STRING	--form-values	Custom filling text per form type, for example username and password.
push-to-proxy	STRING	--push-to-proxy	Push every captured request to this proxy address for downstream scanning.
robots-path	BOOLEAN	--robots-path	Resolve paths from /robots.txt (default false).

Showing key inputs. crawlergo exposes 23 inputs in total.

Full flag reference (23 inputs)

Name	Type	Flag	Description
url	STRING	·	URLs to scan, separated by spaces (must be same host)
fuzz-path	BOOLEAN	--fuzz-path	whether to fuzz the target with common paths. (default: false)
log-level	STRING	--log-level	log print Level, options include debug, info, warn, error and fatal. (default: "Info")
post-data	STRING	--post-data	set PostData to target and use POST method.
encode-url	BOOLEAN	--encode-url	whether to encode url with detected charset. (default: false)
filter-mode	STRING	--filter-mode	filtering Mode used for collected requests. Allowed mode:"simple", "smart" or "strict". (default: "smart")
form-values	STRING	--form-values	custom filling text for each form type. e.g.: -fv username=crawlergo_nice -fv password=admin123
output-mode	STRING	--output-mode	console print or serialize output. Allowed mode:"console" ,"json" or "none". (default: "console")
robots-path	BOOLEAN	--robots-path	whether to resolve paths from /robots.txt. (default: false)
max-tab-count	STRING	--max-tab-count	maximum Number of tabs allowed. (default: 8)
push-pool-max	STRING	--push-pool-max	maximum Number of concurrency when pushing results to proxy. (default: 10)
push-to-proxy	STRING	--push-to-proxy	every request in 'req_list' will be pushed to the proxy Address"
request-proxy	STRING	--request-proxy	all requests connect through defined proxy server.
custom-headers	STRING	--custom-headers	add additional Headers to each request. The input string will be called json.Unmarshal (default: {"Spider-Name": "crawlergo", "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36"})
fuzz-path-dict	FILE	--fuzz-path-dict	Fuzz dict
tab-run-timeout	STRING	--tab-run-timeout	the Timeout of a single tab task. (default: 20s)
before-exit-delay	STRING	--before-exit-delay	the Time of waiting before crawler exit. (default: 1s)
max-crawled-count	STRING	--max-crawled-count	the maximum Number of URLs visited by the crawler in this task. (default: 200)
event-trigger-mode	STRING	--event-trigger-mode	this Value determines how the crawler automatically triggers events.Allowed mode:"async" or "sync". (default: "async")
form-keyword-values	STRING	--form-keyword-values	custom filling text, fuzzy matched by keyword. e.g.: -fkv user=crawlergo_nice -fkv pass=admin123
ignore-url-keywords	STRING	--ignore-url-keywords	crawlergo will not crawl these URLs matched by Keywords. Default [logout quit exit])
event-trigger-interval	STRING	--event-trigger-interval	the Interval of triggering each event. (default: 100ms)
wait-dom-content-loaded-timeout	STRING	--wait-dom-content-loaded-timeout	the Timeout of waiting for a page dom ready. (default: 5s)

example

Run crawlergo

crawlergo · command

# crawl example.com with smart filtering and JSON outputcrawlergo --filter-mode smart --output-mode json --max-crawled-count 100 --fuzz-path https://example.com/

sample output

https://example.com/https://example.com/loginhttps://example.com/api/v1/usershttps://example.com/api/v1/products?id=1https://example.com/dashboardhttps://example.com/static/app.jshttps://example.com/search?q=testhttps://www.example.com/robots.txt

guidance

Choosing crawlergo

Use crawlergo when a target relies on JavaScript and you need a real browser to see its endpoints. It is heavier than a static crawler, so reach for katana when speed matters more than full rendering, and pair it with a proxy-based scanner to test the requests it captures.

katana

Fast crawler with optional JS parsing. Lighter than a full browser, quicker over wide scopes.

hakrawler

Lightweight active crawler. Faster to run, no headless rendering.

gospider

Fast Go spider for quick link discovery, no browser-driven event triggering.

faq

crawlergo questions

It drives a real Chrome browser, so it renders JavaScript and triggers DOM events. That reaches routes and requests a static HTTP crawler never observes.

Run crawlergo yourself

A target feeds crawlergo, which renders it in Chrome to harvest endpoints, then passes them to httpx for live probing before they land as a queryable output.

Run crawlergo in Trickest View the workflow

Facts on this page come from the live Trickest tool library.

Name

Type

Flag

Description

url

STRING

URLs to scan, separated by spaces (must be same host).

output-mode

STRING

--output-mode

Serialize output: console, json, or none.

filter-mode

STRING

--filter-mode

How collected requests are filtered: simple, smart, or strict (default smart).

max-crawled-count

STRING

--max-crawled-count

Maximum number of URLs visited in this task (default 200).

fuzz-path

BOOLEAN

--fuzz-path

Fuzz the target with common paths (default false).

form-values

STRING

--form-values

Custom filling text per form type, for example username and password.

push-to-proxy

STRING

--push-to-proxy

Push every captured request to this proxy address for downstream scanning.

robots-path

BOOLEAN

--robots-path

Resolve paths from /robots.txt (default false).

Name

Type

Flag

Description

url

STRING

URLs to scan, separated by spaces (must be same host)

fuzz-path

BOOLEAN

--fuzz-path

whether to fuzz the target with common paths. (default: false)

log-level

STRING

--log-level

log print Level, options include debug, info, warn, error and fatal. (default: "Info")

post-data

STRING

--post-data

set PostData to target and use POST method.

encode-url

BOOLEAN

--encode-url

whether to encode url with detected charset. (default: false)

filter-mode

STRING

--filter-mode

filtering Mode used for collected requests. Allowed mode:"simple", "smart" or "strict". (default: "smart")

form-values

STRING

--form-values

custom filling text for each form type. e.g.: -fv username=crawlergo_nice -fv password=admin123

output-mode

STRING

--output-mode

console print or serialize output. Allowed mode:"console" ,"json" or "none". (default: "console")

robots-path

BOOLEAN

--robots-path

whether to resolve paths from /robots.txt. (default: false)

max-tab-count

STRING

--max-tab-count

maximum Number of tabs allowed. (default: 8)

push-pool-max

STRING

--push-pool-max

maximum Number of concurrency when pushing results to proxy. (default: 10)

push-to-proxy

STRING

--push-to-proxy

every request in 'req_list' will be pushed to the proxy Address"

request-proxy

STRING

--request-proxy

all requests connect through defined proxy server.

custom-headers

STRING

--custom-headers

add additional Headers to each request. The input string will be called json.Unmarshal (default: {"Spider-Name": "crawlergo", "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36"})

fuzz-path-dict

FILE

--fuzz-path-dict

Fuzz dict

tab-run-timeout

STRING

--tab-run-timeout

the Timeout of a single tab task. (default: 20s)

before-exit-delay

STRING

--before-exit-delay

the Time of waiting before crawler exit. (default: 1s)

max-crawled-count

STRING

--max-crawled-count

the maximum Number of URLs visited by the crawler in this task. (default: 200)

event-trigger-mode

STRING

--event-trigger-mode

this Value determines how the crawler automatically triggers events.Allowed mode:"async" or "sync". (default: "async")

form-keyword-values

STRING

--form-keyword-values

custom filling text, fuzzy matched by keyword. e.g.: -fkv user=crawlergo_nice -fkv pass=admin123

ignore-url-keywords

STRING

--ignore-url-keywords

crawlergo will not crawl these URLs matched by Keywords. Default [logout quit exit])

event-trigger-interval

STRING

--event-trigger-interval

the Interval of triggering each event. (default: 100ms)

wait-dom-content-loaded-timeout

STRING

--wait-dom-content-loaded-timeout

the Timeout of waiting for a page dom ready. (default: 5s)

https://example.com/https://example.com/loginhttps://example.com/api/v1/usershttps://example.com/api/v1/products?id=1https://example.com/dashboardhttps://example.com/static/app.jshttps://example.com/search?q=testhttps://www.example.com/robots.txt

Crawl JavaScript apps with a real Chrome browser

What crawlergo does

Where crawlergo fits

Crawl JavaScript-heavy applications

Capture full requests for a scanner

Exercise forms during the crawl

Bound a wide crawl

crawlergo inputs and flags

Run crawlergo

Choosing crawlergo

katana

hakrawler

gospider

crawlergo questions

More Discovery tools

404checker

anew

apkurlgrep

aquatone

aws-s3-data-finder

bfac

Run crawlergo yourself

Crawl JavaScript apps with a real Chrome browser

What crawlergo does

Where crawlergo fits

Crawl JavaScript-heavy applications

Capture full requests for a scanner

Exercise forms during the crawl

Bound a wide crawl

crawlergo inputs and flags

Run crawlergo

Choosing crawlergo

katana

hakrawler

gospider

crawlergo questions

More Discovery tools

404checker

anew

apkurlgrep

aquatone

aws-s3-data-finder

bfac

Run crawlergo yourself