Discovery
Crawl domains and extract secrets, endpoints, and files
Crawl a domain list and scan responses for endpoints, secrets, tokens, and juicy files.
overview
What cariddi does
cariddi sits after live-host probing: feed it domains, crawl each site, and leave with triaged leads instead of a raw URL dump. One pass can surface juicy endpoints, leaked secrets, interesting extensions, errors, and exposed info.
Hunting modes are separate flags. Use -s for secrets, -e for juicy endpoints, -ext for file extensions, -err for errors, and -info for exposed info. Add custom secret regexes with -sf, shape requests with headers or -ua/-rua, and keep large crawls polite with -c and -d.
katana goes deeper on JavaScript parsing; trufflehog owns repo history. cariddi is the crawl-plus-hunt middle ground. Trickest runs it as a managed Discovery node that reads a domains file and writes a results folder; prefer -json for structured downstream use.
use cases
Where cariddi fits
Crawl and triage a whole scope at once
Hand cariddi a list of live domains and let it crawl every one, collecting endpoints, secrets, and tokens into a single structured set of leads.
Hunt leaked secrets and API keys
Run the secrets mode with -s, optionally with custom regexes via -sf, to flag exposed keys and tokens in page bodies and served scripts across the crawled surface.
Find juicy endpoints and exposed files
Enable -e and -ext to surface interesting paths and files like backups, database dumps, and configs that deserve manual review.
Tune a polite large-scale crawl
Set concurrency with -c, a per-page delay with -d, and a custom or random user agent so a wide crawl moves quickly without tripping rate limits or looking obviously automated.
reference
cariddi inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| domains | FILE | · | List of domains to crawl, the usual pipeline input. |
| hunt-secrets | BOOLEAN | -s | Hunt for secrets and API keys in crawled content. |
| juicy-endpoint | BOOLEAN | -e | Hunt for juicy endpoints worth manual review. |
| juicy-extensions | STRING | -ext | Hunt for juicy file extensions, 1 (juicy) to 7 (not juicy). |
| hunt-info | BOOLEAN | -info | Hunt for useful information exposed in crawled sites. |
| json | BOOLEAN | -json | Print the output as JSON for structured downstream use. |
| concurrency-level | STRING | -c | Concurrency level (default 20). |
| custom-regex-secrets | FILE | -sf | External file of custom regexes for secret hunting. |
Showing key inputs. cariddi exposes 22 inputs in total.
Full flag reference (22 inputs)
| Name | Type | Flag | Description |
|---|---|---|---|
| domains | FILE | · | List of domains to crawl, the primary pipeline input. |
| hunt-secrets | BOOLEAN | -s | Hunt for secrets and API keys in crawled content. |
| juicy-endpoint | BOOLEAN | -e | Hunt for juicy endpoints worth manual review. |
| juicy-extensions | STRING | -ext | Hunt for juicy file extensions, an integer from 1 (juicy) to 7 (not juicy). |
| hunt-info | BOOLEAN | -info | Hunt for useful information exposed in crawled sites. |
| err | BOOLEAN | -err | Hunt for errors surfaced in crawled sites. |
| json | BOOLEAN | -json | Print the output as JSON to stdout. |
| plain | BOOLEAN | -plain | Print only the results, without the banner or extras. |
| debug | BOOLEAN | -debug | Print debug information while crawling. |
| intensive | BOOLEAN | -intensive | Crawl searching for resources matching the second-level domain. |
| concurrency-level | STRING | -c | Concurrency level (default 20). |
| delay-between-crawl | STRING | -d | Delay between one crawled page and the next. |
| timeout | STRING | -t | Timeout in seconds for each request (default 10). |
| proxy | STRING | -proxy | Route requests through a proxy (http and socks5 supported). |
| headers | STRING | -headers | Custom headers for each request, e.g. "Cookie: auth=yes;;Client: type=2". |
| headersfile | FILE | -headersfile | Read custom headers from an external file (same format as -headers). |
| user-agent | STRING | -ua | Use a custom User-Agent string. |
| random-agent | BOOLEAN | -rua | Use a random browser User-Agent on every request. |
| custom-regex-secrets | FILE | -sf | External file of custom regexes for secret hunting (one per line). |
| external-file | FILE | -ef | External file of custom parameters for endpoint hunting (one per line). |
| ignore-url-element | STRING | -i | Ignore URLs containing any element in this list. |
| ignore-url-element-file | FILE | -it | Ignore URLs containing any line from this file. |
example
Run cariddi
# crawl live hosts and hunt secrets, endpoints, and juicy filescat live-hosts.txt | cariddi -s -e -ext 2 -info -json -c 40{"url":"https://example.com/","method":"GET","status_code":200,"words":842,"lines":63,"content_type":"text/html"}{"url":"https://example.com/assets/app.min.js","method":"GET","status_code":200,"words":5120,"lines":18,"content_type":"application/javascript","matches":{"secrets":[{"name":"Stripe API Key","match":"sk_live_EXAMPLEonlyNotReal00"}]}}{"url":"https://example.com/api/v1/orders?id=42","method":"GET","status_code":200,"words":210,"lines":12,"content_type":"application/json","matches":{"parameters":[{"name":"id","attacks":["sqli","idor"]}]}}{"url":"https://example.com/static/backup.sql","method":"GET","status_code":200,"content_type":"application/sql","matches":{"filetype":{"extension":"sql","severity":1}}}{"url":"https://example.com/.env","method":"GET","status_code":200,"words":22,"lines":6,"content_type":"text/plain","matches":{"secrets":[{"name":"Generic API Key","match":"api_key=EXAMPLE000placeholder"}]}}{"url":"https://dev.example.com/legacy/","method":"GET","status_code":500,"words":95,"lines":9,"content_type":"text/html","matches":{"errors":[{"name":"Go Stack Trace","match":"runtime error:"}]}}{"url":"https://example.com/.git/config","method":"GET","status_code":200,"words":30,"lines":5,"content_type":"text/plain","matches":{"infos":[{"name":"Git config","match":"[core]"}]}}guidance
Choosing cariddi
Reach for cariddi when you want one crawl to pull secrets, endpoints, and tokens from live domains. Run it after subfinder and httpx. Prefer katana for deep JavaScript parsing; prefer trufflehog for repository secret history.
katana
Deeper crawler with JavaScript parsing. cariddi adds built-in secret, token, and extension hunting on the crawl.
gospider
Fast crawler for link discovery. cariddi layers secret and juicy-endpoint detection over the crawl.
secretfinder
Pulls secrets from JavaScript files. cariddi hunts secrets across a full multi-domain crawl.
faq
cariddi questions
related
More Discovery tools
apkurlgrep
Extract URLs and endpoints from Android APK files.
crawlergo
Browser-driven crawler that harvests requests for downstream scanners.
dirsearch
Web path scanner.
fallparams
Crawl pages, harvest potential parameters, write a custom wordlist.
feroxbuster
Recursive content discovery with smart defaults and rich response filters.
gau
Passive known-URL fetch from public web archives.
Run cariddi yourself
A domains list is probed by httpx, then cariddi crawls the live hosts and writes the secrets, endpoints, and tokens it finds as a queryable output.
Facts on this page come from the live Trickest tool library.