loading
Loading content
loading
Discovery
Crawl a domain list and scan responses for endpoints, secrets, tokens, and juicy files.
overview
cariddi sits after live-host probing: feed it domains, crawl each site, and leave with triaged leads instead of a raw URL dump. One pass can surface juicy endpoints, leaked secrets, interesting extensions, errors, and exposed info.
Hunting modes are separate flags. Use -s for secrets, -e for juicy endpoints, -ext for file extensions, -err for errors, and -info for exposed info. Add custom secret regexes with -sf, shape requests with headers or -ua/-rua, and keep large crawls polite with -c and -d.
katana goes deeper on JavaScript parsing; trufflehog owns repo history. cariddi is the crawl-plus-hunt middle ground. Trickest runs it as a managed Discovery node that reads a domains file and writes a results folder; prefer -json for structured downstream use.
use cases
Hand cariddi a list of live domains and let it crawl every one, collecting endpoints, secrets, and tokens into a single structured set of leads.
Run the secrets mode with -s, optionally with custom regexes via -sf, to flag exposed keys and tokens in page bodies and served scripts across the crawled surface.
Enable -e and -ext to surface interesting paths and files like backups, database dumps, and configs that deserve manual review.
Set concurrency with -c, a per-page delay with -d, and a custom or random user agent so a wide crawl moves quickly without tripping rate limits or looking obviously automated.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| domains | FILE | · | List of domains to crawl, the usual pipeline input. |
| hunt-secrets | BOOLEAN | -s | Hunt for secrets and API keys in crawled content. |
| juicy-endpoint | BOOLEAN | -e | Hunt for juicy endpoints worth manual review. |
| juicy-extensions | STRING | -ext | Hunt for juicy file extensions, 1 (juicy) to 7 (not juicy). |
| hunt-info | BOOLEAN | -info | Hunt for useful information exposed in crawled sites. |
| json | BOOLEAN | -json | Print the output as JSON for structured downstream use. |
| concurrency-level | STRING | -c | Concurrency level (default 20). |
| custom-regex-secrets | FILE | -sf | External file of custom regexes for secret hunting. |
Showing key inputs. cariddi exposes 22 inputs in total.
| Name | Type | Flag | Description |
|---|---|---|---|
| domains | FILE | · | List of domains to crawl, the primary pipeline input. |
| hunt-secrets | BOOLEAN | -s | Hunt for secrets and API keys in crawled content. |
| juicy-endpoint | BOOLEAN | -e | Hunt for juicy endpoints worth manual review. |
| juicy-extensions | STRING | -ext | Hunt for juicy file extensions, an integer from 1 (juicy) to 7 (not juicy). |
| hunt-info | BOOLEAN | -info | Hunt for useful information exposed in crawled sites. |
| err | BOOLEAN | -err | Hunt for errors surfaced in crawled sites. |
| json | BOOLEAN | -json | Print the output as JSON to stdout. |
| plain | BOOLEAN | -plain | Print only the results, without the banner or extras. |
| debug | BOOLEAN | -debug | Print debug information while crawling. |
| intensive | BOOLEAN | -intensive | Crawl searching for resources matching the second-level domain. |
| concurrency-level | STRING | -c | Concurrency level (default 20). |
| delay-between-crawl | STRING | -d | Delay between one crawled page and the next. |
| timeout | STRING | -t | Timeout in seconds for each request (default 10). |
| proxy | STRING | -proxy | Route requests through a proxy (http and socks5 supported). |
| headers | STRING | -headers | Custom headers for each request, e.g. "Cookie: auth=yes;;Client: type=2". |
| headersfile | FILE | -headersfile | Read custom headers from an external file (same format as -headers). |
| user-agent | STRING | -ua | Use a custom User-Agent string. |
| random-agent | BOOLEAN | -rua | Use a random browser User-Agent on every request. |
| custom-regex-secrets | FILE | -sf | External file of custom regexes for secret hunting (one per line). |
| external-file | FILE | -ef | External file of custom parameters for endpoint hunting (one per line). |
| ignore-url-element | STRING | -i | Ignore URLs containing any element in this list. |
| ignore-url-element-file | FILE | -it | Ignore URLs containing any line from this file. |
example
# crawl live hosts and hunt secrets, endpoints, and juicy filescat live-hosts.txt | cariddi -s -e -ext 2 -info -json -c 40{"url":"https://example.com/","method":"GET","status_code":200,"words":842,"lines":63,"content_type":"text/html"}{"url":"https://example.com/assets/app.min.js","method":"GET","status_code":200,"words":5120,"lines":18,"content_type":"application/javascript","matches":{"secrets":[{"name":"Stripe API Key","match":"sk_live_EXAMPLEonlyNotReal00"}]}}{"url":"https://example.com/api/v1/orders?id=42","method":"GET","status_code":200,"words":210,"lines":12,"content_type":"application/json","matches":{"parameters":[{"name":"id","attacks":["sqli","idor"]}]}}{"url":"https://example.com/static/backup.sql","method":"GET","status_code":200,"content_type":"application/sql","matches":{"filetype":{"extension":"sql","severity":1}}}{"url":"https://example.com/.env","method":"GET","status_code":200,"words":22,"lines":6,"content_type":"text/plain","matches":{"secrets":[{"name":"Generic API Key","match":"api_key=EXAMPLE000placeholder"}]}}{"url":"https://dev.example.com/legacy/","method":"GET","status_code":500,"words":95,"lines":9,"content_type":"text/html","matches":{"errors":[{"name":"Go Stack Trace","match":"runtime error:"}]}}{"url":"https://example.com/.git/config","method":"GET","status_code":200,"words":30,"lines":5,"content_type":"text/plain","matches":{"infos":[{"name":"Git config","match":"[core]"}]}}guidance
Reach for cariddi when you want one crawl to pull secrets, endpoints, and tokens from live domains. Run it after subfinder and httpx. Prefer katana for deep JavaScript parsing; prefer trufflehog for repository secret history.
Deeper crawler with JavaScript parsing. cariddi adds built-in secret, token, and extension hunting on the crawl.
Fast crawler for link discovery. cariddi layers secret and juicy-endpoint detection over the crawl.
Pulls secrets from JavaScript files. cariddi hunts secrets across a full multi-domain crawl.
faq
related
Extract URLs and endpoints from Android APK files.
Web path scanner.
Crawl pages, harvest potential parameters, write a custom wordlist.
Recursive content discovery with smart defaults and rich response filters.
Maintained gau fork for passive archive URL collection.
Extract JavaScript file URLs from a page or URL list.
A domains list is probed by httpx, then cariddi crawls the live hosts and writes the secrets, endpoints, and tokens it finds as a queryable output.
Facts on this page come from the live Trickest tool library.