loading
Loading content
loading
Discovery
Maintained gau fork for passive archive URL collection.
overview
gauplus collects URLs public archives already recorded for a domain. It queries the Wayback Machine, OTX, and Common Crawl so you build a wide historical path list with no traffic to the target.
It is a maintained fork of gau. Pick archives with -providers, raise workers with -t, include subdomains with -subs, and drop noisy extensions with -b. Prefer katana when you need a live crawl that follows links and parses JavaScript.
Trickest provides gauplus as a managed Discovery node. Feed a domains file; take FILE and FOLDER URLs into httpx before fuzzing or scanning. Use gau if you want the upstream getallurls node with urlscan in the default provider set.
source github.com/bp0lr/gauplus
use cases
Pull every known URL for a domain from Wayback, OTX, and Common Crawl so a workflow starts with historical paths and parameters no live crawl would surface.
Surface old admin panels, retired API routes, and stale parameters that archives still hold but the current site no longer links to.
Enable subdomain inclusion with -subs and run several workers so a single pass gathers URLs across every subdomain of a target domain instead of one host at a time.
Skip image and font extensions with -b to keep the list lean, then pass it to httpx for liveness and on to parameter and content fuzzing stages.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| targets-domains | FILE | · | File of target domains to fetch known URLs for. |
| providers | STRING | -providers | Providers to fetch URLs from (default wayback,otx,commoncrawl). |
| include-subs | BOOLEAN | -subs | Include subdomains of the target domain in results. |
| extension-to-skip | STRING | -b | Extensions to skip, e.g. ttf,woff,svg,png,jpg. |
| threads | STRING | -t | Number of parallel workers fetching from providers (default 5). |
| output-json | BOOLEAN | -json | Write output as JSON. |
Showing key inputs. gauplus exposes 8 inputs in total.
| Name | Type | Flag | Description |
|---|---|---|---|
| targets-domains | FILE | · | File of target domains to fetch known URLs for. |
| providers | STRING | -providers | Providers to fetch URLs from (default wayback,otx,commoncrawl). |
| include-subs | BOOLEAN | -subs | Include subdomains of the target domain in results. |
| extension-to-skip | STRING | -b | Extensions to skip, e.g. ttf,woff,svg,png,jpg. |
| threads | STRING | -t | Number of parallel workers fetching from providers (default 5). |
| output-json | BOOLEAN | -json | Write output as JSON. |
| se-random-agent | BOOLEAN | -random-agent | Send a random user-agent on each request to a provider. |
| verbose | BOOLEAN | -v | Enable verbose mode. |
example
# pull known URLs for a domain and its subdomains, skipping noisy extensionsecho example.com | gauplus -subs -t 10 -b ttf,woff,svg,png,jpg -o urls.txthttps://example.com/robots.txthttps://example.com/index.php?id=1http://example.com/wp-login.phphttps://api.example.com/v1/users?id=42https://example.com/assets/app.min.jshttps://blog.example.com/2019/06/release-noteshttps://example.com/admin/login.aspxhttps://example.com/search?q=test&page=2https://dev.example.com/uploads/report.pdfguidance
Use gauplus when you want a domain's known URLs without touching the target. Archives do not prove what is live now, so pair it with httpx to filter responders. For an active crawl of the current site, use katana instead.
Upstream getallurls tool. Same archive-style job; gauplus adds provider control, -t workers, and -b extension skips.
Active crawler that walks the live site and parses JavaScript. Run alongside gauplus for current plus archived URLs.
Wayback-only URL fetcher. gauplus adds OTX and Common Crawl plus more tuning.
faq
related
Extract URLs and endpoints from Android APK files.
Web path scanner.
Crawl pages, harvest potential parameters, write a custom wordlist.
Recursive content discovery with smart defaults and rich response filters.
Extract JavaScript file URLs from a page or URL list.
Reconstruct commits from a dumped .git folder.
A file of domains feeds gauplus, which pulls known URLs from web archives and passes them to httpx, which keeps the ones still responding as a queryable output.
Facts on this page come from the live Trickest tool library.