Discovery
Hunt leaked URLs behind shortener codes
Search archives of URLs exposed via shortener services.
overview
What urlhunter does
urlhunter downloads daily archives of expanded short links (bit.ly, goo.gl) and matches a --keywords file against them. Scope collections with --date (single day or range). Point --archives at a folder so later runs reuse downloads.
Reach for it when private docs, internal dashboards, or one-time shares may have been shortened and later published in those archives. Every hit is a real expanded URL that once hid behind a short code.
Scope with --keywords and --date, then hand the FILE and FOLDER of expanded URLs to httpx or urlfinder. Pair with gau when you also need Wayback and Common Crawl coverage beyond shortener datasets.
source github.com/utkusen/urlhunter
use cases
Where urlhunter fits
Hunt leaked links by keyword
Search the shortener archives for an organization's domain or project name to surface short links that expanded to sensitive internal URLs.
Scope a search to a date range
Use a single date or a range to choose which daily collections urlhunter downloads and searches, keeping a run fast and focused.
Reuse a local archive folder
Point urlhunter at a persistent archives folder so later runs search already-downloaded collections instead of refetching them.
Feed OSINT findings downstream
Pass the matched URLs to a prober or a parameter extractor so discovered links get classified and tested like any other endpoint.
reference
urlhunter inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| keywords | FILE | --keywords | Path to a file of strings to search for in the archives. |
| date | STRING | --date | Single date (2020-11-20) or range (2020-11-10:2020-11-20) of collections to search. |
| archives | FOLDER | --archives | Directory holding the archive files; downloaded to ./archives on first run. |
Showing key inputs. urlhunter exposes 3 inputs in total.
example
Run urlhunter
# search a date range of shortener archives for keywordsurlhunter --keywords keywords.txt --date 2020-11-10:2020-11-20 --archives ./archives[*] Downloading the archive: 2020-11-20.tar.gz[*] Uncompressing the archive: 2020-11-20.tar.gz[*] Matches are being searched across 1 archive(s)...https://bit.ly/2Kx9dLp|https://example.com/shared/q3-forecast.pdfhttps://bit.ly/3aQ1wZt|https://docs.example.com/internal/onboardinghttps://goo.gl/f4Rt2M|https://staging.example.com/dashboard?ref=emailhttps://bit.ly/2Yh8Nre|http://198.51.100.24/backups/db.sql[*] urlhunter is finished! Matches saved to: output.txtguidance
Choosing urlhunter
Use urlhunter when you need OSINT over shortener-exposed URL archives, not general web-archive harvests. Search is offline after the --archives download. Pair with urlfinder or gau for broader passive URL coverage.
urlfinder
Passive URL discovery from archives and OSINT sources. urlhunter searches shortener archives only.
gau
Pulls URLs from Wayback and Common Crawl. urlhunter targets shortened-link collections.
waymore
Deep web-archive harvesting. urlhunter stays on the shortener-exposed URL dataset.
faq
urlhunter questions
related
More Discovery tools
apkurlgrep
Extract URLs and endpoints from Android APK files.
cariddi
Crawl a domain list and scan responses for endpoints, secrets, tokens, and juicy files.
crawlergo
Browser-driven crawler that harvests requests for downstream scanners.
dirsearch
Web path scanner.
fallparams
Crawl pages, harvest potential parameters, write a custom wordlist.
feroxbuster
Recursive content discovery with smart defaults and rich response filters.
Run urlhunter yourself
A keywords file feeds urlhunter, which searches the shortener archives over a date range and writes the matching expanded URLs as a queryable output.
Facts on this page come from the live Trickest tool library.