loading
loading
Discovery
A tool for fetching lots of URLs while staying 'nice' to servers.
overview
meg requests many paths for many hosts without hammering any single server. Instead of fetching every path from one host before moving on, it fetches one path across all hosts, then the next path across all hosts, and repeats. That ordering spreads load so each host sees requests spaced out, which keeps a wide sweep from looking like a flood to any one target.
The work it does is content discovery at scale. Give it a list of hosts and a list of paths, and it checks each combination, saving the raw responses to disk so you can grep for interesting status codes, headers, or bodies afterward. It is the tool you reach for when you want to look for the same handful of paths (config files, admin panels, known vulnerable endpoints) across a few hundred or a few thousand hosts at once.
On Trickest, meg is a Discovery node that takes hosts and paths and writes a folder of responses. Chain it after subdomain enumeration and a prober so it only fetches against hosts that are live, and tune the per-host delay and concurrency so a broad run stays within each target's tolerance.
source github.com/tomnomnom/meg
use cases
Request a single sensitive path, such as /.git/config or /actuator, across every host in scope and save the responses so you can grep for the ones that returned content.
Feed a paths file and a hosts file, then let meg interleave requests across hosts so a large content-discovery sweep spreads load instead of pounding any single server.
Save only responses with a specific status code so the output folder holds the hits worth reviewing rather than every 404, keeping triage fast on a wide run.
Place meg after subdomain enumeration and an httpx probe so it fetches against live hosts only, then pass its response folder to downstream grep or extraction stages.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| hosts-file | FILE | · | Line-by-line hosts with a protocol, the usual pipeline input from discovery. |
| paths-file | FILE | · | File of paths to request, one per line, fetched against every host. |
| path | STRING | · | Request a single path directly as an argument instead of a paths file (use one or the other). |
| delay | STRING | -d | Milliseconds between requests to the same host (default 5000), the lever that keeps a sweep polite. |
| concurency | STRING | -c | Concurrency level across hosts (default 20). |
| save-only-responses | STRING | -s | Save only responses with a specific status code, so the output folder holds the hits. |
| http-method | STRING | -X | HTTP method for each request (default GET). |
| custom-header | STRING | -H | Send a custom HTTP header with every request. |
Showing key inputs. meg exposes 12 inputs in total.
| Name | Type | Flag | Description |
|---|---|---|---|
| path | STRING | · | If you want to request just one path, you can specify it directly as an argument. You should use path or paths-file, arguments excludes one other. |
| delay | STRING | -d | Milliseconds between requests to the same host (default: 5000) |
| concurency | STRING | -c | Set the concurrency level (defaut: 20) |
| hosts-file | FILE | · | Line by line hosts with a protocol |
| paths-file | FILE | · | File with line by line paths |
| http-method | STRING | -X | HTTP method (default: GET) |
| http-timeout | STRING | -t | Set the HTTP timeout (default: 10000) |
| request-body | STRING | -b | Set the request body |
| custom-header | STRING | -H | Send a custom HTTP header |
| rawhttp-library | BOOLEAN | -r | Use the rawhttp library for requests (experimental) |
| follow-redirects | BOOLEAN | -L | Follow redirects / location header |
| save-only-responses | STRING | -s | Save only responses with specific status code |
example
# fetch /.git/config across live hosts; 2s delay, keep only HTTP 200smeg -d 2000 -c 20 -s 200 -H "User-Agent: meg" /.git/config hosts.txthttps://www.example.com/.git/config (200 OK) -> out/www.example.com/_.git_confighttps://app.example.com/.git/config (404) skipped (-s 200)https://api.example.com/.git/config (200 OK) -> out/api.example.com/_.git_confighttps://staging.example.com/.git/config (403) skipped (-s 200)https://dev.example.com/.git/config (200 OK) -> out/dev.example.com/_.git_confighttps://mail.example.com/.git/config (404) skipped (-s 200)https://vpn.example.com/.git/config (200 OK) -> out/vpn.example.com/_.git_configdone: 4/7 savedguidance
Reach for meg when you want to fetch the same set of paths across many hosts and keep the raw responses for review, without battering any single server. For probing liveness and metadata rather than saving bodies, use httpx. For templated vulnerability checks instead of raw fetches, use nuclei.
Probes hosts for status, title, and tech. Use it to find live hosts before meg fetches paths against them.
Fast per-host content fuzzer. Better for exhausting one host's paths; meg spreads a few paths across many hosts.
Template-based scanner. Tests for known issues rather than saving raw responses for manual grepping.
faq
related
Check whether a URL redirects to a masked 404 page.
Append lines to a file only if they are not already there.
Extract URLs and endpoints from Android APK files.
Visual inspection of websites across a large number of hosts.
Find suspicious files across a large set of AWS S3 buckets.
An automated tool that checks for backup artifacts that may disclose a web application's source code.
A host list is probed by httpx, the live hosts feed meg, which fetches a set of paths across all of them and writes the saved responses as a folder to review.
Facts on this page come from the live Trickest tool library.