Utilities
Pull domains, paths, or params from URL lists
Extract one URL component per run.
overview
What xurl does
xurl takes a file of URLs via --input and emits one component: domains, apexes, paths, query, params, values, or a custom format string. Fixed modes cover common reshapes; format mode builds a line when they do not.
Reach for it between a crawler or archive dump and a fuzzer when you need parameter names, unique apexes, or paths alone. Add --unique so downstream stages get a deduplicated column instead of a messy URL dump.
Trickest runs xurl as a managed Utilities node. Wire FILE in, take FILE and FOLDER out. Prefer unfurl when you want the same job with a different mode set; use katana or gau when you still need to collect the URLs.
source github.com/hueristiq/xurl
use cases
Where xurl fits
Extract parameter names for fuzzing
Run params mode over a URL set to get a clean list of parameter names, the exact input a parameter fuzzer or value tester wants next.
Isolate apex domains for scoping
Use apexes mode to reduce a pile of URLs to their root domains, deduplicated, so you can scope or group work by organization.
Pull paths for content discovery
Extract the paths from collected URLs to seed a content-discovery wordlist tailored to how the target structures its routes.
Reshape output with a format string
Use format mode to assemble a custom line from URL parts when none of the fixed modes produces the exact shape the next tool expects.
reference
xurl inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| mode | STRING | · | Mode: domains, apexes, paths, query, params, values, or format. |
| input | FILE | --input | Input file of URLs. |
| format-string | STRING | · | Format string used in format mode to build custom output. |
| unique | BOOLEAN | --unique | Output unique values only. |
| verbosity | STRING | --verbosity | Log level: debug, info, warning, error, fatal, or silent (default info). |
| monochrome | BOOLEAN | --monochrome | Disable output content coloring. |
Showing key inputs. xurl exposes 6 inputs in total.
example
Run xurl
# xurl: unique hostnames from a crawl dumpxurl domains --input urls.txt --uniqueexample.comwww.example.comapi.example.comstaging.example.comcdn.example.commail.example.comdev.example.comapp.example.comguidance
Choosing xurl
Use xurl when a stage already produced URLs and the next tool needs one clean component: params for fuzzing, apexes for scoping, or paths for content discovery. It does not fetch URLs. Collect first with katana or gau, then reshape here. Prefer unfurl if you already standardize on that CLI.
unfurl
Same URL-component extract job. Pick by mode names and format syntax you already use.
gau
Passive URL collection from archives. Feed its output into xurl when you need parts, not full URLs.
faq
xurl questions
related
More Utilities tools
cewl
Spider a URL and return a wordlist for password crackers.
dnsgen
Wordlist and mined-word subdomain permutation.
pup
CSS selectors over HTML, the jq counterpart for markup.
unfurl
Extract chosen URL parts from stdin into clean line lists.
whisper
Speech to txt, vtt, srt, and json from an audio file or folder.
youtube-transcript
Public caption tracks to transcript.txt and results.jsonl.
Run xurl yourself
A file of URLs feeds xurl, which extracts one component, here the parameter names, and writes the clean, deduplicated list as an output for the next stage.
Facts on this page come from the live Trickest tool library.