loading
Loading content
loading
Utilities
Regex-clean a wordlist before the slow stage reads it.
overview
Sit clean-wordlist between a list producer and a consumer. After CeWL, a merge, or an archive dump, it writes a tidied file so ffuf or a DNS resolver does not burn budget on garbage lines.
Reach for it when a list came from a crawl, scrape, or merge. Malformed tokens, blank lines, and encoding artifacts inflate request counts without adding hits.
Input is a single wordlist file; output is a FILE plus a FOLDER. It is a regex cleaner, not a deduper. For uniqueness alone, prefer duplicut or anew. Trickest runs it as a managed Utilities node.
source github.com/BonJarber/SecUtils/tree/master/clean_wordlist
use cases
Run a harvested wordlist through clean-wordlist so a content fuzzer like ffuf or gobuster spends its requests on plausible paths instead of malformed lines.
Clean the output of a generator like CeWL to remove crawl artifacts and noise before the list reaches a cracker or resolver.
After combining several wordlists, run clean-wordlist to drop the malformed and encoding-broken entries that merging tends to introduce.
Clean a subdomain wordlist before a DNS brute-force node reads it, so a resolver like puredns spends its lookups on real candidate labels rather than junk.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| input | FILE | · | Input wordlist file to clean (positional; the tool exposes no flags). |
Showing key inputs. clean-wordlist exposes 1 inputs in total.
example
# clean a harvested wordlist, cleaned entries to a new fileclean_wordlist.sh raw-wordlist.txt > clean-wordlist.txtadminapiapi/v1assetsbackupconfigloginrobots.txtuploadswp-login.phpguidance
Use clean-wordlist when a list comes from a crawl, archive, or merge and you want junk gone before a slow stage reads it. For deduplication alone, duplicut is sharper. For generating words, use CeWL, then clean afterward.
Removes duplicates from very large wordlists fast. clean-wordlist drops malformed entries via regex.
Appends only new lines to a file. Useful for dedupe across runs, not for cleaning malformed entries.
Generates a wordlist from a site. Run clean-wordlist after it to tidy harvested words.
faq
related
Spider a URL and return a wordlist for password crackers.
Wordlist and mined-word subdomain permutation.
Import, export, and upsert workflow data against an Airtable base.
Decode Android APK files into smali sources and resources.
Check a file's values against conditions and exit with a matching code.
Authenticated Route53 hosted-zone inventory.
A raw wordlist feeds clean-wordlist, which runs its regexes over each line and writes a tidied list ready for fuzzing or cracking.
Facts on this page come from the live Trickest tool library.