Can CeWL crawl pages behind a login?

Yes. Set --auth_user, --auth_pass, and --auth_type (basic or digest) so it reaches words on authenticated pages. Route through an intercepting proxy with --proxy_host and --proxy_port if needed.

Does CeWL show how often each word appears?

With --count, every word is printed alongside the number of times CeWL saw it, so you can sort by frequency. Without it, the output is a plain list of unique words.

How is CeWL different from katana?

katana maps links and endpoints. CeWL harvests words for cracking and fuzzing. Use katana to chart the app surface; use CeWL when you need the site's own vocabulary as a list.

What should run after CeWL?

A password cracker or a content/parameter fuzzer usually consumes the list next, so guesses match the target's own terms instead of a generic dictionary. clean-wordlist can tidy the harvest first.

Updated Jul 14, 2026

Utilities

Build a target-specific wordlist from site text

Spider a URL and return a wordlist for password crackers.

Build a target-specific password wordlist Harvest emails and metadata while crawling Crawl authenticated pages

overview

What cewl does

CeWL spiders a URL to a set depth and emits words found on the pages as a cracking wordlist. Product names, jargon, and employee terms beat a generic dictionary for one organization.

Set -d for crawl depth, --min_word_length to drop short tokens, --email and --meta for addresses and document authors, and leave --offsite off unless you mean to leave the host.

Run early to seed a target-specific list, then pass it to a cracker or fuzzer. For endpoints instead of words, use katana. Trickest runs CeWL as a managed node with file and folder output.

source github.com/digininja/CeWL

use cases

Where cewl fits

Build a target-specific password wordlist

Spider a company site to harvest its own terminology, then feed the list to a cracker so guesses match the words employees reuse in passwords.

Harvest emails and metadata while crawling

Enable --email and --meta to pull addresses and document author data alongside the wordlist, useful for OSINT and username generation.

Crawl authenticated pages

Supply basic or digest credentials so CeWL reaches words on pages behind a login, where the richer internal vocabulary tends to live.

Seed a fuzzing stage with real words

Pass the generated wordlist into a content or parameter fuzzer so brute-force guesses draw on terms the target itself exposes.

reference

cewl inputs and flags

25 inputs

Name	Type	Flag	Description
url	STRING	·	The site to spider (passed as the positional target URL).
depth	STRING	-d	Depth to spider to (default 2).
min-word-length	STRING	--min_word_length	Minimum word length to keep (default 3).
count	BOOLEAN	--count	Show the count for each word found.
with-numbers	BOOLEAN	--with-numbers	Accept words containing numbers, not only letters.
email	BOOLEAN	--email	Include email addresses found while crawling.
meta	BOOLEAN	--meta	Include document metadata in the results.
offsite	BOOLEAN	--offsite	Let the spider visit sites beyond the target.

Showing key inputs. cewl exposes 25 inputs in total.

Full flag reference (25 inputs)

Name	Type	Flag	Description
url	STRING	·	The site to spider (passed as the positional target URL).
depth	STRING	-d	Depth to spider to (default 2).
min-word-length	STRING	--min_word_length	Minimum word length to keep (default 3).
max-word-length	STRING	--max_word_length	Maximum word length (default unset).
count	BOOLEAN	--count	Show the count for each word found.
with-numbers	BOOLEAN	--with-numbers	Accept words containing numbers, not only letters.
lowercase	BOOLEAN	--lowercase	Lowercase all parsed words.
groups	BOOLEAN	--groups	Return groups of words as well as single words.
email	BOOLEAN	--email	Include email addresses found while crawling.
meta	BOOLEAN	--meta	Include document metadata in the results.
offsite	BOOLEAN	--offsite	Let the spider visit sites beyond the target.
allowed	STRING	--allowed	Regex pattern a path must match to be followed.
exclude	FILE	--exclude	File listing paths to exclude from the crawl.
header	STRING	--header	Extra header in name:value form; can be passed multiple times.
user-agent	STRING	--ua	User agent string to send with requests.
auth-type	STRING	--auth_type	Authentication type: digest or basic.
auth-user	STRING	--auth_user	Authentication username for pages behind a login.
auth-pass	STRING	--auth_pass	Authentication password.
proxy-host	STRING	--proxy_host	Proxy host to route requests through.
proxy-port	STRING	--proxy_port	Proxy port (default 8080).
proxy-username	STRING	--proxy_username	Username for the proxy, if required.
proxy-password	STRING	--proxy_password	Password for the proxy, if required.
convert-umlauts	BOOLEAN	--convert-umlauts	Convert common Latin-1 umlauts (ä to ae, ö to oe, ü to ue, ß to ss).
verbose	BOOLEAN	--verbose	Verbose output.
debug	BOOLEAN	--debug	Print extra debug information.

example

Run cewl

cewl · command

# spider example.com to depth 2, keep words 5+ chars, show counts, allow digitscewl -d 2 --min_word_length 5 --count --with-numbers https://example.com > wordlist.txt

sample output

example, 318security, 205platform, 154research, 132engineering, 118automation, 96credentials, 74workflow, 61onboarding, 47

guidance

Choosing cewl

Reach for CeWL when a target's own vocabulary will beat a generic dictionary, such as cracking credentials at one organization. Skip it for a static reusable wordlist with nothing to crawl. For links and endpoints rather than raw words, use katana.

katana

Crawls for endpoints and links, not words. Use it to map an app; use CeWL to harvest vocabulary.

gospider

Fast web spider for URLs. CeWL is purpose-built to emit a wordlist for cracking.

clean-wordlist

Cleans and normalizes a list. Run it after CeWL to tidy the harvested words.

faq

cewl questions

Bound the crawl with -d and stay on-site by leaving --offsite off, then raise --min_word_length to drop short, low-value tokens. Add --with-numbers when the target mixes digits into terms, and --count to rank words by frequency.

Run cewl yourself

A target URL feeds CeWL, which spiders the site, collects its words, and writes a target-specific wordlist ready for cracking or fuzzing.

Open cewl in the library Build a workflow with it

Facts on this page come from the live Trickest tool library.