Can I run CeWL online without installing Ruby?

Yes. On Trickest the CeWL node runs in the cloud, so you set a target URL and depth and read the wordlist output without installing Ruby or its gems locally. Wire it into a workflow to crawl and crack from one place.

Can CeWL crawl pages behind a login?

Yes. Set --auth_user, --auth_pass, and --auth_type (basic or digest) so it reaches words on authenticated pages, where the richer internal vocabulary usually lives. Route through an intercepting proxy with --proxy_host and --proxy_port if needed.

How do I keep the wordlist focused and relevant?

Bound the crawl with -d and stay on-site by leaving --offsite off, then raise --min_word_length to drop short, low-value tokens. Add --with-numbers when the target mixes digits into terms, and --count to rank words by how often they appear.

Does CeWL show how often each word appears?

With --count, every word is printed alongside the number of times CeWL saw it, so you can sort by frequency and prioritize the most common terms first. Without it, the output is a plain list of unique words.

How is the wordlist saved, and what runs after CeWL?

On the command line CeWL prints to stdout or writes to a file with -w; in a Trickest workflow the node saves the wordlist to a file and a folder automatically. A password cracker or a fuzzer usually runs next, consuming that target-specific list instead of a generic dictionary.

Updated Jul 14, 2026

Utilities

Generate a custom wordlist by spidering a target site

A Ruby app that spiders a URL and returns a wordlist for password crackers.

Build a target-specific password wordlist Harvest emails and metadata while crawling Crawl authenticated pages

overview

What cewl does

CeWL spiders a given URL to a set depth and collects the words it finds on the pages, then writes them out as a wordlist. Instead of a generic dictionary, you get terms drawn from the target itself: product names, internal jargon, employee names, and phrases that people at that organization reuse in passwords. That makes the list far more relevant to one engagement than an off-the-shelf rockyou.

You control how the spider behaves and what it keeps. Set crawl depth with -d, follow external links or stay on-site, require a minimum word length, and optionally pull email addresses and document metadata while it runs. Authentication and proxy options let it crawl pages behind a login or through an intercepting proxy.

In a Trickest workflow, CeWL takes a URL and writes a file plus a folder of results. Run it early to seed a target-specific wordlist, then feed that list into a password-cracking or fuzzing stage so the rest of the workflow tests credentials and paths that match the target's own vocabulary.

source github.com/digininja/CeWL

use cases

Where cewl fits

Build a target-specific password wordlist

Spider a company site to harvest its own terminology, then feed the list to a cracker so guesses match the words employees reuse in passwords.

Harvest emails and metadata while crawling

Enable --email and --meta to pull addresses and document author data alongside the wordlist, useful for OSINT and username generation.

Crawl authenticated pages

Supply basic or digest credentials so CeWL reaches words on pages behind a login, where the richer internal vocabulary tends to live.

Seed a fuzzing stage with real words

Pass the generated wordlist into a content or parameter fuzzer so brute-force guesses draw on terms the target itself exposes.

reference

cewl inputs and flags

25 inputs

Name	Type	Flag	Description
url	STRING	·	The site to spider (passed as the positional target URL).
depth	STRING	-d	Depth to spider to (default 2).
min-word-length	STRING	--min_word_length	Minimum word length to keep (default 3).
count	BOOLEAN	--count	Show the count for each word found.
with-numbers	BOOLEAN	--with-numbers	Accept words containing numbers, not only letters.
email	BOOLEAN	--email	Include email addresses found while crawling.
meta	BOOLEAN	--meta	Include document metadata in the results.
offsite	BOOLEAN	--offsite	Let the spider visit sites beyond the target.

Showing key inputs. cewl exposes 25 inputs in total.

Full flag reference (25 inputs)

Name	Type	Flag	Description
url	STRING	·	The site to spider (passed as the positional target URL).
depth	STRING	-d	Depth to spider to (default 2).
min-word-length	STRING	--min_word_length	Minimum word length to keep (default 3).
max-word-length	STRING	--max_word_length	Maximum word length (default unset).
count	BOOLEAN	--count	Show the count for each word found.
with-numbers	BOOLEAN	--with-numbers	Accept words containing numbers, not only letters.
lowercase	BOOLEAN	--lowercase	Lowercase all parsed words.
groups	BOOLEAN	--groups	Return groups of words as well as single words.
email	BOOLEAN	--email	Include email addresses found while crawling.
meta	BOOLEAN	--meta	Include document metadata in the results.
offsite	BOOLEAN	--offsite	Let the spider visit sites beyond the target.
allowed	STRING	--allowed	Regex pattern a path must match to be followed.
exclude	FILE	--exclude	File listing paths to exclude from the crawl.
header	STRING	--header	Extra header in name:value form; can be passed multiple times.
user-agent	STRING	--ua	User agent string to send with requests.
auth-type	STRING	--auth_type	Authentication type: digest or basic.
auth-user	STRING	--auth_user	Authentication username for pages behind a login.
auth-pass	STRING	--auth_pass	Authentication password.
proxy-host	STRING	--proxy_host	Proxy host to route requests through.
proxy-port	STRING	--proxy_port	Proxy port (default 8080).
proxy-username	STRING	--proxy_username	Username for the proxy, if required.
proxy-password	STRING	--proxy_password	Password for the proxy, if required.
convert-umlauts	BOOLEAN	--convert-umlauts	Convert common Latin-1 umlauts (ä to ae, ö to oe, ü to ue, ß to ss).
verbose	BOOLEAN	--verbose	Verbose output.
debug	BOOLEAN	--debug	Print extra debug information.

example

Run cewl

cewl · command

# spider example.com to depth 2, keep words 5+ chars, show counts, allow digitscewl -d 2 --min_word_length 5 --count --with-numbers https://example.com > wordlist.txt

sample output

example, 318security, 205platform, 154research, 132engineering, 118automation, 96credentials, 74workflow, 61onboarding, 47

guidance

Choosing cewl

Reach for CeWL when a target's own vocabulary will beat a generic dictionary, such as cracking credentials at one organization. For a static, reusable wordlist there is nothing to crawl. For pulling links and endpoints rather than raw words, use a crawler like katana instead.

katana

Crawls for endpoints and links, not words. Use it to map an app, CeWL to harvest its vocabulary.

gospider

Fast web spider for URLs. CeWL is purpose-built to emit a wordlist for cracking.

clean-wordlist

Cleans and normalizes a list. Run it after CeWL to tidy the harvested words.

faq

cewl questions

CeWL spiders a target site and returns a wordlist built from the words on its pages. Pentesters use that list for password cracking and fuzzing because it captures the organization's own vocabulary, which people tend to reuse in credentials, better than a generic dictionary does.

Run cewl yourself

A target URL feeds CeWL, which spiders the site, collects its words, and writes a target-specific wordlist ready for cracking or fuzzing.

Run cewl in Trickest View the workflow

Facts on this page come from the live Trickest tool library.

Name

Type

Flag

Description

url

STRING

The site to spider (passed as the positional target URL).

depth

STRING

-d

Depth to spider to (default 2).

min-word-length

STRING

--min_word_length

Minimum word length to keep (default 3).

count

BOOLEAN

--count

Show the count for each word found.

with-numbers

BOOLEAN

--with-numbers

Accept words containing numbers, not only letters.

BOOLEAN

--email

Include email addresses found while crawling.

Generate a custom wordlist by spidering a target site

What cewl does

Where cewl fits

Build a target-specific password wordlist

Harvest emails and metadata while crawling

Crawl authenticated pages

Seed a fuzzing stage with real words

cewl inputs and flags

Run cewl

Choosing cewl

katana

gospider

clean-wordlist

cewl questions

More Utilities tools

airtable-integration

apktool-decode

assert-tool

aws-route53-zones

batch-output

batch-output-v2

Run cewl yourself

Generate a custom wordlist by spidering a target site

What cewl does

Where cewl fits

Build a target-specific password wordlist

Harvest emails and metadata while crawling

Crawl authenticated pages

Seed a fuzzing stage with real words

cewl inputs and flags

Run cewl

Choosing cewl

katana

gospider

clean-wordlist

cewl questions

More Utilities tools

airtable-integration

apktool-decode

assert-tool

aws-route53-zones

batch-output

batch-output-v2

Run cewl yourself