Utilities
Load recon files into Elasticsearch and query them back
Persist attack-surface files in Elasticsearch and query them later.
overview
What elasticsearch-index does
Place elasticsearch-index at the end of a recon workflow so folders of hosts, URLs, and findings land in named Elasticsearch indices.
Reach for it when grepping across run output is too slow and you need a searchable store, or when a later stage should pull prior results with --query.
Index a directory with --dir or a single file with --file. Name the index with --index, or let --auto-index derive it from the filename; map list lines with --field.
use cases
Where elasticsearch-index fits
Make recon output searchable
Index folders of subdomains, hosts, and findings into Elasticsearch so the whole attack surface is queryable instead of scattered across files.
Auto-name indices from file names
Turn on auto-index so subdomains.txt lands in a subdomains index without naming each one by hand across a large workflow.
Query prior results into a new run
Use query mode to read back stored data so a later workflow stage can build on what earlier runs discovered.
Map list files into a named field
Set --field when indexing line-oriented lists so each entry becomes a document field you can filter on later.
reference
elasticsearch-index inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| dir | FOLDER | --dir | Directory with files to be indexed. |
| file | FILE | --file | Single file to be indexed. |
| index | STRING | --index | Name of the Elasticsearch index. |
| auto-index | BOOLEAN | --auto-index | Derive the index name automatically from the file name. |
| field | STRING | --field | Field name to use with list files. |
| query | STRING | --query | Query to search the index for. |
| config | FILE | --config | Path to the config YAML file. |
| file-type | STRING | --file-type | Type of the file to be indexed. |
Showing key inputs. elasticsearch-index exposes 9 inputs in total.
Full flag reference (9 inputs)
| Name | Type | Flag | Description |
|---|---|---|---|
| dir | FOLDER | --dir | Directory with files to be indexed |
| file | FILE | --file | File to be indexed |
| field | STRING | --field | Field name to use with "list" files |
| index | STRING | --index | Name of the Elasticsearch index |
| query | STRING | --query | Query to search for |
| config | FILE | --config | Path to the config YAML file |
| file-type | STRING | --file-type | Type of the file to be indexed |
| auto-index | BOOLEAN | --auto-index | Automatically determine the index name based on the file name (e.g. subdomains.txt -> subdomains |
| elastic-id | BOOLEAN | --elastic-id | Use Elasticsearch's automatically-generated IDs |
example
Run elasticsearch-index
# index a recon folder into Elasticsearch; auto-name indices from filenameselasticsearch-index --dir recon-out/ --auto-index --field host --config es-config.yaml[*] elasticsearch-index[*] Config: es-config.yaml[*] Indexing dir recon-out/[*] File subdomains.txt -> index subdomains (field=host) 128 docs[*] File live-hosts.txt -> index live-hosts (field=host) 41 docs[*] File urls.txt -> index urls (field=host) 87 docs[*] Indexed 256 documents across 3 indices[*] Sample: api.example.com, app.example.com, www.example.com[*] Doneguidance
Choosing elasticsearch-index
Use elasticsearch-index to persist and search attack-surface data at the end of a workflow, or to read prior results back with --query. For lightweight stream-side deduplication rather than a searchable store, use anew.
anew
Appends only new lines to a flat file for cheap diffing. elasticsearch-index gives you a full searchable store instead.
airtable-integration
Pushes results into Airtable for a spreadsheet-style view. Sibling storage backend, different query model.
faq
elasticsearch-index questions
related
More Utilities tools
cewl
Spider a URL and return a wordlist for password crackers.
dnsgen
Wordlist and mined-word subdomain permutation.
pup
CSS selectors over HTML, the jq counterpart for markup.
unfurl
Extract chosen URL parts from stdin into clean line lists.
whisper
Speech to txt, vtt, srt, and json from an audio file or folder.
youtube-transcript
Public caption tracks to transcript.txt and results.jsonl.
Run elasticsearch-index yourself
Recon output feeds elasticsearch_index, which loads the files into Elasticsearch and writes a searchable index downstream stages can query.
Facts on this page come from the live Trickest tool library.