loading
loading
Utilities
Manage attack surface data on Elasticsearch.
overview
elasticsearch-index takes the files a recon pipeline produces and loads them into Elasticsearch, turning scattered text output into searchable, structured data. Instead of grepping across folders of subdomains, hosts, and findings, you index them once and query the whole attack surface from one place.
It is built for the realities of pipeline output. Index a single file or a whole directory, name the index yourself or let auto-indexing derive it from the file name, choose the file type, and map list files into a named field. A config YAML keeps connection settings out of the command line, and a query mode reads back what you stored.
On Trickest, elasticsearch-index is a Utilities node that takes a folder or a file and writes a folder and a file. Place it at the end of a workflow so each run's results land in a searchable index, or use its query mode to pull prior results back into a later stage.
use cases
Index folders of subdomains, hosts, and findings into Elasticsearch so the whole attack surface is queryable instead of scattered across files.
Turn on auto-index so subdomains.txt lands in a subdomains index without naming each one by hand across a large workflow.
Use query mode to read back stored data so a later workflow stage can build on what earlier runs discovered.
Set --field when indexing line-oriented lists so each entry becomes a document field you can filter on later.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| dir | FOLDER | --dir | Directory with files to be indexed. |
| file | FILE | --file | Single file to be indexed. |
| index | STRING | --index | Name of the Elasticsearch index. |
| auto-index | BOOLEAN | --auto-index | Derive the index name automatically from the file name. |
| field | STRING | --field | Field name to use with list files. |
| query | STRING | --query | Query to search the index for. |
| config | FILE | --config | Path to the config YAML file. |
| file-type | STRING | --file-type | Type of the file to be indexed. |
Showing key inputs. elasticsearch-index exposes 9 inputs in total.
| Name | Type | Flag | Description |
|---|---|---|---|
| dir | FOLDER | --dir | Directory with files to be indexed |
| file | FILE | --file | File to be indexed |
| field | STRING | --field | Field name to use with "list" files |
| index | STRING | --index | Name of the Elasticsearch index |
| query | STRING | --query | Query to search for |
| config | FILE | --config | Path to the config YAML file |
| file-type | STRING | --file-type | Type of the file to be indexed |
| auto-index | BOOLEAN | --auto-index | Automatically determine the index name based on the file name (e.g. subdomains.txt -> subdomains |
| elastic-id | BOOLEAN | --elastic-id | Use Elasticsearch's automatically-generated IDs |
example
# index a recon folder into Elasticsearch; auto-name indices from filenameselasticsearch-index --dir recon-out/ --auto-index --field host --config es-config.yaml[*] elasticsearch-index[*] Config: es-config.yaml[*] Indexing dir recon-out/[*] File subdomains.txt -> index subdomains (field=host) 128 docs[*] File live-hosts.txt -> index live-hosts (field=host) 41 docs[*] File urls.txt -> index urls (field=host) 87 docs[*] Indexed 256 documents across 3 indices[*] Sample: api.example.com, app.example.com, www.example.com[*] Doneguidance
Use elasticsearch_index to persist and search attack-surface data at the end of a workflow, or to read prior results back in with its query mode. For lightweight stream-side deduplication rather than a searchable store, use anew. This is the node that gives recon output a queryable home.
Appends only new lines to a flat file for cheap diffing. elasticsearch_index gives you a full searchable store instead.
Pushes results into Airtable for a spreadsheet-style view. A sibling for a different storage backend.
faq
related
Import, export, and link workflow data with Airtable.
Decode Android APK files into smali sources and resources.
Check a file's values against conditions and exit with a matching code.
Extract all hosted zones from AWS Route53.
Output file lines by batch size, given START_LINE and END_LINE.
Extract a batch range from a file's lines or a folder's files, with parallel processing.
Recon output feeds elasticsearch_index, which loads the files into Elasticsearch and writes a searchable index downstream stages can query.
Facts on this page come from the live Trickest tool library.