loading
Loading content
loading
Utilities
Output file lines by batch size, given START_LINE and END_LINE.
overview
batch-output takes a file and returns only the lines in a range you give as START_LINE,END_LINE. That small cut is how a workflow turns a million-line target list into even pieces and runs a heavy tool across many of them at once instead of in one long pass.
Pair each range with a parallel copy of the downstream node: one worker gets 1,5000, another 5001,10000, and so on. The same range logic applies to a folder of files when you need to process inputs in groups rather than the entire set.
Place the Utilities node between a large producer and a slow consumer. Prefer batch-output-v2 for new graphs when you want explicit lines or files modes and built-in parallel batch handling on top of the same idea.
source github.com/trickest
use cases
Carve a million-line target file into fixed ranges and feed each range to its own copy of a scanner so the work runs in parallel.
Pass one batch at a time into an expensive node to keep a single run inside time or resource limits instead of processing everything at once.
Pick up where a previous run stopped by selecting the next START_LINE,END_LINE range rather than reprocessing the whole list.
Apply the same range logic to a folder so files are handled in batches when one node should not take the entire set.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| file | FILE | · | File from which to extract batches. |
| batch | STRING | · | START_LINE,END_LINE range to extract from the file or folder input. |
| folder | FOLDER | · | Folder with files from which to extract batches. |
Showing key inputs. batch-output exposes 3 inputs in total.
example
# extract lines 1-5000 from a large target list for one parallel workerbatch-output targets.txt 1,5000 > batch-01.txtapi.example.comapp.example.comstaging.example.comdev.example.commail.example.comvpn.example.com198.51.100.10198.51.100.24203.0.113.5203.0.113.88guidance
Use batch-output to split a large file or folder into ranges so heavy work fans out across parallel nodes. It slices input, it runs no tool, so place it ahead of the consumer you want to parallelize. For a newer engine with explicit modes, see batch-output-v2.
The newer rewrite with explicit lines and files modes plus parallel batch processing. Prefer it for new workflows.
Removes duplicate lines from large lists. Different job, shrinking input rather than slicing it.
Transforms each line rather than selecting a range. A sibling list-manipulation utility.
faq
related
Import, export, and link workflow data with Airtable.
Decode Android APK files into smali sources and resources.
Check a file's values against conditions and exit with a matching code.
Extract all hosted zones from AWS Route53.
Extract a batch range from a file's lines or a folder's files, with parallel processing.
Organize all the community Nuclei templates from across the ecosystem in one place.
A large target list feeds batch-output, which slices it into a line range and passes that batch to httpx so probing runs on a manageable chunk.
Facts on this page come from the live Trickest tool library.