loading
Loading content
loading
Utilities
Extract a batch range from a file's lines or a folder's files, with parallel processing.
overview
batch-output-v2 splits large inputs into ranges so parallel workers can share the load. In lines mode it extracts a START,END span of lines from a file; in files mode it extracts a span from a folder's sorted file list. When --mode is omitted it picks the mode that matches the input you supplied.
The goal is fan-out. A huge target list or a folder of artifacts becomes independent batches that several copies of a downstream scanner or prober can process at once instead of waiting on one serial pass. Pass --batch as START,END, for example 1,10 for the first ten items.
In a Trickest workflow the node takes a file or folder plus a batch range and writes a file and a folder holding the selected slice. Place it between a large producer and an expensive consumer to control how the workload fans out across parallel nodes.
use cases
Split a large file into START,END line ranges and feed each to its own copy of a scanner so the run finishes faster than one serial pass.
Use files mode to extract a range from a sorted folder so a downstream node processes inputs in groups instead of all at once.
Hand it a file or a folder and rely on auto-detection to choose lines or files mode when inputs vary across runs.
Process one batch at a time through an expensive node to stay within time and resource budgets on very large datasets.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| file | FILE | --file | Input file for lines mode (extract line range from this file). |
| folder | FOLDER | --folder | Input folder for files mode (extract file range from sorted file list). |
| batch | STRING | --batch | Batch range as START,END (e.g., "1,10" extracts items 1 through 10). |
| mode | STRING | --mode | Processing mode: 'lines' (extract lines from file) or 'files' (extract files from folder). Auto-detects if not specified. |
Showing key inputs. batch-output-v2 exposes 4 inputs in total.
example
# batch-output-v2: extract lines 1-100 from a host list for a parallel workerbatch-output-v2 --mode lines --file hosts.txt --batch 1,100api.example.comapp.example.comstaging.example.comdev.example.commail.example.comvpn.example.comcdn.example.comwww.example.comguidance
Use batch-output-v2 to split a file or folder into ranges for parallel processing, the newer default over batch-output. It slices input, it runs no tool, so place it ahead of the consumer you want to fan out.
The original line-range slicer. v2 adds explicit modes, folder support, and parallel processing.
Deduplicates large lists. Shrinks the input rather than slicing it into batches.
Appends only new lines. A different list utility, for tracking novelty across runs.
faq
related
Import, export, and link workflow data with Airtable.
Decode Android APK files into smali sources and resources.
Check a file's values against conditions and exit with a matching code.
Extract all hosted zones from AWS Route53.
Output file lines by batch size, given START_LINE and END_LINE.
Organize all the community Nuclei templates from across the ecosystem in one place.
A large target list feeds batch-output-v2, which extracts a START,END batch and passes it to nuclei so scanning runs on one parallelizable chunk.
Facts on this page come from the live Trickest tool library.