loading
Loading content
loading
Discovery
Append lines to a file only if they are not already there.
overview
anew, from tomnomnom, solves one merge problem: take lines from a fresh file, append only those missing from an existing list, and print the additions. It behaves like a deduplicating tee, so a master list grows without duplicates while stdout carries the change set for the next stage.
That pattern is the backbone of incremental recon. Point a new subdomain, URL, or host dump at a persistent master file and anew both updates the master and surfaces what appeared since the last run. Enable -t to trim leading and trailing whitespace before comparison so padding differences do not register as new findings.
Wire it between producers and consumers in a Trickest Discovery workflow: feed new-file and original-file, then hand the new-only output to probing, alerting, or storage so scheduled runs spend cycles on additions alone.
source github.com/tomnomnom/anew
use cases
Append a stage's results into a master list so the combined set stays unique no matter how many overlapping tools fed into it.
Diff a fresh run against the previous master list so the workflow only sees subdomains, URLs, or hosts that are new this time.
Pipe the new lines anew prints to probing or alerting so scheduled runs stay cheap when most of the surface is unchanged.
Turn on -t so lines that differ only by leading or trailing spaces merge as the same entry instead of bloating the master list.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| new-file | FILE | · | Filename with possible new results to merge in. |
| original-file | FILE | · | Original file that lines are compared against and appended to. |
| trim-whitespaces | BOOLEAN | -t | Trim leading and trailing whitespace before comparison. |
Showing key inputs. anew exposes 3 inputs in total.
example
# merge fresh findings into the master list; -t trims whitespace before comparecat new-results.txt | anew -t master.txtapi.example.comstaging.example.comdev.example.commail.example.comvpn.example.comgitlab.example.com198.51.100.24203.0.113.88guidance
Use anew wherever a pipeline needs deduplication or change detection: merging tool outputs into a unique list, or diffing a scheduled run against the last one so only new lines move downstream. It is glue, so place it between stages rather than at the edges.
Cleans and dedupes a wordlist in place. anew is built for incremental merge and diff across runs.
Removes duplicates from huge wordlists efficiently. Use it for one-shot dedup of very large files.
Diffs two Trickest files directly. A more explicit choice when you only need the change set.
faq
related
Check whether a URL redirects to a masked 404 page.
Extract URLs and endpoints from Android APK files.
Visual inspection of websites across a large number of hosts.
Find suspicious files across a large set of AWS S3 buckets.
An automated tool that checks for backup artifacts that may disclose a web application's source code.
Crawl a list of domains and scan for endpoints, secrets, API keys, file extensions, tokens, and more.
Fresh results and a master list feed anew, which appends only the unseen lines and writes the new-only set as a queryable output.
Facts on this page come from the live Trickest tool library.