Enumerate Hostnames via Crawling
Enumerate subdomains by crawling web servers and analyzing their HTML content and headers
Category
Attack Surface Management
Inputs
depth
string
Maximum crawling depth (Default: 2)
web-servers
file
requiredList of web server URLs
Outputs
subdomainssubdomain-details
Features
Enumerate Hostnames via Crawling
Description
Enumerate subdomains by crawling web servers and analyzing their HTML content and headers.
Features
- Discovers subdomains in HTML attributes, HTTP headers, and JavaScript code that may not be identified through other sources.
- Offers customizable crawling depth to balance speed and coverage.
- Capable of processing tens of thousands of web servers simultaneously.
Inputs
Required
- web-servers: a list of web servers
https://dashboard.example.com
https://payments.example.com
http://shop.example.com:8080
Optional
- depth: maximum crawling depth (default: 2)
Outputs
- subdomains: List of discovered subdomains
admin.example.com
assets.example.com
- subdomain-details: JSONLines records of subdomain discovery details.
{"hostname": "admin.example.com", "domain_name": "example.com", "data_source": "crawling", "context": "a href", "linked_asset": "dashboard.example.com"}
{"hostname": "assets.example.com", "domain_name": "example.com", "data_source": "crawling", "context": "script text", "linked_asset": "payments.example.com"}
Note: The subdomain-details
outputs may contain duplicates if a hostname was discovered in multiple locations.
Changelog
- v1.0
- Initial release