​

Enumerate Hostnames via Crawling

​

Description

Enumerate subdomains by crawling web servers and analyzing their HTML content and headers.

​

Features

• Discovers subdomains in HTML attributes, HTTP headers, and JavaScript code that may not be identified through other sources.
• Offers customizable crawling depth to balance speed and coverage.
• Capable of processing tens of thousands of web servers simultaneously.

​

Inputs

​

Required

• web-servers: a list of web servers

https://dashboard.example.com

https://payments.example.com

http://shop.example.com:8080

​

Optional

• depth: maximum crawling depth (default: 2)

​

Outputs

• subdomains: List of discovered subdomains

admin.example.com

assets.example.com

• subdomain-details: JSONLines records of subdomain discovery details.

{"hostname": "admin.example.com", "domain_name": "example.com", "data_source": "crawling", "context": "a href", "linked_asset": "dashboard.example.com"}

{"hostname": "assets.example.com", "domain_name": "example.com", "data_source": "crawling", "context": "script text", "linked_asset": "payments.example.com"}

Note: The subdomain-details outputs may contain duplicates if a hostname was discovered in multiple locations.

​

Changelog

• v1.0
  • Initial release