Content Discovery

Content discovery uncovers the parts of a web application that no link points to. A site exposes far more than its navigation reveals: backup files, admin panels, old API versions, config files, and staging endpoints all sit at predictable paths the public never sees. Content discovery requests a long list of candidate paths and records which ones return a meaningful response, building a map of what actually exists on the server.

The core technique is directory brute-forcing, which sends requests for each entry in a wordlist and filters responses by status code, length, and word count to separate real hits from generic 404 pages. It complements web crawling, which follows links the application does present, since the two methods cover different blind spots. Content discovery overlaps with fuzzing when you brute-force parameters and extensions rather than paths.

Discovery matters because hidden endpoints are where the interesting bugs live. An exposed .git directory, a forgotten phpinfo page, or an unauthenticated /admin route often leads straight to compromise.

In an automated workflow, you feed live hosts into a content discovery step, tune the wordlist to the detected tech stack, and pass newly found endpoints into scanning. Recursion drives discovered directories deeper.