Discovery
Extract linked S3 and Azure strings from pages
Spider pages and scrape linked cloud resource strings.
overview
What cloudscraper does
cloudscraper crawls from -u (or -l) and extracts cloud resource strings already present in HTML and scripts: S3 buckets, Azure blobs, GCS objects, DigitalOcean spaces. It reports what the app links or embeds, not invented names.
Depth (-d) caps how far links are followed; -p sets parallel processes. Prefer this when you care what the target already references. For keyword brute force of public names, use cloud-enum instead.
Trickest runs it as a managed Discovery node: URL or target list in, file and folder of findings out. Pipe bucket names into s3scanner when you need exposure checks.
use cases
Where cloudscraper fits
Find cloud buckets linked from an app
Crawl a product site with -u so embedded S3 or Azure URLs surface without guessing names.
Seed exposure checks
Pass scraped bucket names into s3scanner or cloud permission checkers to see which assets are public.
Bulk-scrape several targets
Use -l with a line-delimited URL list and raise -p when many apps need the same cloud scrape.
reference
cloudscraper inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| url | STRING | -u | Target scope URL to spider. |
| depth | STRING | -d | Max link depth to crawl (default 5). |
| target-list | FILE | -l | File of line-delimited target URLs. |
| processes | STRING | -p | Number of parallel processes (default 2). |
| verbose | BOOLEAN | -v | Enable verbose output. |
| no-verify | BOOLEAN | --no-verify | Skip TLS verification. |
Showing key inputs. cloudscraper exposes 6 inputs in total.
example
Run cloudscraper
# spider example.com two levels deep for cloud resource stringscloudscraper -u https://example.com -d 2 -p 2 -v[+] crawling https://example.com[+] found s3: example-static.s3.amazonaws.com[+] found s3: example-uploads.s3.amazonaws.com[+] found azure: examplemedia.blob.core.windows.net[+] found gcs: example-cdn.storage.googleapis.com[+] crawling https://www.example.com/docs[+] found s3: example-docs-public.s3.amazonaws.com[+] done: 7 cloud referencesguidance
Choosing cloudscraper
Use cloudscraper when you want the cloud resources a site already references, not guessed names. Pair it with the app under test. For keyword brute force of public names, use cloud-enum. To list assets in accounts you own, use cloudlist.
cloud-enum
Brute-forces public cloud names from a keyword. cloudscraper scrapes references from the target's own pages.
s3scanner
Tests S3 buckets for exposure. Feed it the references cloudscraper finds.
katana
General-purpose crawler. cloudscraper focuses on extracting cloud resource strings while it spiders.
faq
cloudscraper questions
related
More Discovery tools
apkurlgrep
Extract URLs and endpoints from Android APK files.
cariddi
Crawl a domain list and scan responses for endpoints, secrets, tokens, and juicy files.
crawlergo
Browser-driven crawler that harvests requests for downstream scanners.
dirsearch
Web path scanner.
fallparams
Crawl pages, harvest potential parameters, write a custom wordlist.
feroxbuster
Recursive content discovery with smart defaults and rich response filters.
Run cloudscraper yourself
A target URL feeds CloudScraper, which scrapes cloud references from its pages and passes them to s3scanner so exposed buckets land as output.
Facts on this page come from the live Trickest tool library.