loading
Loading content
loading
Discovery
Spider pages and scrape linked cloud resource strings.
overview
cloudscraper crawls from -u (or -l) and extracts cloud resource strings already present in HTML and scripts: S3 buckets, Azure blobs, GCS objects, DigitalOcean spaces. It reports what the app links or embeds, not invented names.
Depth (-d) caps how far links are followed; -p sets parallel processes. Prefer this when you care what the target already references. For keyword brute force of public names, use cloud-enum instead.
Trickest runs it as a managed Discovery node: URL or target list in, file and folder of findings out. Pipe bucket names into s3scanner when you need exposure checks.
use cases
Crawl a product site with -u so embedded S3 or Azure URLs surface without guessing names.
Pass scraped bucket names into s3scanner or cloud permission checkers to see which assets are public.
Use -l with a line-delimited URL list and raise -p when many apps need the same cloud scrape.
reference
| Name | Type | Flag | Description |
|---|---|---|---|
| url | STRING | -u | Target scope URL to spider. |
| depth | STRING | -d | Max link depth to crawl (default 5). |
| target-list | FILE | -l | File of line-delimited target URLs. |
| processes | STRING | -p | Number of parallel processes (default 2). |
| verbose | BOOLEAN | -v | Enable verbose output. |
| no-verify | BOOLEAN | --no-verify | Skip TLS verification. |
Showing key inputs. cloudscraper exposes 6 inputs in total.
example
# spider example.com two levels deep for cloud resource stringscloudscraper -u https://example.com -d 2 -p 2 -v[+] crawling https://example.com[+] found s3: example-static.s3.amazonaws.com[+] found s3: example-uploads.s3.amazonaws.com[+] found azure: examplemedia.blob.core.windows.net[+] found gcs: example-cdn.storage.googleapis.com[+] crawling https://www.example.com/docs[+] found s3: example-docs-public.s3.amazonaws.com[+] done: 7 cloud referencesguidance
Use cloudscraper when you want the cloud resources a site already references, not guessed names. Pair it with the app under test. For keyword brute force of public names, use cloud-enum. To list assets in accounts you own, use cloudlist.
Brute-forces public cloud names from a keyword. cloudscraper scrapes references from the target's own pages.
Tests S3 buckets for exposure. Feed it the references cloudscraper finds.
General-purpose crawler. cloudscraper focuses on extracting cloud resource strings while it spiders.
faq
related
Extract URLs and endpoints from Android APK files.
Web path scanner.
Crawl pages, harvest potential parameters, write a custom wordlist.
Recursive content discovery with smart defaults and rich response filters.
Maintained gau fork for passive archive URL collection.
Extract JavaScript file URLs from a page or URL list.
A target URL feeds CloudScraper, which scrapes cloud references from its pages and passes them to s3scanner so exposed buckets land as output.
Facts on this page come from the live Trickest tool library.