Name:crawlergo
Category:Discovery
Publisher:trickest-mhmdiaa
Created:12/26/2022
Container:quay.io/trickest/crawlergo:v0.4.4-patch-2
Output Type:
License:Unknown

Parameters

​
url
string
required
URLs to scan, separated by spaces (must be same host)
​
fuzz-path
boolean
--fuzz-pathwhether to fuzz the target with common paths. (default: false)
​
log-level
string
--log-levellog print Level, options include debug, info, warn, error and fatal. (default: Info)
​
post-data
string
--post-dataset PostData to target and use POST method.
​
encode-url
boolean
--encode-urlwhether to encode url with detected charset. (default: false)
​
filter-mode
string
--filter-modefiltering Mode used for collected requests. Allowed mode:simple, smart or strict. (default: smart)
​
form-values
string
--form-valuescustom filling text for each form type. e.g.: -fv username=crawlergo_nice -fv password=admin123
​
output-mode
string
--output-modeconsole print or serialize output. Allowed mode:console ,json or none. (default: console)
​
robots-path
boolean
--robots-pathwhether to resolve paths from /robots.txt. (default: false)
​
max-tab-count
string
--max-tab-countmaximum Number of tabs allowed. (default: 8)
​
push-pool-max
string
--push-pool-maxmaximum Number of concurrency when pushing results to proxy. (default: 10)
​
push-to-proxy
string
--push-to-proxyevery request in 'req_list' will be pushed to the proxy Address
​
request-proxy
string
--request-proxyall requests connect through defined proxy server.
​
custom-headers
string
--custom-headersadd additional Headers to each request. The input string will be called json.Unmarshal (default: {Spider-Name: crawlergo, User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36})
​
fuzz-path-dict
file
--fuzz-path-dictFuzz dict
​
tab-run-timeout
string
--tab-run-timeoutthe Timeout of a single tab task. (default: 20s)
​
before-exit-delay
string
--before-exit-delaythe Time of waiting before crawler exit. (default: 1s)
​
max-crawled-count
string
--max-crawled-countthe maximum Number of URLs visited by the crawler in this task. (default: 200)
​
event-trigger-mode
string
--event-trigger-modethis Value determines how the crawler automatically triggers events.Allowed mode:async or sync. (default: async)
​
form-keyword-values
string
--form-keyword-valuescustom filling text, fuzzy matched by keyword. e.g.: -fkv user=crawlergo_nice -fkv pass=admin123
​
ignore-url-keywords
string
--ignore-url-keywordscrawlergo will not crawl these URLs matched by Keywords. Default [logout quit exit])
​
event-trigger-interval
string
--event-trigger-intervalthe Interval of triggering each event. (default: 100ms)
​
wait-dom-content-loaded-timeout
string
--wait-dom-content-loaded-timeoutthe Timeout of waiting for a page dom ready. (default: 5s)