zeiver 

URLS


# Download

CommandInputDescription
Page DepthSpecify the maximum depth for recursive scraping. Default: 20. More Info
Accept FilesUsing [Regex], specify which files to accept for scraping. This option takes precedence over reject option. More Info
Reject FilesUsing [Regex], specify which files to reject for scraping. The accept option takes precedence over this option. More Info

# Recorder

CommandInputDescription
Activate RecorderActivates the Recorder. Save the links to each file. More Info
Record Links OnlyActivates the Recorder. After scraping, instead of downloading the files, save the links to them. More Info
Rename Record FileChanges the name of the record file. This file is where the recorder will store the links. Default: URL_Records.txt. More Info
No Stat File(s)Prevents the Recorder from creating stat files. These files keep a record for the total amount of files downloaded along with other info. More Info

# File/Directory

CommandInputDescription
Input FileRead URLs from a local or external file. More Info
Save LocationSave file location. Default: ./. More Info
Cut Directory(ies)Ignores a specific number of remote directories from being created. Default: 0. More Info
No DirectoriesDo not create a hierarchy of remote directories. More Info

# HTTP

CommandInputDescription
Set HeadersThe default headers to use for every request. *Use the header$value format. Each header must also be separated by a comma. More Info
User-AgentThe User Agent header to use. Default: Zeiver/VERSION. More Info
RetriesRetry amount for a failed connection/request. Default: 20. More Info
Wait DelayWait a specified number of seconds between each scraping & download request. More Info
Retry DelayThe wait time between each failed request. Default: 10. More Info
Random Wait DelayWait a random amount of seconds between each request. The time between requests will vary between 0.5 * Wait Delay (inclusive) to 1.5 * Wait Delay (exclusive). More Info
TimeoutAdds a request timeout for a specified number of seconds. More Info
RedirectsMaximum redirects to follow. Default: 10. More Info
ProxyThe proxy to use. More Info
Proxy AuthThe basic authentication needed to use the proxy. *Must use the username:password format. More Info