NAME
scrapy-ctl - Python Scrapy control script
SYNOPSIS
scrapy-ctl [command] [OPTIONS] ...
DESCRIPTION
Scrapy is controlled through the scrapy-ctl control script. The script
provides several commands, for different purposes. Each command
supports its own particular syntax. In other words, each command
supports a different set of arguments and options.
OPTIONS
fetch [OPTION] URL
Fetch a URL using the Scrapy downloader
--headers
Print response HTTP headers instead of body
runspider [OPTION] spiderfile
Run a spider
--output=FILE
Store scraped items to FILE in XML format
settings [OPTION]
Query Scrapy settings
--get=SETTING
Print raw setting value
--getbool=SETTING
Print setting value, intepreted as a boolean
--getint=SETTING
Print setting value, intepreted as an integer
--getfloat=SETTING
Print setting value, intepreted as an float
--getlist=SETTING
Print setting value, intepreted as an float
--init Print initial setting value (before loading extensions and
spiders)
shell URL | file
Launch the interactive scraping console
startproject projectname
Create new project with an initial project template
--help, -h
Print command help and options
--version
Print Scrapy version and exit
--logfile=FILE
Log file. if omitted stderr will be used
--loglevel=LEVEL, -L LEVEL
Log level (default: None)
--nolog
Disable logging completely
--spider=SPIDER
Always use this spider when arguments are urls
--profile=FILE
Write python cProfile stats to FILE
--lsprof=FILE
Write lsprof profiling stats to FILE
--pidfile=FILE
Write process ID to FILE
--set=SET
Set/override setting (may be repeated)
--settings=MODULE
Python path to the Scrapy project settings
AUTHOR
Scrapy-ctl was written by the Scrapy Developers <scrapy-
developers@googlegroups.com>.
This manual page was written by Ignace Mouzannar <mouzannar@gmail.com>,
for the Debian project (but may be used by others).
October 17, 2009