当前环境是 Win8 64位的,使用的Python 3.5 版本。
一 安装Scrapy
1,安装 lxml
pip install lxml -i https://pypi.douban.com/simple
2, 安装 Scrapy
pip install Scrapy -i https://pypi.douban.com/simple
在windows下安装有可能失败,报以下异常。
building 'twisted.test.raiser' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
解决方案:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted 下载twisted对应版本的whl文件(如我的Twisted-17.5.0-cp35-cp35m-win_amd64.whl),cp后面是Python版本,amd64代表64位,运行命令:
pip install Twisted-17.5.0-cp35-cp35m-win_amd64.whl
安装完成后,再次运行 pip install Scrapy -i https://pypi.douban.com/simple , 就可以安装成功 Scrapy.
3, 验证安装
在DOS 状态下输入 Scrapy命令,如果提示如下命令,就证明安装成功啦,如果失败了,请检查上述步骤有何疏漏。
C:\Users\si\Desktop\333>Scrapy
Scrapy 1.4.0 - no active project
Usage:
scrapy <command> [options] [args]
Available commands:
bench Run quick benchmark test
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy
[ more ] More commands available when run from project directory
Use "scrapy <command> -h" to see more info about a command
参考资料:
https://doc.scrapy.org/en/latest/intro/tutorial.html
一个知识点,你自己看懂了,那是一个层次;你会用,是另外一个层次;你写出来,你写出来让别人懂,那又是更高的一个层次;你用最通俗的言语,把知识点讲出来,让别人一下子明白,又是更高的一个层次。