使用JobboleSpider' object has no attribute 'browser'
来源:9-4 selenium集成到scrapy中
精慕门9737150
2017-07-07
使用'ArticleSpider.middlewares.JSPageMiddleware':
产生问题object has no attribute 'browser'
2017-07-07 05:08:50 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2017-07-07 05:08:50 [scrapy.core.scraper] ERROR: Error downloading <GET http://blog.jobbole.com/all-posts/>
Traceback (most recent call last):
File "C:\Users\sarah\Envs\artical_spider\lib\site-packages\twisted\internet\defer.py", line 1301, in _inlineCallbacks
result = g.send(result)
File "C:\Users\sarah\Envs\artical_spider\lib\site-packages\scrapy\core\downloader\middleware.py", line 37, in process_request
response = yield method(request=request, spider=spider)
File "C:/Users/sarah/Desktop/LearnPy/s0vkaq/ArticleSpider\ArticleSpider\middlewares.py", line 101, in process_request
return HtmlResponse(url=spider.browser.current_url, body=spider.browser.page_source, encoding="utf-8", request=request)
AttributeError: 'JobboleSpider' object has no attribute 'browser'
2017-07-07 05:08:50 [scrapy.core.engine] INFO: Closing spider (finished)
前面的DEBUG是这样的(就是可以进入浏览器可以进入这个伯乐在线主页,但后续不能爬取)
1回答
-
bobby
2017-07-10
是否没有在spider中初始化 browser对象 所以在后面访问的时候没有该对象
00