使用ItemLoader后下载图片报错
来源:4-23 itemloader提取信息
麦兜兜里豆不逗
2024-07-02
数据可以正常写入数据库,图片下载报错
报错代码:
2024-07-01 21:55:04 [scrapy.pipelines.media] ERROR: [Failure instance: Traceback: <class 'scrapy.pipelines.files.FileException'>:
E:\Envs\article_spider\Lib\site-packages\twisted\internet\defer.py:536:addCallbacks
E:\Envs\article_spider\Lib\site-packages\twisted\internet\defer.py:1078:_runCallbacks
E:\Envs\article_spider\Lib\site-packages\scrapy\pipelines\media.py:197:_check_media_to_download
E:\Envs\article_spider\Lib\site-packages\twisted\internet\defer.py:536:addCallbacks
--- <exception caught here> ---
E:\Envs\article_spider\Lib\site-packages\twisted\internet\defer.py:1078:_runCallbacks
E:\Envs\article_spider\Lib\site-packages\scrapy\pipelines\files.py:459:media_failed
]
NoneType: None
下载图片代码:
class ArticleImagePipeline(ImagesPipeline):
def file_path(self, request, response=None, info=None, *, item=None):
name = request.url.split('/')[-1]
return name
def get_media_requests(self, item, info):
print('get_media_requests---')
for img_url in item['front_img_url']:
yield scrapy.Request(url=img_url)
def item_completed(self, results, item, info):
if results:
ok, res = results[0]
if ok:
item['front_img_path'] = res["path"]
return item
写回答
1回答
-
bobby
2024-07-05
还有没有更多的错误栈,这里看不出来具体的错误信息
022024-07-19
相似问题