拉勾网刚开始就302

来源:7-4 Rule和LinkExtractor使用

三肥牛元气

2018-11-05

(article_spider) λ scrapy shell https://www.lagou.com/jobs/2065398.html

然后
DEBUG: Crawled (200) <GET https://www.lagou.com/utrack/trackMid.html?f=https%3A%2F%2Fpassport.lagou.com%2Flogin%2Flogin.html%3Fmsg%3Dvalidation%26uStatus%3D2%26clientIp%3D61.241.194.191&t=1541400661&_ti=1> (referer: None)
[s] Available Scrapy objects:
[s] scrapy scrapy module (contains scrapy.Request, scrapy.Selector, etc)
[s] crawler <scrapy.crawler.Crawler object at 0x00000222DA27B470>
[s] item {}
[s] request <GET https://www.lagou.com/jobs/2065398.html>
[s] response <200 https://www.lagou.com/utrack/trackMid.html?f=https%3A%2F%2Fpassport.lagou.com%2Flogin%2Flogin.html%3Fmsg%3Dvalidation%26uStatus%3D2%26clientIp%3D61.241.194.191&t=1541400661&_ti=1>
[s] settings <scrapy.settings.Settings object at 0x00000222DBB6B9B0>
[s] spider <LagouSpider ‘lagou’ at 0x222dc32ca58>
[s] Useful shortcuts:
[s] fetch(url[, redirect=True]) Fetch URL and update local objects (by default, redirects are followed)
[s] fetch(req) Fetch a scrapy.Request and update local objects
[s] shelp() Shell help (print this help)
[s] view(response) View response in a browser
就跳转到这登陆界面。
看了他们的也没找出答案。。。。后面的视频虽然看了,不过做不下去了。。

写回答

2回答

qq_下弦月_1

2021-03-04

我的也是,刚开始就是302

0
1
bobby
scrapy shell需要添加user-agent否则会被直接识别出来
2021-03-05
共1条回复

bobby

2018-11-08

你加我qq 442421039 我看看

0
4
Young___
回复
bobby
老师 我qq好友请求您一直不通过呢 547008925 麻烦您通过一下...............
2019-04-02
共4条回复

Scrapy打造搜索引擎 畅销4年的Python分布式爬虫课

带你彻底掌握Scrapy,用Django+Elasticsearch搭建搜索引擎

5796 学习 · 6290 问题

查看课程