我在deepin上安装fake-useragent后,实例化ua = UserAgent()时出现错误

来源:8-5 通过downloadmiddleware随机更换user-agent - 2

慕课网就是好

2018-05-03

(article_spider) mly@mly-PC:~/PycharmProjects/ArticleSpider/ArticleSpider$ python

Python 3.5.4rc1 (default, Jul 25 2017, 08:53:34) 

[GCC 6.4.0 20170704] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> from fake_useragent import UserAgent

>>> ua = UserAgent()

Error occurred during loading data. Trying to use cache server http://d2g6u4gh6d9rq0.cloudfront.net/browsers/fake_useragent_0.1.10.json

Traceback (most recent call last):

  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open

    h.request(req.get_method(), req.selector, req.data, headers)

  File "/usr/lib/python3.5/http/client.py", line 1107, in request

    self._send_request(method, url, body, headers)

  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request

    self.endheaders(body)

  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders

    self._send_output(message_body)

  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output

    self.send(msg)

  File "/usr/lib/python3.5/http/client.py", line 877, in send

    self.connect()

  File "/usr/lib/python3.5/http/client.py", line 1253, in connect

    super().connect()

  File "/usr/lib/python3.5/http/client.py", line 849, in connect

    (self.host,self.port), self.timeout, self.source_address)

  File "/usr/lib/python3.5/socket.py", line 712, in create_connection

    raise err

  File "/usr/lib/python3.5/socket.py", line 703, in create_connection

    sock.connect(sa)

socket.timeout: timed out


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/home/mly/.virtualenvs/article_spider/lib/python3.5/site-packages/fake_useragent/utils.py", line 67, in get

    context=context,

  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen

    return opener.open(url, data, timeout)

  File "/usr/lib/python3.5/urllib/request.py", line 466, in open

    response = self._open(req, data)

  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open

    '_open', req)

  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain

    result = func(*args)

  File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open

    context=self._context, check_hostname=self._check_hostname)

  File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open

    raise URLError(err)

urllib.error.URLError: <urlopen error timed out>


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/home/mly/.virtualenvs/article_spider/lib/python3.5/site-packages/fake_useragent/utils.py", line 154, in load

    for item in get_browsers(verify_ssl=verify_ssl):

  File "/home/mly/.virtualenvs/article_spider/lib/python3.5/site-packages/fake_useragent/utils.py", line 97, in get_browsers

    html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)

  File "/home/mly/.virtualenvs/article_spider/lib/python3.5/site-packages/fake_useragent/utils.py", line 84, in get

    raise FakeUserAgentError('Maximum amount of retries reached')

fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

Traceback (most recent call last):

  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open

    h.request(req.get_method(), req.selector, req.data, headers)

  File "/usr/lib/python3.5/http/client.py", line 1107, in request

    self._send_request(method, url, body, headers)

  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request

    self.endheaders(body)

  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders

    self._send_output(message_body)

  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output

    self.send(msg)

  File "/usr/lib/python3.5/http/client.py", line 877, in send

    self.connect()

  File "/usr/lib/python3.5/http/client.py", line 1253, in connect

    super().connect()

  File "/usr/lib/python3.5/http/client.py", line 849, in connect

    (self.host,self.port), self.timeout, self.source_address)

  File "/usr/lib/python3.5/socket.py", line 712, in create_connection

    raise err

  File "/usr/lib/python3.5/socket.py", line 703, in create_connection

    sock.connect(sa)

socket.timeout: timed out


写回答

2回答

bobby

2018-05-05

这个是去下载url的时候出错了 http://d2g6u4gh6d9rq0.cloudfront.net/browsers/fake_useragent_0.1.10.json 你试试用浏览器去请求这个看看能不能获取到 

0
2
慕课网就是好
好的,谢谢老师。
2018-05-05
共2条回复

廖小胖

2019-06-19

http://d2g6u4gh6d9rq0.cloudfront.net/browsers/fake_useragent_0.1.10.json这个地址访问不了怎么解决

0
1
bobby
https://github.com/find456789/mock-useragent 可以了解一下这个项目替换课程中fake-useragent,这个项目讲解里面也提到了 fake-useragent容易出现的问题
2019-06-20
共1条回复

Scrapy打造搜索引擎 畅销4年的Python分布式爬虫课

带你彻底掌握Scrapy,用Django+Elasticsearch搭建搜索引擎

5795 学习 · 6290 问题

查看课程