网上查找的方法都解决不了这个fake_useragent.errors.FakeUserAgentError

来源:11-3 通过user-agent反爬

两努

2020-06-27

现在报错

fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

网上查来三种解决方法

# ua = UserAgent()
方法一
ua = UserAgent(cache=False)
方法二
ua = UserAgent(verify_ssl=False)
方法三
ua = UserAgent(use_cache_server=False)

# headers = {
#     "User-Agent": ua.random
# }
方法四
headers = {"User-Agent": UserAgent(use_cache_server=False).chrome}

以上四种方法,都还是报错。

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 67, in get
    context=context,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/django_manage.py", line 52, in <module>
    run_command()
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/django_manage.py", line 46, in run_command
    run_module(manage_file, None, '__main__', True)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/mg/Gamification/manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 332, in execute
    self.check()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 364, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 351, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/registry.py", line 73, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
    all_namespaces = _load_all_namespaces(resolver)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
    url_patterns = getattr(resolver, 'url_patterns', [])
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/utils/functional.py", line 36, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/urls/resolvers.py", line 536, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/utils/functional.py", line 36, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/urls/resolvers.py", line 529, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/mg/Gamification/Gamification/urls.py", line 46, in <module>
    from english_books.views import WordsListView, WordsView, WordProficiencyView, SentenceListView
  File "/Users/mg/Gamification/Gamification/apps/english_books/views.py", line 15, in <module>
    from Gamification.media.books.audios.spider_words import get_icibe_word
  File "/Users/mg/Gamification/Gamification/media/books/audios/spider_words.py", line 41, in <module>
    headers = {"User-Agent": UserAgent(use_cache_server=False).chrome}
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/fake.py", line 69, in __init__
    self.load()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/fake.py", line 78, in load
    verify_ssl=self.verify_ssl,
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 250, in load_cached
    update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 245, in update
    write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 178, in load
    raise exc
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 154, in load
    for item in get_browsers(verify_ssl=verify_ssl):
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 97, in get_browsers
    html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 84, in get
    raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached
写回答

1回答

bobby

2020-06-28

这是因为fake useragent访问的网站 在某些同学的网络下访问不了 可能是有些人使用了连通或者移动的网了, https://github.com/alecxe/scrapy-fake-useragent 可以使用其他的useragent库 github上很多的

0
2
bobby
回复
两努
当然能了啊
2020-07-05
共2条回复

Python爬虫工程师实战 大数据时代必备

慕课网严选精品教程,高质量内容+服务!

2377 学习 · 1158 问题

查看课程