CERTIFICATE_VERIFY_FAILED 错误

Question

Python Django 错误：

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
unable to get local issuer certificate (_ssl.c:1131)>

异常位置：/usr/lib/python3.8/urllib/request.py，第 1357 行，在 do_open 中

运行：Python 3.8.10、Django 4.0.3、Ubuntu 20.04、Apache 2

我正在使用 Django 制作一个简单的联系表单应用程序，该应用程序目前可以运行并且不会抛出任何错误。当我使用此 Django 库时，会发生 CERTIFICATE_VERIFY_FAILED 错误https://github.com/tiesjan/django-hcaptcha-field将 hCaptcha 添加到表单中。

该问题似乎与 django-hcaptcha-field 无关；当访问 hCaptcha API 时，它似乎是 Ubuntu 和 SSL 证书的问题。

我已经查看了 SO 上的多个问题（尤其是https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error（尽管它适用于 OS X）并询问 Ubuntu证书问题。

我已经尝试过所有这些“修复方法”：

pip install pyOpenSSL --upgrade
apt-get install --reinstall python3-certifi
pip install --upgrade certifi --force
apt install --reinstall openssl
apt install ca-certificates
update-ca-certificates --fresh
export SSL_CERT_DIR=/etc/ssl/certs

我“强制”更新了我的 Let's Encrypt SSL。

我尝试更新证书：

wget --quiet https://curl.haxx.se/ca/cacert.pem
export SSL_CERT_FILE=$HOME/cacert.pem

我的 Python 代码中不需要任何内容import ssl

我还可以尝试什么？

---

诊断输出：

dpkg -l | grep cert返回

ica-certificates    20210119~20.04.2    all    Common CA certificates
certbot    0.40.0-1ubuntu0.1    all    automatically configure HTTPS using Let's Encrypt
dirmngr    2.2.19-3ubuntu2.1    amd64    GNU privacy guard - network certificate management service
python-certbot-apache    0.36.0-1    all    transitional dummy package
python3-certbot    0.40.0-1ubuntu0.1    all   main library for certbot
python3-certbot-apache   0.39.0-1    all    Apache plugin for Certbot
ipython3-certifi    2019.11.28-1    all    root certificates for validating SSL certs and verifying TLS hosts (python3)
ssl-cert    1.0.39    all    simple debconf wrapper for OpenSSL

dpkg -l | grep openssl返回

libxmlsec1-openssl:amd64    1.2.28-2    amd64        Openssl engine for the XML security library
openssl    1.1.1f-1ubuntu2.12    amd64        Secure Sockets Layer toolkit - cryptographic utility
perl-openssl-defaults:amd64    4    amd64        version compatibility baseline for Perl OpenSSL packages
python3-openssl    19.0.0-1build1    all          Python 3 wrapper around the OpenSSL library

whereis openssl返回

openssl: /usr/bin/openssl /usr/local/bin/openssl /usr/include/openssl /usr/share/man/man1/openssl.1ssl.gz

which openssl /usr/bin/openssl返回

/usr/local/bin/openssl
/usr/bin/openssl

ldd $(which wget)返回

linux-vdso.so.1 (0x00007ffd9f10f000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007efdb3e3d000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007efdb3e34000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007efdb3e12000)
libssl.so.1.1 => /usr/local/lib/libssl.so.1.1 (0x00007efdb3d7a000)
libcrypto.so.1.1 => /usr/local/lib/libcrypto.so.1.1 (0x00007efdb3a8e000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efdb3a72000)
libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007efdb3a5d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efdb386b000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efdb3848000)
/lib64/ld-linux-x86-64.so.2 (0x00007efdb3f6b000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007efdb36c6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efdb36c0000)

dpkg -l | grep python3-certifi返回

python3-certifi    2019.11.28-1    all  root certificates for validating SSL certs and verifying TLS hosts (python3)

追溯：

Request Method: POST
Request URL: https://example.com/contact/contact/contact/

Django Version: 4.0.3
Python Version: 3.8.10
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'contactform.apps.ContactformConfig',
 'encrypted_files',
 'hcaptcha_field']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1425, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib/python3.8/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()

During handling of the above exception ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)), another exception occurred:
  File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/var/www/html/example.com/public_html/contact/contactform/views.py", line 26, in contact
    if form.is_valid():
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 200, in errors
    self.full_clean()
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 433, in full_clean
    self._clean_fields()
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 445, in _clean_fields
    value = field.clean(value)
  File "/usr/local/lib/python3.8/dist-packages/django/forms/fields.py", line 199, in clean
    self.validate(value)
  File "/usr/local/lib/python3.8/dist-packages/hcaptcha_field/fields.py", line 129, in validate
    response = opener.open(request, timeout=hcaptcha_settings.TIMEOUT)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)

Exception Type: URLError at /contact/contact/
Exception Value: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)>

hcaptcha_field 的 fields.py https://github.com/tiesjan/django-hcaptcha-field

import json
import logging
import ssl # added #############
import certifi # added #############
from urllib.error import HTTPError
from urllib.parse import urlencode
from urllib.request import build_opener, Request, ProxyHandler

from django import forms
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _

from hcaptcha_field.settings import hcaptcha_settings
from hcaptcha_field.widgets import hCaptchaWidget


LOGGER = logging.getLogger('hcaptcha_field')


DATA_ATTRIBUTE_CONFIG = frozenset([
    'theme',
    'size',
    'tabindex',
    'callback',
    'expired-callback',
    'chalexpired-callback',
    'open-callback',
    'close-callback',
    'error-callback',
])


QUERY_PARAMETER_CONFIG = frozenset([
    'onload',
    'render',
    'hl',
    'recaptchacompat'
])


class hCaptchaField(forms.Field):
    widget = hCaptchaWidget
    default_error_messages = {
        'error_hcaptcha': _(
            # Translators: Error shown when an internal server error occurred.
            'Something went wrong while verifying the hCaptcha. '
            'Please try again.'
        ),
        'invalid_hcaptcha': _(
            # Translators: Error shown when visitor did not pass the hCaptcha check.
            'hCaptcha could not be verified.'
        ),
        'required': _(
            # Translators: Error shown when visitor forgot to fill in the hCaptcha.
            'Please prove you are human.'
        ),
    }

    def __init__(self, sitekey=None, **kwargs):
        """
        Initializer for `hCaptchaField` class. It determines data attributes
        for the widget class and constructs a widget if none is given. This
        constructed widget receives the URL of the JavaScript resource for the
        hCaptcha integration and the `sitekey` of the site to protect.
        """
        # Retrieve settings
        DEFAULT_CONFIG = hcaptcha_settings.DEFAULT_CONFIG
        JS_API_URL = hcaptcha_settings.JS_API_URL
        SITEKEY = hcaptcha_settings.SITEKEY

        # Determine widget data attributes
        self.widget_data_attrs = {}
        for setting in DATA_ATTRIBUTE_CONFIG:
            if setting in kwargs:
                self.widget_data_attrs[setting] = kwargs.pop(setting)
            elif setting in DEFAULT_CONFIG:
                self.widget_data_attrs[setting] = DEFAULT_CONFIG[setting]

        # If the `widget` argument is not given, instantiate `self.widget` with
        # the hCaptcha API url and the sitekey
        if 'widget' not in kwargs:
            # Determine hCaptcha API url
            query_params = {}
            for setting in QUERY_PARAMETER_CONFIG:
                if setting in kwargs:
                    query_params[setting] = kwargs.pop(setting)
                elif setting in DEFAULT_CONFIG:
                    query_params[setting] = DEFAULT_CONFIG[setting]
            if query_params:
                js_api_url = '%s?%s' % (JS_API_URL, urlencode(query_params))
            else:
                js_api_url = JS_API_URL

            # Determine hCaptcha sitekey
            self.sitekey = sitekey or SITEKEY

            # Instantiate widget
            kwargs['widget'] = self.widget(
                    js_api_url=js_api_url, sitekey=self.sitekey)

        super().__init__(**kwargs)

    def widget_attrs(self, widget):
        """
        Returns the widget attributes, including all the data attributes
        determined in the initializer.
        """
        attrs = super().widget_attrs(widget)
        for key, value in self.widget_data_attrs.items():
            attrs['data-%s' % key] = value
        return attrs

    def validate(self, value):
        """
        Validates the field by verifying the value of the hidden field
        `h-captcha-response` with their API endpoint.
        """
        super().validate(value)

        # Build request
        opener = build_opener(ProxyHandler(hcaptcha_settings.PROXIES))
        post_data = urlencode({
            'secret': hcaptcha_settings.SECRET,
            'response': value,
            'sitekey': self.sitekey,
        }).encode('utf-8')
        request = Request(hcaptcha_settings.VERIFY_URL, post_data)

        # Perform request
        try:
            context=ssl.create_default_context(cafile=certifi.where()) # added ############
            response = opener.open(request, timeout=hcaptcha_settings.TIMEOUT)
        except HTTPError:
            LOGGER.exception("Failed to verify response with hCaptcha API.")
            raise ValidationError(
                self.error_messages['error_hcaptcha'],
                code='error_hcaptcha'
            )

        # Check response
        response_data = json.loads(response.read().decode('utf-8'))
        if not response_data.get('success'):
            LOGGER.error("Failed to pass hCaptcha check: %s", response_data)
            raise ValidationError(
                self.error_messages['invalid_hcaptcha'],
                code='invalid_hcaptcha'
            )

Answer 1

看起来需要context传递到HTTPSHandler以便使用这些certifi证书。

/usr/local/lib/python3.8/dist-packages/hcaptcha_field/fields.py尝试使用以下内容修补您的本地副本。

正如您已经完成的那样，在导入中添加：

import certifi
import ssl

此外，扩大导入范围urllib.request以包括HTTPSHandler：

from urllib.request import build_opener, Request, ProxyHandler, HTTPSHandler

另外，正如您已经完成的那样，创建一个使用certifi证书的 SSL 上下文。不过，我会将其添加到导入块下方，而不是模块代码深处：

context=ssl.create_default_context(cafile=certifi.where())

然后，紧接着在此新行下方，使用context=...以下命令创建一个新实例：HTTPSHandlercontext

https_handler = HTTPSHandler(context=context)

然后，修改opener使用调用定义的行build_opener(...)以包含新HTTPSHandler实例：

opener = build_opener(https_handler, ProxyHandler(hcaptcha_settings.PROXIES))

---

如果我正在阅读build_opener文档正确地说，当机器尝试打开 URL 时，build_opener它已经使用了默认实例。HTTPSHandler希望用这个新实例替换那个默认实例https_handler，该新实例将与包含证书的 SSL 上下文一起运行certifi，从而允许urlopen调用正常工作。

如果这做工作，那么向上游项目提出问题/PR 可能是一个好主意，以集成这个逻辑并提供默认行为（certifi如果证书可用则自动提取证书），或者包含一个配置选项以允许用户接入证书certifi。

---

更新 2022-04-14：因为这确实解决了这个问题，我将添加更多细节来解释为什么需要进行此修复以及它是如何起作用的。

HTTPS要求您访问的网站提供可验证其身份的证书。否则，您如何知道您的流量没有被干扰、中间人攻击等？网站在您浏览时向您提供的证书是由网站运营商颁发的证书颁发机构。该机构充当受信任的第三方，为网站运营商“担保”，并且“是的，您可以信任他们颁发的证书”。

有好几个这样的发证机构。这些机构颁发自己的证书（“证书颁发机构证书”或“CA 证书”），这些证书与各个网站提供的证书完全不同。您的浏览器（或 URL 打开器）需要访问这些 CA 证书才能完成网站提供的证书的“受信任的第三方”验证。 OP 在这里面临的关键错误是 URL 打开器无法找到一组用于第三方验证的良好 CA 证书：

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)>

该消息unable to get local issuer certificate意味着在系统上urlopen找不到所需的issuer certificate（CA证书）local。

所以，解决方案是指向包urlopen提供的 CA 证书certifi：

• 为了告诉opener在哪里可以找到它们，你必须在创建时提供它们的opener位置build_opener

• build_opener基于处理程序对象集合工作，因此 CA 证书信息必须包含在合适的处理程序中

• 唯一一个包含有关 SSL 上下文信息的处理程序（我在文档中可以找到）是HTTPSHandler

• HTTPSHandler获取 SSL 相关配置信息的方式是通过context创建新实例时的关键字参数，该参数采用“SSL 上下文”对象

• ssl.create_default_context是（据我所知）用于创建 SSL 上下文对象的工厂函数，可以通过多种方式进行定制

• 可定制的方式之一是使用 CA 证书存储的位置（通过参数cafile）

• certificertifi.where()通过以下函数提供其精选 CA 证书存储在磁盘上的位置

如果你从下到上扫描这些项目符号，它们会很好地映射到答案中的每个步骤。