m-click/requests_pkcs12

Add PKCS#12 support to the Python requests library in a clean way, without monkey patching or temporary files

PKCS#12 support for requests

This library adds PKCS#12 support to the Python requests library.

It is a clean implementation: it uses neither monkey patching nor unencrypted temporary files. Instead, it is integrated into requests as recommended by its authors: creating a custom TransportAdapter, which provides a custom SSLContext.

This library is meant to be a transitional solution until this functionality is provided by requests directly. However, that will take some time. See the corresponding issue for more details.

Usage

For simple one-off requests you can use this library as a drop-in replacement for the requests library:

from requests_pkcs12 import get

r = get('https://example.com/test', pkcs12_filename='clientcert.p12', pkcs12_password='correcthorsebatterystaple')

If you are using requests sessions, use the Pkcs12Adapter:

from requests import Session
from requests_pkcs12 import Pkcs12Adapter

with Session() as s:
    s.mount('https://example.com', Pkcs12Adapter(pkcs12_filename='clientcert.p12', pkcs12_password='correcthorsebatterystaple'))
    r = s.get('https://example.com/test')

Installation

This library is available as PyPI package:

pip install requests-pkcs12

Alternatively, you can retrieve the latest development version via Git:

git clone https://github.com/m-click/requests_pkcs12

Arguments

The following keyword arguments are supported:

• pkcs12_filename is a byte string or unicode string that contains the file name of the encrypted PKCS#12 certificate.
  • Either this argument or pkcs12_data must be provided.
• pkcs12_data is a byte string that contains the encrypted PKCS#12 certificate data.
  • Either this argument or pkcs12_filename must be provided.
• pkcs12_password is a byte string or unicode string that contains the password.
  • This argument must be provided whenever pkcs12_filename or pkcs12_data is provided.
• ssl_protocol is a protocol version from the ssl library.
  • This argument is optional and defaults to ssl.PROTOCOL_TLS.

If you use these parameters, don’t use the built-in cert parameter of requests at the same time. However, do use the other parameters. In particular, do use the "verify" parameter to verify the server-side certificate.

requests_pkcs12's Issues

OpenSSL.SSL.Error: [('SSL routines', '', 'ca md too weak')]

On a machine I'm running it has OpenSSL 3.x, the CA I'm using cannot be changed and is stuck using sha1.

Is there a way from code with this library to set it/open ssl to allow this now considered 'weak' hashing ?

I've been trying to guess and check with things like the following -- but no change in behavior

requests_pkcs12.urllib3.contrib.pyopenssl.DEFAULT_SSL_CIPHER_LIST = 'ALL:@SECLEVEL=0'
requests_pkcs12.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS = 'ALL:@SECLEVEL=0'
requests_pkcs12.OpenSSL.crypto.DEFAULT_CIPHER = 'ALL:@SECLEVEL=0'

Getting Post-Handshake Authentication to work with requests_pkcs12 and TLSv1.3

Hello,

First off, I don't know if this is a requests_pkcs12 problem or a requests problem, so sorry for the noise if it hasn't anything todo with your repo.

Using requests_pkcs12 as drop-in replacement for requests, without much problems, except being able to use TLSv1.3 when using a p12 certificate.

As soon as I configure my apache server to use TLSv1.3 via SSLProtocol -All +TLSv1.3, I get the following error:

[Mon Dec 23 13:25:54.645447 2019] [ssl:error] [pid 19365:tid 139995724711680] [client 82.196.24.2:26826] AH: verify client post handshake
[Mon Dec 23 13:25:54.645487 2019] [ssl:error] [pid 19365:tid 139995724711680] [client 82.196.24.2:26826] AH10158: cannot perform post-handshake authentication
[Mon Dec 23 13:25:54.645506 2019] [ssl:error] [pid 19365:tid 139995724711680] SSL Library Error: error:14268117:SSL routines:SSL_verify_client_post_handshake:extension not received

On the python client side, I get a HTTP 403.

Having read alot about that problem, I tried to find a way to force post handshake authentication which seems necessary for TLSv1.3, see https://bugs.python.org/issue37428
TLSv1.3 post handshake support has been included in urllib3 1.25.7 as it seems (urllib3/urllib3#1635 )

I am using requests 2.22.0, urllib3 1.25.7 and requests_pkcs12 1.6.

I've searched for a way to force adding post handshaking by adding a on line 41 of requests_pkcs12.py:

ssl_context.post_handshake_auth = True

I tried to read PyOpenSSLContext class, but in the end I don't know where to force enable the post handshake authentication.

Could you help me perhaps ?
Willing to make a lot of tests if required.

Best regards.

[Edit]
I've searched for https://docs.python.org/3/library/ssl.html#ssl.SSLContext.post_handshake_auth
Seems this only works with Python 3.8.
Decided to upgrade, then set the option.
I am unable to get this to work.
[/Edit]

Improve error reporting

When specifying an incorrect password for the certificate requests_pkcs12 will raise a general exception Error([('PKCS12 routines', 'PKCS12_parse', 'mac verify failure')]), which is a pain to catch. Would it be possible to use a more specific exception, or one of requests.exceptions? Also I couldn't find any information about which exceptions can be raised. Or is there any easy way to handle that exception I don't know about?

Create `ssl_context` only once

The ssl_context should be created only once, instead of storing the PKCS#12 password in memory for the whole lifetime of the adapter.

How do I make a POST request?

Can't find the body/payload argument in the post function

create_ssl_sslcontext raises `TypeError: Encryption algorithm must be a KeySerializationEncryption instance`

I get TypeError: Encryption algorithm must be a KeySerializationEncryption instance when invoking create_ssl_sslcontext function

After changing https://github.com/m-click/requests_pkcs12/blob/master/requests_pkcs12.py#L69 to
cryptography.hazmat.primitives.serialization.BestAvailableEncryption(password=pkcs12_password_bytes)
everything is working fine

Invalid empty password

In version 1.16 the argument pkcs12_password can not be an empty string, but in version 1.15 it works.
This is the error received in version 1.16 raise ValueError("Password must be 1 or more bytes.")

This is an expected behavior?

must ‘pkcs12_filename’ be set to a full path?

`

• def init(self, url, method, payload=None, json=None, parameter=None, header=None):

•     """ set some parameters about the request"""
  

•     self.url = url
  

•     self.method = method
  

•     self.parameter = parameter
  

•     self.payload = payload
  

•     self.json = json
  

•     self.header = header
  

•     self.pkcs12_filename = "./cert/arpdyqrqay.p12"
  

•     self.pkcs12_password = '123'
  

`
fault:
Traceback (most recent call last):
File "/Users/zz/PycharmProjects/SDP-test/test1.py", line 31, in
ApiTest.sign_up("[email protected]", 'young', 'QQqq11!!')
File "/Users/zz/PycharmProjects/SDP-test/API/api_summary.py", line 70, in sign_up
sign_result = sign.send_method()
File "/Users/zz/PycharmProjects/SDP-test/API/send_method.py", line 31, in send_method
response = requests_pkcs12.request(method=self.method, url=self.url, data=self.payload, json=self.json,
File "/Users/zz/PycharmProjects/SDP-test/venv/lib/python3.8/site-packages/requests_pkcs12.py", line 121, in request
pkcs12_adapter = Pkcs12Adapter(
File "/Users/zz/PycharmProjects/SDP-test/venv/lib/python3.8/site-packages/requests_pkcs12.py", line 93, in init
with open(pkcs12_filename, 'rb') as pkcs12_file:
FileNotFoundError: [Errno 2] No such file or directory: './cert/arpdyqrqay.p12'

Library does not work correctly with transfer-encoding: chunked

Although the library is working correctly with p12 files when receiving headers and the first chunk of a response with no content-length but transfer-chunked enabled, it has a problem when attempting to receive the next chunks, it throws an exception of IncompleteRead when in fact it should be reading the next chunk.

It is not a problem of the server side because I am getting the correct information using Java. Specifically, I used HttpURLConnection Java class and it works fine (plus setting the parameters -Djavax.net.ssl.keyStoreType=pkcs12 -Djavax.net.ssl.keyStore=... -Djavax.net.ssl.keyStorePassword=...). It also works fine with Java class HttpClient.

requests-pkcs12=1.3 can't have pkcs12_password=''

Hi
This worked fine in v1.2:

with Session() as s:
    s.mount(args.url, Pkcs12Adapter(pkcs12_filename='Profil.NET.Api.ResponseCenter.Client.p12', pkcs12_password=''))

But in v1.3:

Traceback (most recent call last):
  File ".\connect.py", line 41, in <module>
    s.mount(args.url, Pkcs12Adapter(pkcs12_filename='Profil.NET.Api.ResponseCenter.Client.p12', pkcs12_password=''))
  File "D:\dev\flask\lib\site-packages\requests_pkcs12.py", line 71, in __init__
    raise ValueError('Insufficient data to create SSL Context')
ValueError: Insufficient data to create SSL Context

my *.p12 cert does not have a password. which is a fully allowed config.

Using a retry strategy

Hi,
is it possible using an Pkcs12Adapter and still using a retry-strategy like in the following post:

https://majornetwork.net/2022/04/handling-retries-in-python-requests/

from urllib3 import Retry
session = requests.Session()
adapter = HTTPAdapter(max_retries=Retry(total=4, backoff_factor=1, allowed_methods=None, status_forcelist=[429, 500, 502, 503, 504]))

Background of my question:
We detect many connection refused errors, when connecting from a specific client (flask app that using an Pkcs12Adapter), like:

Max retries exceeded with url: / (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x7f492268db20>: Failed to establish a new connection: [Errno 111] Connection refused’)

To resolve that problem i am wants to try out a retry-strategy.

Does it work with a pfx file?

Cannot Set verify_mode to CERT_NONE when check_hostname is Enabled

Latest version doesn't work with verify=False, getting the following error

Cannot Set verify_mode to CERT_NONE when check_hostname is Enabled

Works fine on version 1.14

cert now not being read as x509 instance

Just last week, the implementation that I used to make requests with pkcs12 file format was working perfectly using a file with .p12 extension. This morning, it is now not able to verify my cert as an x509 instance even though I am able to extract the certificate from the .p12 file and verify that the cert is an OpenSSL.crypto.X509 object. Whenever I run the following:
p12 = crypto.load_pkcs12(open(path_to_cert, 'rb').read(), password)
crt = p12.get_certificate()
print(crt)

The type of crt is X509.

I am making a get request and arguments include pkcs12_filename with corresponding .p12 file and pkcs12_password with corresponding password. The below is the error that is returning:

File "/opt/python39/lib/python3.9/site-packages/requests_pkcs12.py", line 132, in get return request('get', *args, **kwargs)
File "/opt/python39/lib/python3.9/site-packages/requests_pkcs12.py", line 118, in request pkcs12_adapter = Pkcs12Adapter(
File "/opt/python39/lib/python3.9/site-packages/requests_pkcs12.py", line 95, in __init__ self.ssl_context = create_pyopenssl_sslcontext(pkcs12_data, pkcs12_password_bytes, ssl_protocol)
File "/opt/python39/lib/python3.9/site-packages/requests_pkcs12.py", line 45, in create_pyopenssl_sslcontext ssl_context._ctx.use_certificate(cert)
File "/opt/python39/lib/python3.9/site-packages/OpenSSL/SSL.py", line 861, in use_certificate raise TypeError("cert must be an X509 instance")
TypeError: cert must be an X509 instance

Again, it was working just last week with the same exact .p12 file and I have not made any changes at all. I am curious if this is the cause of a change in this repo or not. If anyone is able to replicate this, please comment below. Thank you.

`DeprecationWarning` for`urllib3.contrib.pyopenssl`

I am getting a DeprecationWarning from urllib3 as described here: urllib3/urllib3#2691. Is it possible to fix this?

On a related note, are there plans to allow urllib3>=2.0.0?

urllib3 version minimum

The following packages and versions are a valid environment set-up, given the current dependencies in requests_pkcs12 and its dependencies.

certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.0.12
cryptography==39.0.0
idna==3.4
pycparser==2.21
pyOpenSSL==23.0.0
requests==2.28.1
requests-pkcs12==1.14
urllib3==1.26.0

However, this environment results in the following error, when one tries to import requests_pkcs12.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jessicamoore/Documents/PythonEnvs/test-requestspkcs12/lib/python3.10/site-packages/requests_pkcs12.py", line 30, in <module>
    import urllib3.contrib.pyopenssl
  File "/Users/jessicamoore/Documents/PythonEnvs/test-requestspkcs12/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 53, in <module>
    from cryptography.hazmat.backends.openssl.x509 import _Certificate
ModuleNotFoundError: No module named 'cryptography.hazmat.backends.openssl.x509'

The easiest solution seems to be adding a minimum version of urllib3 (>=1.26.13).

Fails to install using python36 when system encoding is not UTF-8

Noticed a small issue when trying to install this from a source distribution on an system where the locale is set to POSIX and python version is 3.6 (this was the situation we had on a Jenkins build box we were using this project on)

[test@localhost requests_pkcs12]$ export LANG=POSIX
[test@localhost requests_pkcs12]$ export LC_ALL=POSIX
[test@localhost requests_pkcs12]$ locale
LANG=POSIX
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=POSIX
[test@localhost requests_pkcs12]$ pip3 install .
Processing /var/tmp/requests_pkcs12
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-k4kckmm2-build/setup.py", line 6, in <module>
        long_description=open('README.rst').read(),
      File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2244: ordinal not in range(128)
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-k4kckmm2-build/

We figured out this is due to the open() function call in setup.py not being explicit on the encoding (uses the system encoding returned locale.getpreferredencoding())

In our case the simple fix was to run this before the build:

export LC_ALL=en_US.utf8

I have also done a PR that makes sure the README.rst can be decoded correctly by setup.py regardless of the system locale. #18

Error connecting to the url

I am trying to connect to encrypted server.
I am running the provided example code from ReadMe:

from requests_pkcs12 import get

r = get('https://127.0.0.1:8443/serviceregistry/echo',
        pkcs12_filename='C:/Users/aa/Documents/project/newcertificates/mycloud.aitia/sysop.p12',
        pkcs12_password='743656375')
print(r)

The output is as following:

Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\runpy.py", line 88, in _run_code        
    exec(code, run_globals)
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher/../..\debugpy\__main__.py", line 39, in <module>
    cli.main()
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 430, in main
    run()
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "c:\Users\aa\.vscode\extensions\ms-python.debugpy-2024.6.0-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "C:\Users\aa\Documents\project\client-library-python\examples\quickstart\test.py", line 3, in <module>
    r = get('https://127.0.0.1:8443/serviceregistry/echo',
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests_pkcs12.py", line 152, in get
    return request('get', *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests_pkcs12.py", line 145, in request
    return session.request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests_pkcs12.py", line 124, in send
    return super(Pkcs12Adapter, self).send(request, stream, timeout, verify, cert, proxies)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\urllib3\connection.py", line 362, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "C:\Users\aa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\urllib3\util\ssl_.py", line 399, in ssl_wrap_socket
    return context.wrap_socket(sock)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 517, in wrap_socket       
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\ssl.py", line 1032, in _create
    raise ValueError("check_hostname requires server_hostname")
ValueError: check_hostname requires server_hostname

Import PyOpenSSLContext directly instead of via requests.packages

We should import PyOpenSSLContext directly instead of via requests.packages.

See psf/requests#5561 for further details.

requests.exceptions.SSLError with tls_process_server_certificate, certificate verify failed for valid p12

Hi, thanks for the library, i was struggling to make p12 file with requests. I also have pem and cer files too but could not make it work yet.

Like expected I entered p12 path and password, verified they are used by giving wrong values etc. I thing i am getting error from server side that certificate is not accepted, but I am using soap-ui with this p12 file and same password and it just works from same laptop.

can you think a reason why it is not working, I checked it is tls 1.2 not 1.3 also. Checked closed issues but could not find related info.

code is very simple

from requests_pkcs12 import get
r = get('https://wssoap.services.xxxxxx.com/xxxxxx?wsdl',
pkcs12_filename='/project_folder/certs/xxxxx.p12', pkcs12_password='xxxxxx')

Traceback (most recent call last):
File "/project_folder/functions/list-trames/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
cnx.do_handshake()
File "/project_folder/functions/list-trames/OpenSSL/SSL.py", line 1934, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "/project_folder/functions/list-trames/OpenSSL/SSL.py", line 1671, in _raise_ssl_error
_raise_current_error()
File "/project_folder/functions/list-trames/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

Exception has occurred: ModuleNotFoundError

Line from requests_pkcs12 import get,post raises that exception, and altough I've tried several hints, they all don't work:
No module named 'cryptography.hazmat.backends.openssl.x50
I'm running python in a venv, no warnings during pip install - any ideas?

Adding headers to the call

Hey @vog, just got around to checking your solution out from that main issue thread. Quick question: is there a way to pass header information along with the request call? I'm poking around the code a little bit but don't see it, but I'm also not super familiar with requests so I'm reading their docs concurrently.

I tried adding headers to the init function and updating the session's header info in the request function but keep getting a 405 error when I call it.

Thanks,
Mike

linking closed, duplicate requests issue instead of open variant

Hi,
just a quick observation:
README.rst is linking requests issue 1573 which has since been closed as duplicate of 2519. That issue is still open ans afaics there is no solution.

anyway, I though it silly to create a pull request :D

`cryptography.x509.Certificate.not_valid_after` is deprecated

cryptography.x509.Certificate.not_valid_after is deprecated since version 42.0.0. It is used in library:

requests_pkcs12/requests_pkcs12.py

Line 38 in fe83fb5

if cert_not_after < datetime.datetime.utcnow():

It can use cryptography.x509.Certificate.not_valid_after_utc instead but then need cryptography>= 42.0.0 in setup.py.

What are you thinking?

I can do a PR.

Error import -> from requests_pkcs12 import get

When i import library with command from requests_pkcs12 import get i receive error:

No module named 'cryptography.hazmat.bindings._constant_time'

Any idea?

Thanks.

OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'sslv3 alert certificate unknown')]

Hi,

Thank you for the library. I wrote a piece of codes on Repl and that works very well. However, when I run the same piece of code on my machine, the error stated in the title occurs... The different I notice is that OpenSSL on local machine is 1.0.2 and on the Repl is 1.0.1. May I know what else may cause this kind of issue? Thank you!

Yours,
Victor

OpenSSL.crypto.Error: [('asn1 encoding routines', 'asn1_check_tlen', 'wrong tag'), ('asn1 encoding routines', 'asn1_item_embed_d2i', 'nested asn1 error')]

From the given info:

        if soap:
            from requests import Session
            from requests_pkcs12 import Pkcs12Adapter
            with Session() as s:
                s.mount(url, Pkcs12Adapter(pkcs12_filename=cert_bundle, pkcs12_password=password))
                r = s.post(url,
                    headers={'Content-Type': 'application/xml'},
                    data=soap(day=date.strftime('%Y-%m-%d')))

python -m requests.help                                                                                                                                            
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": "2.2.2"
  }, 
  "idna": {
    "version": "2.6"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.10"
  }, 
  "platform": {
    "release": "17.4.0", 
    "system": "Darwin"
  }, 
  "pyOpenSSL": {
    "openssl_version": "1010008f", 
    "version": "17.5.0"
  }, 
  "requests": {
    "version": "2.18.4"
  }, 
  "system_ssl": {
    "version": "20000000"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": true
}

... this works

MockRequest = namedtuple('request', 'status_code text content url')

def post(url, date, soap):
    from io import BytesIO
    import pycurl
    buffer = BytesIO()
    c = pycurl.Curl()
    c.setopt(c.URL, url)
    c.setopt(c.SSLCERT, oati_cert)
    c.setopt(c.SSLCERTPASSWD, password)
    c.setopt(c.POST, True)
    c.setopt(c.POSTFIELDS, unicode(soap(day=date.strftime('%Y-%m-%d'))))
    c.setopt(c.SSL_VERIFYPEER, False)
    c.setopt(c.VERBOSE, 1)
    c.setopt(c.WRITEDATA, buffer)
    c.perform()
    c.close()
    body = buffer.getvalue()
    return MockRequest(status_code=200,
                       content=body,
                       text=None,
                       url=url)

ImportError: cannot import name 'PyOpenSSLContext'

ImportError: cannot import name 'PyOpenSSLContext'

when i try to use this library. Can anyone help me with the fix? I cant find PyOpenSSLContext anywhere online.

>>> from requests_pkcs12 import Pkcs12Adapter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\requests_pkcs12.py", line 24, in <module>
    from requests.packages.urllib3.contrib.pyopenssl import PyOpenSSLContext
ImportError: cannot import name 'PyOpenSSLContext'

Create an `ssl.SSLContext`

httpx is growing in popularity as the "next generation" requests.py. It can accept custom ssl.SSLContext objects but not PyOpenSSLContext, encode/httpx#924.

create_ssl_context returns a PyOpenSSLContext. Could that optionally return an ssl.SSLContext or another function added for that support?

Thanks.

Problem with non-TLS-1.2 server

I'm trying to read the following:
r = get("https://eloverblik.dk/api/authorizationsv2", pkcs12_filename="C:/PathToCertificate.p12", pkcs12_password="mypassword")
however I get the result
SSLError: HTTPSConnectionPool(host='eloverblik.dk', port=443): Max retries exceeded with url: /api/authorizationsv2 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'SSL3_GET_RECORD', 'wrong version number')],)",),))
I'm able to open and read the certificate using OpenSSL, so the certificate is working. I can also read the url in Firefox.
I am using requests_pkcs12 version 1.0.5 and python 3.6.1.
Any ideas what goes wrong?

not_valid_after property is invoked as a method producing errors

@vog I believe not_valid_after is a property that returns datetime.datetime object, not a callable?

line 37, in check_cert_not_after
   cert_not_after = cert.not_valid_after()
TypeError: 'datetime.datetime' object is not callable

Commit is from cb09e98

REF: https://github.com/pyca/cryptography/blob/87f43fb77e7f1c56471d3233a56000a51dc4a311/src/cryptography/x509/base.py#L118

DeprecationWarning: PKCS#12 support in pyOpenSSL is deprecated

Thank you for the great package, I'm reporting the following deprecation warning that is triggered within one of my applications:

/Users/REDACTED/Library/Caches/pypoetry/virtualenvs/REDACTED-PACKAGE-2-MN45-t-py3.9/lib/python3.9/site-packages/requests_pkcs12.py:20: DeprecationWarning: PKCS#12 support in pyOpenSSL is deprecated. You should use the APIs in cryptography.
  from OpenSSL.crypto import load_pkcs12, dump_certificate, dump_privatekey, FILETYPE_PEM

I have not had a chance to look into how difficult this would be to address but wanted to let you know in case you were not aware.

Thanks again for the great package, I hope to take a look at this and see if I may be able to contribute a fix, I know it can be frustrating to get bug reports with no solutions or help. If you are interested, please let me know so as to not create any confusion

403 Forbidden error when using requests_pkcs12 to send POST request with valid certificate

Issue Summary:
I am encountering a 403 Forbidden error when attempting to send a POST request using the "requests_pkcs12" library, despite having a valid certificate and successfully receiving a 200 response when using curl request.

I am using 'requests_pkcs12' Version:-1.24

`import requests_pkcs12
cert_path = "./SSLRequests/carefi_tu_uat_p12_cert.p12"
cert_password = "hello.@101"
url = "https://*******.com/acquire/credit-assessment/v1/consumer-cir-cv"
header = {}
payload={}

response = requests_pkcs12.post(
url,
headers=header,
data=payload,
pkcs12_filename=cert_path,
pkcs12_password=cert_password
)
The response.__dict__ is as following:-{'_content': b'{"message":"Forbidden. You don't have permission to view this. Please contact your system administrator.","status_code":403,"aud":"d4ae1f449b5968gjk68995vhj688d9d2d0","ray_id":"88764a7","ip_address":"12..14.","is_warp":false,"is_gateway":false,"mtls_status":"NONE"}',
'_content_consumed': True,
'_next': None,
'status_code': 403,
'headers': {'Date': 'Tue, 21 May 2024 18:30:17 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'CF-Access-Aud': 'd4a5bbbbb9d9ee1f44757f8d9d2dc1f68946b0', 'CF-Access-Domain': '', 'CF-RAY': '8876********-BOM', 'cf-version': '1701-77418d4', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'Server': 'cloudflare', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip'},
'raw': <urllib3.response.HTTPResponse at 0x7f88eb53d480>,
'url': 'https://************.com/acquire/credit-assessment/v1/consumer-cir-cv',
'encoding': 'utf-8',
'history': [],
'reason': 'Forbidden',
'cookies': <RequestsCookieJar[]>,
'elapsed': datetime.timedelta(microseconds=115372),
'request': <PreparedRequest [POST]>,
'connection': <requests_pkcs12.Pkcs12Adapter at 0x7f88f06ba540>}`

SSLV3_ALERT_BAD_CERTIFICATE when switching to Pkcs12Adapter

When switching from classic requests cert parameter to Pkcs12Adapter I always get SSLV3_ALERT_BAD_CERTIFICATE from the server.
In wireshark I can see that there are no certificates send to the server. The certificates part is empty. Any ideas?

`

    backend = default_backend()
    p12file = self.application.flink_client_keystore()

    with open(p12file, 'rb') as pkcs12_file:
        pkcs12_data = pkcs12_file.read()
    pkcs12_password_bytes = "confluent".encode('utf8')

    p12 = crypto.load_pkcs12(pkcs12_data, pkcs12_password_bytes)
    pycaP12 = load_key_and_certificates(pkcs12_data, pkcs12_password_bytes, backend)
    pk_bytes = pycaP12[0].private_bytes(Encoding.DER, PrivateFormat.PKCS8, NoEncryption())
    cert_bytes = pycaP12[1].public_bytes(Encoding.DER)

    cert = dump_certificate(FILETYPE_PEM, p12.get_certificate())
    pk = dump_privatekey(FILETYPE_PEM, p12.get_privatekey())

    with open('test_cert.pem', 'wb') as pem_file:
        pem_file.write(cert)
    with open('test_cert.key', 'wb') as key_file:
        key_file.write(pk)

    self.session = requests.Session()

    # WORKS!
    #self.session.cert = ('test_cert.pem', 'test_cert.key')

    # DOESN'T WORK
    self.session.mount('{baseurl}'.format(baseurl=self.active_jobmanager_url), Pkcs12Adapter(pkcs12_data=pkcs12_data, pkcs12_password="confluent".encode('utf8')))

`

Incompatibility with requests 2.32.3 - custom SSLContext via a Transport Adapter

This might be a false positive - but from preliminary experimentation, it seems like a recent change in requests might cause issues with requests_pkcs12.

With requests 2.32.0, a breaking change was introduced that returned every call with that involved requests_pkcs12 with SSLContext via a Transport Adapter.

requests 2.32.3 (still unreleased as of right now, apparently scheduled for Tuesday) aims to fix this issue (Allow for overriding of specific pool key params #6716), but now all calls involving requests_pkcs12 fail with SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate for me.

I'll try to dig a little deeper and see if this is indeed something that requires changes here - I just wanted to provide a heads-up in case 2.32.3 is indeed shipping a breaking change.

Minimal example:

import requests
from requests_pkcs12 import Pkcs12Adapter

client = requests.Session()
client.mount(
    'https://some.system/',
    Pkcs12Adapter(
        pkcs12_data=self.cert.read(),
        pkcs12_password=self.cert_password,
)

client.get('https://some.system/foo/bar')

HTTPSConnectionPool(host='some.system', port=443): Max retries exceeded with url: /foo/bar (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)')))

Add example of two-way TLS/SSL configuration

When I read the last line of the README.md file:

If you use these parameters, don't use the built-in cert parameter of requests at the same time.

I thought that the verify field shouldn't be used as well, which resulted in lots of attempts which didn't happen to work.
The only way I succeded in setting up two-way SSL was by puting the root CA's certificate filename in the verify field (rootCA.pem in my case).

Suggestion: add a line below the quoted one saying something like this.

This doesn't mean that You can't use the verify field, which should be used for two-way SSL. Check out the requests library documentation for the verify field.

P.S. Thank you for this small, but useful library! 🙂