roadmaster/forcediphttpsadapter

A requests TransportAdapter allowing to force a specific IP for HTTPS connections.

forcediphttpsadapter's Issues

if verify=False needed?

Hi,

I have not been able to use this adapter, maybe I just misunderstood the goal, maybe I misused it...
I wanted to use it to test a configuration before the dns is property set in the situation I have proper certificate in place: the same test I would obtain cheating with resolution in /etc/hosts. In this frame I need that verification of th ssl chain is done! that's part of what I want to test.

In your example, you explicitly set it to False. Do I misunderstand what is this adapter intended to do or is impossible to achieve?

Moreover, in the example you write session.get('/some/path', ...) it seems to me that you need to put there an address, while the path, seems to refer to parsed_url.path

Python 3.10 and higher emits deprecation warning for upcoming Python 3.12

python3.11/site-packages/forcediphttpsadapter/adapters.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if StrictVersion(requests.__version__) < StrictVersion('2.4.0'):

See https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated

Multiple host names on IP

Hello, can you please help me - I am trying to understand how to get responses from IP hosting several websites
I found than 34.240.28.46 hosts the site I am trying to get responses from. With normal requests I get:

SSLError: HTTPSConnectionPool(host='34.240.28.46', port=443): Max retries exceeded with url: /jobs/accountancy (Caused by SSLError(CertificateError("hostname '34.240.28.46' doesn't match either of 'totaljobs.com', 'careerstructure.com', 'caterer.com', 'catererglobal.com', 'cityjobs.com', 'cwjobs.co.uk', 'emedcareers.com', 'jobsite.co.uk', 'justengineers.net', 'milkround.com', 'retailchoice.com'")))

When using ForcedIPHTTPSAdapter:

url = 'https://totaljobs.com/jobs/accountancy'
session = requests.Session()
session.mount(url, ForcedIPHTTPSAdapter(dest_ip='34.240.28.46'))
r = session.get(url, verify=False, timeout=10)

I get timeout error.
Can somone please help me understand what I am doing wrong

AttributeError: 'ForcedIPHTTPSConnectionPool' object has no attribute 'strict'

It does not work since I have upgraded to Python 3.10 and reinstalled the venv.

Traceback:

  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/urllib3/connectionpool.py", line 769, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/urllib3/connectionpool.py", line 295, in _get_conn
    return conn or self._new_conn()
  File "/home/my_user/.local/share/virtualenvs/testproject-iixzfc0b/lib/python3.10/site-packages/forcediphttpsadapter/adapters.py", line 141, in _new_conn
    strict=self.strict, **conn_kw)
AttributeError: 'ForcedIPHTTPSConnectionPool' object has no attribute 'strict'

The problem is here: https://github.com/Roadmaster/forcediphttpsadapter/blob/master/forcediphttpsadapter/adapters.py#L141

And it is a non-existent attribute indeed.

Support proxies

The module works fine, but the proxies parameter is ignored and does not use proxies.

session = requests.Session()
session.mount(protocol_domain, ForcedIPHTTPSAdapter(dest_ip=ip))
r = session.get(address, verify=False, proxies=proxies)

Doesn't work. I changed to random dest_ip and got the same result (in python 2)

Doesn't work. I changed to random dest_ip and got the same result (in python 2)

Licence missing

Sorry for bothering you with such mundane things.

I found your code is really helping in tests (bypassing CloudFlare and having a direct request to the server), but I can't use it, because there is no license available (therefore, any use of it would be a copyright violation).

Can you, please, license it as MIT?

ForcedIPHTTPSAdapter is not pickleable

from forcediphttpsadapter.adapters import ForcedIPHTTPSAdapter
import pickle
adapter = ForcedIPHTTPSAdapter(dest_ip='127.0.0.1')
pickle.loads(pickle.dumps(adapter))

throws:
AttributeError: 'ForcedIPHTTPSAdapter' object has no attribute 'dest_ip'

The problem is that "dest_ip" field is not returned in __getstate__ method.

Have not a example how to import the package

By example:

>>> from forcediphttpsadapter import ForcedIPHTTPSAdapter
ImportError: cannot import name 'ForcedIPHTTPSAdapter' from 'forcediphttpsadapter'

In readme.md have not an example for this.

TypeError: <lambda>() got an unexpected keyword argument 'key_dest_ip'

I got the following backtrace when working with this module:

    response = session.get(conf.api_url + f"/show/{name}")
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 412, in send
    conn = self.get_connection(request.url, proxies)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 315, in get_connection
    conn = self.poolmanager.connection_from_url(url)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 298, in connection_from_url
    return self.connection_from_host(
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 245, in connection_from_host
    return self.connection_from_context(request_context)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 258, in connection_from_context
    pool_key = pool_key_constructor(request_context)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 124, in _default_key_normalizer
    return key_class(**context)
TypeError: <lambda>() got an unexpected keyword argument 'key_dest_ip'

I'm using it as described in the docs:

from forcediphttpsadapter.adapters import ForcedIPHTTPSAdapter                          
session.mount(f"https://{customfqdn}", ForcedIPHTTPSAdapter(dest_ip=customaddr))

The only thing that strikes me is that I'm reusing the same session for several requests, and the first works, but the second request I get the backtrace.