brainlabs-digital/polly

Polly - rel-alternate-hreflang parser and validator

polly's Issues

Relative URLs not allowed in hreflang annotations

Hey there, first of all: thanks for sharing polly as open-source software 👍! I discovered these lines in your code:

https://github.com/DistilledLtd/polly/blob/8f506c6fd108aaff9f47f4762f429afdf036b89c/polly/polly.py#L196-L198

However, https://support.google.com/webmasters/answer/189077?hl=en clearly states this:

Alternate URLs must be fully-qualified, including the transport method (http/https), so:
https://example.com/foo, not //example.com/foo or /foo

The article linked in the code comment is about canonical annotations, not hreflang annotations. My suggestion would be to ditch the relative URL resolving and report an error when a hreflang contains a relative URL.

requests-2.5.3-py2.py3-none-any.whl: 4 vulnerabilities (highest severity is: 7.5)

Vulnerable Library - requests-2.5.3-py2.py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/95/54/44dc83b5f11c6da06bf9abd18c8a0905e0e297e0a9c3bfbc0c6ee4bdd33d/requests-2.5.3-py2.py3-none-any.whl

Path to dependency file: /requirements.txt

Path to vulnerable library: /requirements.txt

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (requests version)	Remediation Possible**
CVE-2018-18074	High	7.5	requests-2.5.3-py2.py3-none-any.whl	Direct	2.20.0	✅
CVE-2023-32681	Medium	6.1	requests-2.5.3-py2.py3-none-any.whl	Direct	2.32.0	✅
CVE-2024-35195	Medium	5.6	requests-2.5.3-py2.py3-none-any.whl	Direct	2.32.0	✅
CVE-2015-2296	Low	3.7	requests-2.5.3-py2.py3-none-any.whl	Direct	2.6.0	✅

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2018-18074

Vulnerable Library - requests-2.5.3-py2.py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/95/54/44dc83b5f11c6da06bf9abd18c8a0905e0e297e0a9c3bfbc0c6ee4bdd33d/requests-2.5.3-py2.py3-none-any.whl

Path to dependency file: /requirements.txt

Path to vulnerable library: /requirements.txt

Dependency Hierarchy:

• ❌ requests-2.5.3-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

The Requests package before 2.20.0 for Python sends an HTTP Authorization header to an http URI upon receiving a same-hostname https-to-http redirect, which makes it easier for remote attackers to discover credentials by sniffing the network.

Publish Date: 2018-10-09

URL: CVE-2018-18074

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://nvd.nist.gov/vuln/detail/CVE-2018-18074

Release Date: 2018-10-09

Fix Resolution: 2.20.0

⛑️ Automatic Remediation will be attempted for this issue.

CVE-2023-32681

Vulnerable Library - requests-2.5.3-py2.py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/95/54/44dc83b5f11c6da06bf9abd18c8a0905e0e297e0a9c3bfbc0c6ee4bdd33d/requests-2.5.3-py2.py3-none-any.whl

Path to dependency file: /requirements.txt

Path to vulnerable library: /requirements.txt

Dependency Hierarchy:

• ❌ requests-2.5.3-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

Requests is a HTTP library. Since Requests 2.3.0, Requests has been leaking Proxy-Authorization headers to destination servers when redirected to an HTTPS endpoint. This is a product of how we use rebuild_proxies to reattach the Proxy-Authorization header to requests. For HTTP connections sent through the tunnel, the proxy will identify the header in the request itself and remove it prior to forwarding to the destination server. However when sent over HTTPS, the Proxy-Authorization header must be sent in the CONNECT request as the proxy has no visibility into the tunneled request. This results in Requests forwarding proxy credentials to the destination server unintentionally, allowing a malicious actor to potentially exfiltrate sensitive information. This issue has been patched in version 2.31.0.

Publish Date: 2023-05-26

URL: CVE-2023-32681

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-j8r2-6x86-q33q

Release Date: 2023-05-26

Fix Resolution: 2.32.0

⛑️ Automatic Remediation will be attempted for this issue.

CVE-2024-35195

Vulnerable Library - requests-2.5.3-py2.py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/95/54/44dc83b5f11c6da06bf9abd18c8a0905e0e297e0a9c3bfbc0c6ee4bdd33d/requests-2.5.3-py2.py3-none-any.whl

Path to dependency file: /requirements.txt

Path to vulnerable library: /requirements.txt

Dependency Hierarchy:

• ❌ requests-2.5.3-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

Requests is a HTTP library. Prior to 2.32.2, when making requests through a Requests Session, if the first request is made with verify=False to disable cert verification, all subsequent requests to the same host will continue to ignore cert verification regardless of changes to the value of verify. This behavior will continue for the lifecycle of the connection in the connection pool. This vulnerability is fixed in 2.32.2.

Publish Date: 2024-05-20

URL: CVE-2024-35195

CVSS 3 Score Details (5.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: High
  • Privileges Required: High
  • User Interaction: Required
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-9wx4-h78v-vm56

Release Date: 2024-05-20

Fix Resolution: 2.32.0

⛑️ Automatic Remediation will be attempted for this issue.

CVE-2015-2296

Vulnerable Library - requests-2.5.3-py2.py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/95/54/44dc83b5f11c6da06bf9abd18c8a0905e0e297e0a9c3bfbc0c6ee4bdd33d/requests-2.5.3-py2.py3-none-any.whl

Path to dependency file: /requirements.txt

Path to vulnerable library: /requirements.txt

Dependency Hierarchy:

• ❌ requests-2.5.3-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

The resolve_redirects function in sessions.py in requests 2.1.0 through 2.5.3 allows remote attackers to conduct session fixation attacks via a cookie without a host value in a redirect.

Publish Date: 2015-03-18

URL: CVE-2015-2296

CVSS 3 Score Details (3.7)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-2296

Release Date: 2015-03-18

Fix Resolution: 2.6.0

⛑️ Automatic Remediation will be attempted for this issue.

---

⛑️Automatic Remediation will be attempted for this issue.

Bug in hreflang-check.py

I checked out polly and while testing it as specified in the readme I got the following error:

Errors by hreflang_key:
Traceback (most recent call last):
  File "polly/hreflang-check.py", line 44, in <module>
    print "\t" + hreflang_key + " = " + str(test_page.errors_for_key[hreflang_key])
AttributeError: 'PollyPage' object has no attribute 'errors_for_key'

requests-2.31.0-py3-none-any.whl: 1 vulnerabilities (highest severity is: 5.6)

Vulnerable Library - requests-2.31.0-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (requests version)	Remediation Possible**
CVE-2024-35195	Medium	5.6	requests-2.31.0-py3-none-any.whl	Direct	requests - 2.32.2	✅

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2024-35195

Vulnerable Library - requests-2.31.0-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Dependency Hierarchy:

• ❌ requests-2.31.0-py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

Requests is a HTTP library. Prior to 2.32.2, when making requests through a Requests Session, if the first request is made with verify=False to disable cert verification, all subsequent requests to the same host will continue to ignore cert verification regardless of changes to the value of verify. This behavior will continue for the lifecycle of the connection in the connection pool. This vulnerability is fixed in 2.32.2.

Publish Date: 2024-05-20

URL: CVE-2024-35195

CVSS 3 Score Details (5.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: High
  • Privileges Required: High
  • User Interaction: Required
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-9wx4-h78v-vm56

Release Date: 2024-05-20

Fix Resolution: requests - 2.32.2

⛑️ Automatic Remediation will be attempted for this issue.

---

⛑️Automatic Remediation will be attempted for this issue.

cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl: 1 vulnerabilities (highest severity is: 7.5)

Vulnerable Library - cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl

cryptography is a package which provides cryptographic recipes and primitives to Python developers.

Library home page: https://files.pythonhosted.org/packages/f6/79/227c6f7e98657cf9387d5797d56e983165f294ed838679b2b8ca12118e18/cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (cryptography version)	Remediation Possible**
CVE-2024-26130	High	7.5	cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl	Direct	42.0.4	✅

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2024-26130

Vulnerable Library - cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl

cryptography is a package which provides cryptographic recipes and primitives to Python developers.

Library home page: https://files.pythonhosted.org/packages/f6/79/227c6f7e98657cf9387d5797d56e983165f294ed838679b2b8ca12118e18/cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Dependency Hierarchy:

• ❌ cryptography-42.0.1-cp37-abi3-manylinux_2_28_x86_64.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

cryptography is a package designed to expose cryptographic primitives and recipes to Python developers. Starting in version 38.0.0 and prior to version 42.0.4, if pkcs12.serialize_key_and_certificates is called with both a certificate whose public key did not match the provided private key and an encryption_algorithm with hmac_hash set (via PrivateFormat.PKCS12.encryption_builder().hmac_hash(...), then a NULL pointer dereference would occur, crashing the Python process. This has been resolved in version 42.0.4, the first version in which a ValueError is properly raised.

Publish Date: 2024-02-21

URL: CVE-2024-26130

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-6vqw-3v5j-54x4

Release Date: 2024-02-21

Fix Resolution: 42.0.4

⛑️ Automatic Remediation will be attempted for this issue.

---

⛑️Automatic Remediation will be attempted for this issue.

urllib3-2.0.7-py3-none-any.whl: 1 vulnerabilities (highest severity is: 4.4)

Vulnerable Library - urllib3-2.0.7-py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/d2/b2/b157855192a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be524/urllib3-2.0.7-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (urllib3 version)	Remediation Possible**
CVE-2024-37891	Medium	4.4	urllib3-2.0.7-py3-none-any.whl	Direct	2.2.2	✅

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2024-37891

Vulnerable Library - urllib3-2.0.7-py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/d2/b2/b157855192a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be524/urllib3-2.0.7-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/polly

Path to vulnerable library: /tmp/ws-scm/polly

Dependency Hierarchy:

• ❌ urllib3-2.0.7-py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 5968a85eb652bf2c260b53d2624c1f48f0f0876b

Found in base branch: master

Vulnerability Details

urllib3 is a user-friendly HTTP client library for Python. When using urllib3's proxy support with ProxyManager, the Proxy-Authorization header is only sent to the configured proxy, as expected. However, when sending HTTP requests without using urllib3's proxy support, it's possible to accidentally configure the Proxy-Authorization header even though it won't have any effect as the request is not using a forwarding proxy or a tunneling proxy. In those cases, urllib3 doesn't treat the Proxy-Authorization HTTP header as one carrying authentication material and thus doesn't strip the header on cross-origin redirects. Because this is a highly unlikely scenario, we believe the severity of this vulnerability is low for almost all users. Out of an abundance of caution urllib3 will automatically strip the Proxy-Authorization header during cross-origin redirects to avoid the small chance that users are doing this on accident. Users should use urllib3's proxy support or disable automatic redirects to achieve safe processing of the Proxy-Authorization header, but we still decided to strip the header by default in order to further protect users who aren't using the correct approach. We believe the number of usages affected by this advisory is low. It requires all of the following to be true to be exploited: 1. Setting the Proxy-Authorization header without using urllib3's built-in proxy support. 2. Not disabling HTTP redirects. 3. Either not using an HTTPS origin server or for the proxy or target origin to redirect to a malicious origin. Users are advised to update to either version 1.26.19 or version 2.2.2. Users unable to upgrade may use the Proxy-Authorization header with urllib3's ProxyManager, disable HTTP redirects using redirects=False when sending requests, or not user the Proxy-Authorization header as mitigations.

Publish Date: 2024-06-17

URL: CVE-2024-37891

CVSS 3 Score Details (4.4)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: High
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-34jh-p97f-mpxf

Release Date: 2024-06-17

Fix Resolution: 2.2.2

⛑️ Automatic Remediation will be attempted for this issue.

---

⛑️Automatic Remediation will be attempted for this issue.

Bug in Validation if the HTTP Response code of initial URL is other than 200

I Checked polly to run URL's ( Example site : https://www.jobstreet.com/ ) whose HTTP response is 301 & 302 and getting below error

  File "hreflang-check.py", line 7, in <module>
    test_page = PollyPage(initial_url, allow_underscore=True)
  File "C:\Users\mounicm\Documents\polly\polly.py", line 43, in __init__
    self.fetch_page()
  File "C:\Users\mounicm\Documents\polly\polly.py", line 127, in fetch_page
    raise ValueError("HTTP Response Code was not 200.")
ValueError: HTTP Response Code was not 200.```

 - Curious to know if it can process the final redirected URL