origamiofficial/rclone-http-updater

A Python script that updates rclone HTTP Remote configuration file (rclone.conf) with the latest URLs from specified websites using XPath, and notifies via Telegram if changes are detected.

Rclone HTTP Remote URL Updater

A Python script that updates rclone HTTP Remote configuration file (rclone.conf) with the latest URLs from specified websites using XPath, and notifies via Telegram if changes are detected.

Features

• Supports multiple websites as a fallback.
• Automatically checks if the website is up and running.
• Verifies the correctness of XPath expressions used to extract data from the website.
• Stores information about the website links in an SQLite database to track updates.
• Updates the rclone configuration file (rclone.conf) with the latest URLs from the website.
• Executes customizable post-update command only after the rclone.conf updates.
• Notifies specified Telegram channel about updates using the Telegram Bot API.

Requirements

• Python 3.6 or higher
• requests library
• lxml library
• sqlite3 library
• TELEGRAM_CHAT_ID and TELEGRAM_BOT_API_KEY environment variables with valid values

Usage

1. Clone or download this repository.
2. Install the required libraries by running the following command:

pip install -r requirements.txt

3. Set the WEBSITE_URLS environment variable with the website/s you want to fetch.
4. Set the TELEGRAM_CHAT_ID and TELEGRAM_BOT_API_KEY environment variables with your Telegram chat ID and bot API key.
5. Set the POST_COMMAND with the command you want to execute only after update.
6. Change the name_mappings according to your setup

"Website Hypertext": "Rclone Remote Name",

7. Run the script using the following command:

python main.py

Contribution

If the website administrators make changes and break things, you may need to update the XPath expressions. Contributions in the form of pull requests are welcome. Remember that you don't need to update the script version if you make changes; the script will automatically update itself.

How it works

The Rclone HTTP Remote URL Updater script is written in Python and uses various libraries to perform its tasks. It utilizes the requests library to fetch the website, and the lxml library to parse the HTML on the page and extract relevant information using specified XPath values. The script connects to an SQLite database to check if a link is already in the database. If yes, it compares the old link with the new link and updates the link in the database, then updates the rclone configuration file (rclone.conf) with the new URL. It sends notifications to a Telegram channel using the Telegram Bot API with the information about the updated URL. The script uses the TELEGRAM_CHAT_ID and TELEGRAM_BOT_API_KEY environment variables to send notifications to the Telegram chat.

Credit

Everything in this repo developed using natural language processing capabilities from OpenAI's GPT-3.5

rclone-http-updater's Issues

requests-2.31.0-py3-none-any.whl: 1 vulnerabilities (highest severity is: 5.6)

Vulnerable Library - requests-2.31.0-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/samftp-rclone-updater/requirements.txt

Path to vulnerable library: /tmp/ws-scm/samftp-rclone-updater/requirements.txt

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (requests version)	Remediation Possible**
CVE-2024-35195	Medium	5.6	requests-2.31.0-py3-none-any.whl	Direct	requests - 2.32.0	❌

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2024-35195

Vulnerable Library - requests-2.31.0-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl

Path to dependency file: /tmp/ws-scm/samftp-rclone-updater/requirements.txt

Path to vulnerable library: /tmp/ws-scm/samftp-rclone-updater/requirements.txt

Dependency Hierarchy:

• ❌ requests-2.31.0-py3-none-any.whl (Vulnerable Library)

Found in base branch: main

Vulnerability Details

Requests is a HTTP library. Prior to 2.32.0, when making requests through a Requests Session, if the first request is made with verify=False to disable cert verification, all subsequent requests to the same host will continue to ignore cert verification regardless of changes to the value of verify. This behavior will continue for the lifecycle of the connection in the connection pool. This vulnerability is fixed in 2.32.0.

Publish Date: 2024-05-20

URL: CVE-2024-35195

CVSS 3 Score Details (5.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: High
  • Privileges Required: High
  • User Interaction: Required
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-9wx4-h78v-vm56

Release Date: 2024-05-20

Fix Resolution: requests - 2.32.0

Step up your Open Source Security Game with Mend here