iv-org/smart-ipv6-rotator

IPv6 rotator for specific subnets - unblock restrictions on IPv6 enabled websites (Google by default but customizable for others)

smart-ipv6-rotator's Issues

BUG: host bits set

OS: Debian 12
Executed as root.

Debug info:

[INFO] You have IPv6 connectivity. Continuing.
[INFO] No cleanup of previous setup needed.
Traceback (most recent call last):
File "/home/smart-ipv6-rotator/smart-ipv6-rotator.py", line 310, in
SmartIPv6Rotator()
File "/home/smart-ipv6-rotator/smart-ipv6-rotator.py", line 53, in init
getattr(self, args.command)()
File "/home/smart-ipv6-rotator/smart-ipv6-rotator.py", line 183, in run
ipv6_network = IPv6Network(args.ipv6range)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/ipaddress.py", line 2227, in init
raise ValueError('%s has host bits set' % self)
ValueError: xxxx:xxxx:xxx:xx:xxxx:xxxx:xxxx:1/112 has host bits set

The --services option is useless

The README includes a --services option:

Define IPV6 ranges of popular services (e.g., --services google, twitter)

Problem is, twitter's IPv6 ranges are not hardcoded, despite the example. Actually, no service's IPv6 ranges are hardcoded, other than Google's.

can't start ipv6 rotator

hello, it's the first time that i use smart ipv6 rotator. I want to start for the first time th py fil but when i start the command i have this error :

Failed to remove the configured (Google) IPv6 subnets.
May be expected if the route were not yet configured and that was a cleanup due to an error.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work!
Your provider probably does not allow setting any arbitrary IPv6 address.
Or did you correctly configured the IPv6 subnet to use?
Exception:
HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x70a5dc55ca30>: Failed to establish a new connection: [Errno 113] No route to host'))

My internet provider is Free (french) and I don't know is this information is important

Is it an alternative to DynDNS/No-IP?

invidious: only rotate when needed

to make it less suspicious, only rotate when the ipv6 address is blocked.

VPS with IPv6 rotation support?

I tried to use smart-ipv6-rotator on the DigitalOcean droplet. But as it turned out it doesn't support it.

What should I pay attention to when choosing a VPS to support Ipv6 rotation? Can you suggest one?

FYI, I want to use smart-ipv6-rotator for live streams download via yt-dlp. Downloading regular videos works without any errors. but downloading live streams gives me HTTP Error 429:

[youtube] Extracting URL: https://www.youtube.com/live/4xDzrJKXOOY?si=CPtqhI-NFLyk6hXB
[youtube] 4xDzrJKXOOY: Downloading webpage
[youtube] 4xDzrJKXOOY: Downloading ios player API JSON
[youtube] 4xDzrJKXOOY: Downloading android player API JSON
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
[youtube] Extracting URL: https://www.youtube.com/watch?v=4xDzrJKXOOY
[youtube] 4xDzrJKXOOY: Downloading webpage
[youtube] 4xDzrJKXOOY: Downloading ios player API JSON
[youtube] 4xDzrJKXOOY: Downloading android player API JSON
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
[youtube] 4xDzrJKXOOY: Downloading m3u8 information
WARNING: [youtube] Failed to download m3u8 information: HTTP Error 429: Too Many Requests
ERROR: [youtube] 4xDzrJKXOOY: No video formats found!; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

Script do not work in docker environment

Hi, I am using the docker image and ran docker run --rm --privileged --name smart-ipv6-rotator --network host quay.io/invidious/smart-ipv6-rotator:latest run --ipv6range=<IPV6RANGE>/64. It ran successfully the first time, but showed the following error subsequently. Is this error related to my network or to the docker image? Thank you.

[DEBUG] Debug info:
random_ipv6_address --> <Redacted>
random_ipv6_address_mask --> 64
gateway --> <Redacted>
interface_index --> 2
interface_name --> <Redacted>
ipv6_subnet --> <Redacted>
    ret = self.nlm_request(
          ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py", line 870, in nlm_request
    return tuple(self._genlm_request(*argv, **kwarg))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py", line 1214, in nlm_request
    for msg in self.get(
               ^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py", line 873, in get
    return tuple(self._genlm_get(*argv, **kwarg))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py", line 550, in get
    raise msg['header']['error']
pyroute2.netlink.exceptions.NetlinkError: (17, 'File exists')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/smart-ipv6-rotator.py", line 310, in <module>
    SmartIPv6Rotator()
  File "/app/smart-ipv6-rotator.py", line 53, in __init__
    getattr(self, args.command)()
  File "/app/smart-ipv6-rotator.py", line 239, in run
    self.clean_previous_setup(memory_settings)
TypeError: SmartIPv6Rotator.clean_previous_setup() missing 1 required positional argument: 'args'

Incompatible with Python < 3.9

When running with python<3.9 The following issue occurs (Tested with py3.7, debian):
AttributeError: module 'argparse' has no attribute 'BooleanOptionalAction'
This appears to be a Python version Issue.

Where is the subnet?

I'm following the guide and sorry about my ignorance but where is subnet on Subnet Calculator?

I see a lot of links and numbers but no one is specifically tagged like subnet and I'm new on IPV6. So, what's the value of the subnet on the detailed table?

Can anyone show an example of subnet on IPV6, at least?

Could you clarify the in the README if it acts on the host or the container please?

I was just about to deploy when I realized this might be changing the server's network, I'm not sure though.

I modified my compose to specify specific macvlan subnet and addresses so I can tell the traffic apart from the server's which itself isn't allowed to connect to the Internet.

# invidious service
    networks:
      z0a00:
        ipv4_address: 10.10.0.48
# db service:
    networks:
      z0a00:
        ipv4_address: 10.10.0.49
# network defs:
networks:
  z0a00:
    external: true

I adjusted the last part as:

networks:
  z0a00:
    external: true
    enable_ipv6: true
      ipam:
        config:
          - subnet: 2001:0DB9::/112
            gateway: 2001:0DB9::1

I assumed the address is just a placeholder given the range, but otherwise I don't know if that's correct. I've set up macvlan on compose only once before…in this same config. :/

The original code was:

networks:
  default:
    enable_ipv6: true
      ipam:
        config:
          - subnet: 2001:0DB9::/112
            gateway: 2001:0DB9::1

If I remember correctly — it's been a while since I switched it and I normally don't use compose if I can avoid it — that's (default:) when it shares the network with the host, ain't it? Therefore it would be changing the host's network, not the container's network, it seems. The host has about six other networks, all dual stack; this is the least preferred. It will get very messy if it does.

Thanks!

Rotate only when blocked

Avoid rotating too much, for maybe avoiding YouTube detecting a frequent ipv6 rotation.

For detecting the block:

• periodically try to load a video using /latest_version
• use the new PR: 4085

BUG: Randomly fails

The script (executed as root) randomly fails. If I manually execute it again after if failed (or during next cron execution) it seems to work again.

Debug output:

[INFO] You have IPv6 connectivity. Continuing.
[INFO] Finished cleaning up previous setup. [INFO] Waiting for the propagation in the Linux kernel.
[DEBUG] Debug info:
random_ipv6_address --> 2a01:4f8:200:33ee:ac3c:270:da01:29ae
random_ipv6_address_mask --> 64
gateway --> fe80::1
interface_index --> 2
interface_name --> eno1
ipv6_subnet --> 2a01:4f8:200:33ee::/64
[Error] Failed to remove the test IPv6 subnet.
May be expected if the route were not yet configured and that was a cleanup due to an error.
[Error] Failed to remove the configured (Google) IPv6 subnets.
May be expected if the route were not yet configured and that was a cleanup due to an error.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[Error] Failed to configure the test IPv6 route. The setup did not work!
Exception:
(22, 'Invalid argument')

Next execution:

[INFO] You have IPv6 connectivity. Continuing.
[INFO] No cleanup of previous setup needed.
[DEBUG] Debug info:
random_ipv6_address --> 2a01:4f8:200:33ee:f769:b034:af7b:8fd1
random_ipv6_address_mask --> 64
gateway --> fe80::1
interface_index --> 2
interface_name --> eno1
ipv6_subnet --> 2a01:4f8:200:33ee::/64
[INFO] Correctly using the new random IPv6 address, continuing.
[INFO] Correctly configured the IPv6 routes for Google IPv6 ranges.
[INFO] Successful setup. Waiting for the propagation in the Linux kernel.

Notice: Often only works after container restarts

This is not a bug but something I noticed during the past days.
On my server, the rotation (and therefore the unblocking) only really takes effect after a container restart.
So, simply running the ipv6rotator will not (immediately) unblock the instance.
When I run ipv6rotate (my wrapper for /srv/ipv6rotate/smart-ipv6-rotator.py run --ipv6range=***) the blocking persists.
However, as soon as one of the instances restarts (during the hourly restart cron) it is unblocked.
And if i manually run ipv6rotate && systemctl restart [email protected] all of them restart and are unblocked.

@unixfox Do you have some advice whats the best practise at this point?
Should I just cron ipv6rotate and the restart of ALL containers hourly, together?

script often fails with error

Hi. Script often fails for me:

[INFO] You have IPv6 connectivity. Continuing.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[DEBUG] Debug info:
random_ipv6_address --> XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX 
random_ipv6_address_mask --> 64
gateway --> fe80::1
interface_index --> 2
interface_name --> eth0
ipv6_subnet --> XXXX:XXXX:XXXX:XXXX::/64
[Error] Failed to remove the configured (Google) IPv6 subnets.
        May be expected if the route were not yet configured and that was a cleanup due to an error.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work!
        Your provider probably does not allow setting any arbitrary IPv6 address.
        Or did you correctly configured the IPv6 subnet to use?
       Exception:
HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fe489f6b010>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

However, sometimes (rarely) it works without errors. New IP+route is then listed ( ip a and ip r )
I have not yet been able to determine any regularity.

Machine is a root server at netcup.

EDIT:
Debian12.4 x64
Kernel 6.1.0-17-amd64
Nothing special

Thanks for advices

Debian 11 & 12 not working

/usr/local/lib/python3.11/dist-packages/requests/init.py:102: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (5.1.0)/charset_normalizer (2.0.12) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
[INFO] You have IPv6 connectivity. Continuing.
[INFO] No cleanup of previous setup needed.
Traceback (most recent call last):
File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 310, in
SmartIPv6Rotator()
File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 53, in init
getattr(self, args.command)()
File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 183, in run
ipv6_network = IPv6Network(args.ipv6range)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/ipaddress.py", line 2227, in init
raise ValueError('%s has host bits set' % self)
ValueError: myaddress:1/64 has host bits set

..........

.....

Help Needed Configuring the IPv6 Rotator

Hello there.

First of all, apologies for opening an issue, but I don't have Matrix or IRC.

I think I followed the IPv6 rotator instructions correctly but apparently I must have done something wrong, because while I can issue the IPv6 check command on the Indivious container, the ping command always fails, and I cannot access the external URL for the service (using a Cloudflare tunnel).

I have:

• Added "experimental" and "ip6tables" to the Docker daemon json, and restarted it;
• Added "force_resolve: ipv6" to the Indivious Docker Compose file (under INVIDIOUS_CONFIG, along with the rest of the database, hmac key, and other options);
• Added the network configuration section to the bottom of my Docker Compose file;
• Confirmed via "docker exec indivious ip a" that the container does indeed have an IPv6 address on the 2001:0DB9::/112 subnet (cannot confirm if the gateway is responding or not, though);
• Run "ip a" on the Docker host machine, to get its IPv6 address, plugged it into the IPv6 calculator to get my subnet, and ran the rotator script apparently successfully (also added to crontab), since it now also contains an IPv6 address on the same subnet as the rest of the IPv6 addresses of that machine, only with a "forever" duration;
• Confirmed the host machine still has IPv6 connectivity. However, running the "curl -m 5 ipv6.icanhazip.com" command returns my ISP-assigned IPv6 address, and not the new one created by the rotator script (not sure if relevant).

I would like to know what I might be missing, please.

For completeness, here's a redacted copy of the Docker Compose file:

services:

  invidious:
    container_name: indivious
    image: quay.io/invidious/invidious:latest
    # image: quay.io/invidious/invidious:latest-arm64 # ARM64/AArch64 devices
    restart: unless-stopped
    ports:
      #- "127.0.0.1:30000:3000"
      - "30000:3000"
    environment:
      #- TZ=Europe/Lisbon
      # Please read the following file for a comprehensive list of all available
      # configuration options and their associated syntax:
      # https://github.com/iv-org/invidious/blob/master/config/config.example.yml
      INVIDIOUS_CONFIG: |
        db:
          dbname: invidious
          user: [redacted]
          password: [redacted]
          host: invidious-db
          port: 5432
        check_tables: true
        # external_port:
        domain: [redacted]
        https_only: true
        # statistics_enabled: false
        hmac_key: "[redacted]"
        registration_enabled: false
        force_resolve: ipv6
    healthcheck:
      test: wget -nv --tries=1 --spider http:127.0.0.1:3000/api/v1/comments/jNQXAC9IVRw || exit 1
      interval: 30s
      timeout: 5s
      retries: 2
    logging:
      options:
        max-size: "1G"
        max-file: "4"
    depends_on:
      - invidious-db
    networks:
      - default

  invidious-db:
    container_name: indivious-db
    image: docker.io/library/postgres:14
    restart: unless-stopped
    networks:
      - default
    volumes:
      - [redacted]:/var/lib/postgresql/data
      - [redacted]:/config/sql
      - [redacted]:/docker-entrypoint-initdb.d/init-invidious-db.sh
    environment:
      #- TZ=[redacted]
      POSTGRES_DB: invidious
      POSTGRES_USER: [redacted]
      POSTGRES_PASSWORD: [redacted]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]

networks:
  default:
    enable_ipv6: true
    ipam:
      config:
        - subnet: 2001:0DB9::/112
          gateway: 2001:0DB9::1

Lastly, not sure if relevant, but I have noticed that each time I restart the service, the db container complains with the line "invalid record length at 0/11BACF98: wanted 24, got 0".

Thank you in advance for any help you might be able to give me.

IP rotation?

I setupped this up and everything worked fine, but all requests for yt-dlp are showing me the same ipv6 address, does this works for invidious only?

python3.12 smart-ipv6-rotator.py run --ipv6range=XXXX:XXXX:XXX:2::/64
[INFO] You have IPv6 connectivity. Continuing.
[INFO] No cleanup of previous setup needed.
[DEBUG] Debug info:
ranges --> ['XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX:/36', 'XXXX:XXXX:XXX:/36', 'XXXX:XXXX:XXX::/36']
random_ipv6_address --> XXXX:XXXX:XXX:2:33c9:910a:5bf:4ffc
gateway --> XXXX:XXXX:XXX:1::11
interface_index --> 2
interface_name --> enp129s0f0
ipv6_subnet --> XXXX:XXXX:XXX:2::/64
random_ipv6_address_mask --> 64
[INFO] Correctly using the new random IPv6 address, continuing.
[INFO] Correctly configured the IPv6 routes for IPv6 ranges ['XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX::/36', 'XXXX:XXXX:XXX:/36', 'XXXX:XXXX:XXX:/36', 'XXXX:XXXX:XXX::/36'].
[INFO] Successful setup. Waiting for the propagation in the Linux kernel.

Set the new ip address as not preferred

Use the ability to set preferred_lft 0 for the ipv6 address in order to not having it a the default source address: https://www.davidc.net/networking/ipv6-source-address-selection-linux