streams no longer available right the way

[madiele]

streams no longer available right the way

[madiele]

Thanks for the segnalation of the wrong env in the Readme, fixed it, the correct name is MP3_BITRATE, I will investigate the fact that it does not work for you.

For the 4 minutes delay I doubt it's just yt-dlp being slow for some reason, my instance does not have this problem, maybe youtube is breaking something, I'll give it a look when I have time though.

[madiele]

I'll write here a checklist of todo to improve logging to make troubleshooting envs and bottleneck problems easily.
Once done I'll ping you so you can try again with the new logs

• print all settings on startup (only show if set or unset for sensitive infos)
• print a log before each external dep. call (yt-dlp, apis, etc.)

@Eisaichen In the meanwhile does this issue happen only on 1.1.0 for you? If you use the precompiled container image you can try the previous version by changing latest to 1.0.5, in 1.1.0 I changed lots of the internals so I don't exclude a new bug from the rewrite

[Eisaichen]

I'll write here a checklist of todo to improve logging to make troubleshooting envs and bottleneck problems easily. Once done I'll ping you so you can try again with the new logs

• print all settings on startup (only show if set or unset for sensitive infos)
• print a log before each external dep. call (yt-dlp, apis, etc.)

@Eisaichen In the meanwhile does this issue happen only on 1.1.0 for you? If you use the precompiled container image you can try the previous version by changing latest to 1.0.5, in 1.1.0 I changed lots of the internals so I don't exclude a new bug from the rewrite

Thank you so much for the fast reply!
For the bitrate issue, I didn't change a thing, but it's somehow fixed now, sorry for bothering you. I don't know when the setting lost effect because I always set the bitrate to 320k in the past, I only changed it recently to troubleshoot the delay issue.

And about the delay issue, is complicated. v1.0.5 was running well, until a couple of months ago, it completely stopped working even though it's the same v1.0.5. errors with "URL parsing" alike. I didn't change anything because it was running like a charm before, and the docker image is also the same, so I think YouTube changed something and can only be fixed by update the components like yt-dlp.
later, I found you updated the docker-compose file in the master branch and released a v1.1.0 beta image. So I upgarde to v1.1.0 beta with the new podtube removed compose file. This fixes the previous issue and makes vod2pod usable again. but also introducing the delay issue. Now I'm running the image with latest tag and the issue persists.

Here are the logs and my config, please notice I set the log level to debug, so it's really long. (Client were Google Podcast Web)
logs.txt

config

      initContainers:
        - name: init-redis
          image: busybox:1.28
          command: ['sh', '-c', "until nc -z redis.core.svc.cluster.local 6379; do echo waiting for redis; sleep 2; done"]
      containers:
        - name: vod2pod
          image: docker.io/madiele/vod2pod-rss
          imagePullPolicy: "Always"
          ports:
            - containerPort: 8080
              name: http
          env:
            - name: TZ
              value: "America/Los_Angeles"
            - name: BITRATE
              value: "192"
            - name: TRANSCODE
              value: "true"
            - name: SUBFOLDER
              value: "/"
            - name: RUST_LOG
              value: "DEBUG"
            - name: REDIS_ADDRESS
              value: "redis.core.svc.cluster.local"
            - name: REDIS_PORT
              value: "6379"
            - name: YT_API_KEY
              valueFrom:
                secretKeyRef:
                  name: yt-api-key
                  key: key
           #  - name: TWITCH_SECRET
           #    value: "${TWITCH_SECRET:-}"
           #  - name: TWITCH_CLIENT_ID
           #    value: "${TWITCH_CLIENT_ID:-}"
      enableServiceLinks: false

Every headers from the Internet will be forwarded to vod2pod with the following begin added.

    forceSTSHeader: true
    stsIncludeSubdomains: false
    stsPreload: true
    stsSeconds: 31536000

[madiele]

Thanks for the debug logs! By checking the logs it does seem like the culprit is yt-dlp taking too long to fetch the stream url, to verify this can you try to lunch this command from inside the container by attaching to it using bash?

sudo docker exec -it <container> /bin/bash

The when inside do

yt-dlp -f bestaudio --get-url https://www.youtube.com/watch?v=BWcp9-4TLVQ

If this takes 4 minutes we have the problem, if it does try to see if removing -f or changing its value improves things (it's a pretty recent addition)

[Eisaichen]

Just tried, it will take 4-5 minutes to get the URL, and removing the -f flag didn't help. there no error occurred, just slow.
I also noticed that Metube also takes much longer to start a download mission than usual. so, probably Youtube did something?
Maybe we can add a new feather to automatically generate mp3 for the new feed and cache it, with some options to tweak that?

[madiele]

Unfortunately generating and caching the url ahead of time is not super clean as a solution, because the generated url has an expiry date, actually the generated url is already cached once generated for 5 hours, so if you try to query the same video again and it should not wait 4 minutes any more (if it does it's a bug).

A temporary workaround for you could be to write a side script to query the transcode url you need ahead of time to put them in the cache, probably chatgpt is able to code something like this pretty easily in python.

I'm open add a feature for caching ahead of time every x hours but before committing I would like to know if other people have the same issue as you, to make sure it's not something specific to your setup (mostly because it's harder for me to maintain feature that I personally don't use, so I want to be sure it's needed). I'll leave the issue open and see if someone check in.

In the meanwhile I suggest you to install yt-dlp on your personal pc or another server and check if it's still slow, maybe the ip you query youtube from is rate limited by youtube or something. Also I live in Italy so it's possible youtube is testing something in your region

[Eisaichen]

After spending a lot of hours on this today, I finally figured it out, It was ipv6. apparently, yt-dlp will try to contact YouTube with ipv6 first, but my environment didn't allow containers to use ipv6, so the yt-dlp wasted all the time waiting for the ipv6 connction time out. The problem was solved immediately after I permitted ipv6 traffic.
So sorry for letting you look into this, the problem is on my end after all.

[madiele]

Glad it was solved! Enjoy your podcasts ✌️