Looking for an efficient way to monitor for download folder changes

[opensuse]

Looking for an efficient way to monitor for download folder changes

[andrii-suse]

So far it works only for *Current.iso, but I can add support for the same logic for GNOME*.iso , e.g.

curl -sI https://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-DVD-x86_64-Current.iso | grep -i location
location: https://download.opensuse.org/tumbleweed/iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20230129-Media.iso

[andrii-suse]

Maybe using json output will resolve the issue?
https://download.opensuse.org/repositories/GNOME:/Medias/images/iso/?P=GNOME_Next*&json

[okurz]

I wasn't aware how to find JSON routes. This looks helpful however the content is bigger than the checksum file so it would be less efficient to fetch that and compare in content. Also the "last-modified" approach would be more efficient

[andrii-suse]

So what last-modified should refer to when using pattern or regex for file name? The max mtime of files that match the pattern / regex?

[okurz]

So what last-modified should refer to when using pattern or regex for file name? The max mtime of files that match the pattern / regex?

I guess it could simply be the same "last-modified" of the complete page regardless of the filtering. So it would be like the "minimum mtime". The filtering would be applied on top.

[andrii-suse]

But then it will not be usable for monitoring for changes with file filter, because a change in the mtime may be related to some other files. Do I get it right?

[andrii-suse]

Another way may be to check for header response. For now it works only for mirrorcache.o.o , but I probably need to fix it for download.o.o as well the same way:

curl -Is https://mirrorcache.opensuse.org/repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64.iso | grep location
location: /repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64-43.2-Build23.9.iso

[andrii-suse]

FYI GNOME_Next now redirects to particular build - this may be good way to track changes:

curl -Is https://download.opensuse.org/repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64.iso | grep -I location
location: /repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64-43.2-Build23.34.iso

We can close the ticket or let it wait til there will be priority to provide something like mtime in response header.

[okurz]

That looks good. But I don't think I can instruct jenkins to read just the head and the always-changing "csrf-token" as described in the original description still seems to be problematic.