msys2/msys2-web

Online Repo Browser

MSYS2 GitHub homepage

Users

Start on the website, there's news and documentation.

Developers

See the repositories on the org page.

Issues

We expect your reports in MINGW packages or MSYS packages, depending on which repository the affected software comes from.

msys2-web's Issues

gentoo mapping/version issues

We currently have no keywords or masks for the packages, there is a graphql API but it has no pagination and times out if not just one package is queried:

import requests
import json
import pprint

graphql_url = 'https://packages.gentoo.org/api/graphql/'
query = '''
{
  packages(Atom: "app-editors/emacs") {
    Versions {
        Version
        Keywords
    }
  }
}
'''
headers = {
    'Content-Type': 'application/json',
    'Accept': 'application/json',
}
payload = json.dumps({'query': query})
response = requests.post(graphql_url, headers=headers, data=payload)
response.raise_for_status()
pprint.pprint(response.json(), width=120)

{'data': {'packages': [{'Versions': [{'Keywords': '', 'Version': '30.0.9999'},
                                     {'Keywords': '~alpha amd64 arm arm64 ~hppa ~ia64 ~m68k ~mips ppc ppc64 ~riscv '
                                                  '~sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '29.1-r6'},
                                     {'Keywords': '~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 '
                                                  '~riscv ~sparc ~x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '29.1.90'},
                                     {'Keywords': '~alpha amd64 arm arm64 hppa ~ia64 ~m68k ~mips ppc ppc64 sparc x86 '
                                                  '~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '25.3-r19'},
                                     {'Keywords': '~alpha amd64 arm arm64 hppa ~ia64 ~mips ppc ppc64 ~riscv sparc x86 '
                                                  '~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '26.3-r16'},
                                     {'Keywords': '', 'Version': '29.1.9999'},
                                     {'Keywords': '~alpha amd64 arm arm64 hppa ~ia64 ~m68k ~mips ppc ppc64 ~riscv '
                                                  'sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '29.1-r1'},
                                     {'Keywords': '~alpha amd64 arm arm64 hppa ~ia64 ~m68k ~mips ppc ppc64 ~riscv '
                                                  'sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '28.2-r10'},
                                     {'Keywords': '~alpha amd64 arm arm64 hppa ~ia64 ~m68k ~mips ppc ppc64 ~riscv '
                                                  'sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '27.2-r14'},
                                     {'Keywords': '~alpha amd64 arm arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~riscv '
                                                  'sparc x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos',
                                      'Version': '29.1-r5'},
                                     {'Keywords': 'amd64 x86', 'Version': '18.59-r15'}]}]}}

Parse PKGBUILD files in MSYS2-packages/MINGW-packages directly

Instead of doing it in this repo we could parse them directly with a github workflow on the master branch. The work involved is:

• Have a special git tag holding the parsed PKGBUILD cache
• Pull the cache
• Check if all PKGBUILDs in git are int he cache and update if needed
• Replace the cache

This would remove the webhook dance we currently have to make with appveyor.

Add max-height to all dependency lists

max-height: 500px; overflow:auto or something, so ling lists don't take over the whole page.

package removal list needs to check reverse deps

A package can only be removed if nothing in the repo depends on it anymore.

This also includes if it provides something and there is no real package with that name, and that provides is used somewhere.

Add logging

It sometimes stops updating, I should add logging, or add sentry.

queue: sort packges build PKGBUILD last modified date

It seems nicer to have recently edited packages on top, as they are probably the most awaited.

Compare versions with cygwin

We currently compare package versions with Arch to decide if the package is outdated.

Since we also have various packages which only exist in cygwin we should generalize this and compare with both Arch and Cygwin.

Make everything makedepend on base/base-devel

When figuring out the build order and dependencies. So that autobuild builds everything in base/base-devel first and also pulls it in if needed.

ntldd: "A newer upstream version () is available"

for ntldd page this label is showed somehow

Buildqueue API

Bugs:

• https://packages.msys2.org/api/buildqueue?include_new=true&include_update=false
  new PKGBUILDS only contain one package
• PKGBUILDS depending/makedepending on new packages should be skipped if include_new=false just let it fail at build time
• include package relations -> done

missing reverse deps for provides

http://quodlibet.duckdns.org/msys2/package/mingw-w64-i686-x264-git

it should show the reverse deps of x264

file search

check if the server can handle it

handling of missing packages

http://quodlibet.duckdns.org/msys2/package/msys2-runtime?repo=msys&variant=x86_64

catgets, which is no longer in the repo, shows up in Conflicts but not in Replaces. It should even if the link leads to nothing.

Remove msys32

Move more dependency handling into the API

The current API response is (it grew over time..):

{
    "repo_url": "https://github.com/msys2/MINGW-packages",
    "repo_path": "mingw-w64-mono",
    "version": "6.4.0.198-1",
    "name": "mingw-w64-mono",
    "source": false,
    "packages": {
        "ucrt64": [
            "mingw-w64-ucrt-x86_64-mono"
        ],
        "clang64": [
            "mingw-w64-clang-x86_64-mono"
        ]
    },
    "new": [
        "ucrt64",
        "clang64"
    ],
    "depends": {
        "msys": [
            "libedit",
            "perl-libwww"
        ]
    }
}

and we hardcode in autobuild that ucrt64 depends on msys etc.

Ideally the response would be this, so all the info is neatly separated by build type:

{
    "repo_url": "https://github.com/msys2/MINGW-packages",
    "repo_path": "mingw-w64-mono",
    "version": "6.4.0.198-1",
    "name": "mingw-w64-mono",
    "source": false,
    "builds": {
        "ucrt64": {
            "packages": [
                "mingw-w64-ucrt-x86_64-mono"
            ],
            "new": true,
            "depends": {
                "msys": [
                    "libedit",
                    "perl-libwww"
                ]
            }
        },
        "clang64": {
            "packages": [
                "mingw-w64-clang-x86_64-mono"
            ],
            "new": true,
            "depends": {
                "msys": [
                    "libedit",
                    "perl-libwww"
                ]
            }
        }
    }
}

Move all backend parsing off the main thread

it's blocking requests.

Affects everything in fetch.py basically.

fake "base"-groups

similar to pkgbase we could fake a base group name in the UI. Every group showing up multiple times is starting to get confusing.

See if we can include any signature information

It would be nice to see who (or which key) built which package.

group list bugged

http://quodlibet.duckdns.org/msys2/base/gdb is in the base-devel group, but not showing it

build status: filter out uploaded packages

1. upload status
2. upload packages, but not for all arches
3. pending page shows status for all packages instead of just the remaining ones

Add webhook support

It would be nice if we could receive github webhooks and trigger an appveyor build so we can get rid of the appveyor cron job.

• https://github.com/bloomberg/python-github-webhook or https://github.com/nickfrostatx/flask-hookserver
• https://www.appveyor.com/docs/api/projects-builds/#start-build-of-branch-most-recent-commit

This requires two secrets on the deployment side of things: github webhook secret and appveyor api token.

Add a todo page for Python 2

• Show packages that depend on Python 2 but are not a dependency of any other package.

Add dep cycle info into the api response

We currently throw out missing packages due to dep cycles which makes autobuild alter fail. Ideally autobuild wouldn't even start them and write out in the queue status that some dep is missing.

queue: clarify the udpate frequency

it's every few hours and only affects the git info, so things can get out of sync.

add id/link to pending removals on queue page

On the queue page, there is no real indication there is another list after the giant pending package updates list. On discord, @elieux suggested adding ids to the lists, but I suggest going one step further and also adding some sort of "TOC" type thing with a link to that id, and also indicating that there is in fact another section down there.

Trigger a autobuild webhook when something changes

In the future we might want to start some build if anything in our responses changes, maybe with some debounce to avoid triggering it too much.

• We currently already detect changes for etag handling
• We currently always "detect" a change when the appveyor job runs because it always uploads a new release and the mtime changes. We should avoid updating the parsing release if nothing has changed.

show the real package if there are provides

https://packages.msys2.org/package/mingw-w64-x86_64-qt5-connectivity

Feature request: filter "repo updates" page by repo

With all the ucrt64 package updates lately, updates for any other repo are kind of buried. It would be nice to be able to be able to see the latest updates for a given repo.

The "pending package updates" page has a similar issue, but it doesn't have a 'repo' column, so I don't know if something like that would work there.

`prefix=/mingw64` missing in `openssl.pc` file

buildqueue: better handling of provided packages for VCS packages

Assume we have "foo" in the repo and try to build "foo-git" and "bar" depending on "foo". We shouldn't replace "foo" with "foo-git" in the deps because -git variants will likely fail and will not be used for building.

Not sure how pacman resolves this, but we should only replace a provides if there is not a real package with that name in the repo or in the queue.

some bugs

libcrypt is in the repo but no in the interface

lzo2 is in the queue and marked as not built, but it's there.

Add links for bugs reports per package

[Show Bug Reports] | [File Bug Report]

package site is down

Just opening an issue in case no one is aware yet

https://packages.msys2.org/ is down. Might be a certificate issue?

Also, can't download packages such as https://repo.msys2.org/mingw/ucrt64/mingw-w64-ucrt-x86_64-curl-8.8.0-1-any.pkg.tar.zst

sometimes hover info for status is off the top of the screen

I don't know if it's just due to my small tablet screen or something that happens to everyone but

I can't try to select the contents because it disappears if I move the mouse off the status field, or scroll because the page scrollbar is already at the top.

Host the package name mapping in a separate json file

So we can update it without needing to restart the server

required by bugs

mingw-w64-i686-python2-networkx not listing mingw-w64-i686-python2-openmdao as required by.

[Feature Request]: Add option to search by description

The current search bar only searches by package name in either Base Packages or Packages meaning if I were to want to look up a Fourth compiler, I would need to go to https://packages.msys2.org/package or https://packages.msys2.org/base and use my browser's find function to search for Fourth to bring up one of the 4th packages

avoid worker restart on timeout

when gunicorn times out it restarts the process which takes the site offline. Ideally this shouldn't happen.

root paths in file listing

so they can be copied and used as is in the shell

arch mapping bugs

• package vulkan-loader is wrongly binded to Arch Linux vulkan-headers package
  it need to be binded to vulkan-icd-loader

• dragon package is not the same as in Arch Linux too

Add cygwin links

• Fetch http://mirrors.kernel.org/sourceware/cygwin/x86_64/setup.ini
• Extract the source name from the "source:" tarball name and the matching version
• Link to https://cygwin.com/packages/summary/<src-name>-src.html

Return 500 when not fully populated

In case the server gets restarted it will just return empty everything until it fetches everything. Since we now depend on the API for building this leads to the CI job removing all already built packages.

The API should return 50x in case the data hasn't been fetched yet.

search: sort by score

Compute a score and sort by that instead of a simple filter. example: https://packages.msys2.org/search?t=pkg&q=qt5

Package links are currently broken

Click e.g. on any of the dependencies on https://packages.msys2.org/package/mingw-w64-clang-x86_64-graphicsmagick?repo=clang64

Copy to Clipboard Button

Having one for the installation section would be good.
I personally like the style in https://brew.sh but, yes it can be anything.

Add option to filter outdated by specific rdeps

I want to see all packages that are outdated and are transitive dependencies of package "foo" and "bar" for example.

It's hard to care about all outdated packages, but it's easier to care about a small set imo

arch renames

libgd -> gd