datadrake / cuppa Goto Github PK

View Code? Open in Web Editor NEW

67.0 8.0 9.0 199 KB

Comprehensive Upstream Provider Polling Assistant

License: Apache License 2.0

Makefile 1.42% Go 98.58%

cuppa's Introduction

cuppa

Comprehensive Upstream Provider Polling Assistant

Motivation

As a package maintainer, it's a challenging task to keep track of every new release of a piece of software. Usually this involves subscribing to mailing lists, signing up for notifications from FOSS portals like Github, or even subscribing to news sites. For a distro, this might also mean a repeated effort amongst its package maintainers. The inefficiency and time requirements of such an approach is inevitable. This has led several distros to create their own upstream tracking platforms to automate the process of tracking upstream releases. However, these platforms are often distro specific, leading to further duplication of effort between distros.

Goals

Support as many upstream providers as possible
Be completely distro agnostic
Extensibility
A+ Rating on Report Card

Progress

Supported Providers

CPAN
Github (with API Key support)
GitLab
GNOME
Hackage
Jetbrains
KDE
Launchpad
PyPi
RubyGems
Sourceforge

Planned Providers

BitBucket
FTP
Git

Stretch Goal Providers

Both of these will require some level of scraping to get useful info.

Unsupported Providers

NPM Completely pointless as this will just pivot to another provider
Stackage Not really in scope for this project and they seem to be missing a web API

Installation

Clone repo and enter its
make
sudo make install

Configuration

Your configuration file must be located at $HOME/.config/cuppa

Github Personal Access Keys

Github limits the number of requests per day for unauthenticated clients. If you would like to get around this limitation, you can configure Cuppa to use a Personal Access Key (PAK) by following the instructions here. You do not need to enable any OAuth Scopes.

Example:

[github]
key = "<personal access key>"

Usage

All cuppa commands follow the format:

cuppa CMD URL

where CMD is the action to perform and URL is the link to an existing source.

Commands (CMD)

CMD	Alias	Description
help	?	Get help for the other commands.
latest	l	Get the details for the latest (non-beta) release.
quick	q	Get just the new version number and URL if found.
releases	r	Get all known previous (non-beta) releases.

Example Sources

Provider	URL
CPAN	https://cpan.metacpan.org/authors/id/T/TO/TODDR/IO-1.39.tar.gz
Git	https://github.com/DataDrake/cuppa.git
Github	https://github.com/DataDrake/cuppa/archive/v1.0.4.tar.gz
GitLab	https://gitlab.com/corectrl/corectrl/-/archive/v1.0.6/corectrl-v1.0.6.tar.gz
GNOME	https://download.gnome.org/sources/gnome-music/3.28/gnome-music-3.28.2.tar.xz
Hackage	https://hackage.haskell.org/package/mtl-2.2.2/mtl-2.2.2.tar.gz
HTML	http://telepathy.freedesktop.org/releases/telepathy-logger/telepathy-logger-0.8.2.tar.bz2
JetBrains	https://download.jetbrains.com/ruby/RubyMine-2017.3.3.tar.gz
KDE	https://download.kde.org/stable/applications/18.12.0/src/akonadi-18.12.0.tar.xz
Launchpad	https://launchpad.net/catfish-search/1.4/1.4.4/+download/catfish-1.4.4.tar.gz
PyPi	https://pypi.python.org/packages/2c/a9/69f67f6d5d2fd80ef3d60dc5bef4971d837dc741be0d53295d3aabb5ec7f/pyparted-3.10.7.tar.gz
Rubygems	https://rubygems.org/downloads/sass-3.4.25.gem
Soureforge	https://sourceforge.net/projects/libmtp/files/libmtp/1.1.17/libmtp-1.1.17.tar.gz/download

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

cuppa's People

Contributors

Stargazers

Watchers

Forkers

happy-ferret siphomateke nilsonlinux sahwar ebonjaeger girtablulu autamus kimmalmo ionutnechita

cuppa's Issues

Current build fails

Trying to package it for alpine using upstream instructions leads to.

 ⮞  16:56:27  BUILD  
cuppa.go:20:2: cannot find package "github.com/DataDrake/cuppa/cli" in any of:
	/usr/lib/go/src/github.com/DataDrake/cuppa/cli (from $GOROOT)
	/home/impreza/usr/src/aports/testing/cuppa/src/src/github.com/DataDrake/cuppa/cli (from $GOPATH)
make: *** [Makefile:29: build] Error 1
>>> ERROR: cuppa: build failed

erroneous cpan results

Tested cuppa, version: 1.0.4 on some cpan links.

Returns: code1

cuppa l https://cpan.metacpan.org/authors/id/R/RG/RGARCIA/Sub-Identify-0.14.tar.gz

Returns:

 ⮞  CPAN checking for match:
 🗲  Could not get latest Sub, code: 1

Returns first result

cuppa l https://cpan.metacpan.org/authors/id/D/DO/DOY/Package-Stash-0.37.tar.gz

returns

Name      : Package
Version   : 0.26
Location  : https://cpan.metacpan.org/authors/id/I/IN/INGY/Package-0.26.tar.gz

Seems to grab the first result 'Package-...' it can find.
Should return:

Name      : Package-Stash
Version   : 0.37
Location  : https://cpan.metacpan.org/authors/id/D/DO/DOY/Package-Stash-0.37.tar.gz

Same behavior for:
'https://cpan.metacpan.org/authors/id/T/TO/TOKUHIROM/Test-Requires-0.10.tar.gz'
'https://cpan.metacpan.org/authors/id/E/ET/ETHER/namespace-autoclean-0.28.tar.gz'
'https://cpan.metacpan.org/authors/id/D/DR/DROLSKY/DateTime-Locale-1.22.tar.gz'

Usage Instructions/Examples

I'd like to use cuppa to help track versions for some PKGBUILD files but when I run cuppa releases https://github.com/neovim/neovim for example none of the providers return a match and no releases are found.

It's not very clear how the program is intended to be used. Am I just formatting the command wrong, is it supposed to be run against build files, if so would changes be required for different packaging formats, etc.

Problems with spaces in Makefile on old GNU make

Not critical, but I was just trying to build Cuppa on macOS and get an error:

$ make
Makefile:19: *** missing separator.  Stop.

The default make version in macOS 10.12 Sierra is GNU Make 3.81. I installed a newer make from Homebrew and the project compiles fine with the spaces, but it might be worth mentioning in the README.md that a newer GNU Make is needed, or switching to using tabs?

Makefile issue

/bin/sh: 4: pushd: not found
error: pathspec 'tags/v0.4.1' did not match any file(s) known to git.
/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cuppa/config/config.go:20:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/lib/go-1.10/src/github.com/BurntSushi/toml (from $GOROOT)
	/mnt/data/Sources/cuppa/build/src/github.com/BurntSushi/toml (from $GOPATH)
/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cuppa/cli/latest.go:20:2: cannot find package "github.com/DataDrake/cli-ng/cmd" in any of:
	/usr/lib/go-1.10/src/github.com/DataDrake/cli-ng/cmd (from $GOROOT)
	/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cli-ng/cmd (from $GOPATH)
/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cuppa/cli/root.go:21:2: cannot find package "github.com/DataDrake/waterlog" in any of:
	/usr/lib/go-1.10/src/github.com/DataDrake/waterlog (from $GOROOT)
	/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/waterlog (from $GOPATH)
/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cuppa/cli/quick.go:23:2: cannot find package "github.com/DataDrake/waterlog/format" in any of:
	/usr/lib/go-1.10/src/github.com/DataDrake/waterlog/format (from $GOROOT)
	/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/waterlog/format (from $GOPATH)
/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/cuppa/cli/root.go:23:2: cannot find package "github.com/DataDrake/waterlog/level" in any of:
	/usr/lib/go-1.10/src/github.com/DataDrake/waterlog/level (from $GOROOT)
	/mnt/data/Sources/cuppa/build/src/github.com/DataDrake/waterlog/level (from $GOPATH)
/bin/sh: 7: popd: not found
Makefile:51: recipe for target 'setup-deps' failed

Usage and configuration examples

I found it a bit tricky to configure and run cuppa. It would be nice if you could add a few examples to the readme to get users started:

How to build (install golang and run make)
How run cuppa (ie, example command lines of some source releases that work)
Location of config file (I got the location via reading the code in config/config.go)

Also, is it possible to have a sort of configuration where you list the projects and versions you're using, or are you supposed to run cuppa in a loop in a shell script? For example, something like this in the ~/.config/cuppa config file:

[projects]
restic 0.9.2 https://github.com/restic/restic/releases/download/v0.9.2/restic-0.9.2.tar.gz

cuppa finds older releases

Both cuppa quick and cuppa latest find the same older releases.

This is the latest timewarrior release — https://github.com/GothenburgBitFactory/timewarrior/archive/v1.1.1.tar.gz. Post111ToMaster seems to be the first release from Feb 4, 2018, the same day 1.1.1 was released.

Here, https://github.com/ansible/ansible/archive/v2.7.5.tar.gz is the last one. What cuppa found is much older — https://github.com/ansible/ansible/releases?after=v2.1.4.0-0.2.rc2 but it's under /releases/latest. Coincidentally, I found that the official Ansible releases can be found here — https://releases.ansible.com/ansible/, but cuppa doesn't work on https://releases.ansible.com/ansible/ansible-2.7.5.tar.gz

Potential corner cases in GitHub lookup

As promised, here's some examples from the report of some odd results. Tbh, they might be issues with how the authors mark the releases, but are worth a look.

ibus-libzhuyin 1.7.10  https://github.com/epico/ibus-libzhuyin/archive/.tar.gz
mate-menu 18.04.3 16.10.1 https://github.com/ubuntu-mate/mate-menu/archive/16.10.1.tar.gz
naev 0.7.0 NAEV-0.3.8 https://github.com/naev/naev/archive/NAEV-0.3.8.tar.gz
flare-engine 1.06 1.0 https://github.com/clintbellanger/flare-engine/archive/v1.0.tar.gz
neovim-qt 0.2.8  https://github.com/equalsraf/neovim-qt/archive/staging.tar.gz
numix-icon-theme 17.12.25  https://github.com/numixproject/numix-icon-theme/archive/.tar.gz
numix-icon-theme-circle 18.04.04  https://github.com/numixproject/numix-icon-theme-circle/archive/.tar.gz
numix-icon-theme-square 18.04.04  https://github.com/numixproject/numix-icon-theme-square/archive/.tar.gz

Bad list of release with GNU

There is a listing error when using GNU. The list of release is making by using the order the FTP but the order is false because the FTP sort filenames by characters and not by version so the version 1.9 is greater than 1.20 but this comparison is false for cuppa who need to have 1.9 then 1.20.

Example cuppa latest https://ftp.gnu.org/gnu/wget/wget-1.20.tar.gz:
Give this :

Name : wget
Version : 1.9.1
Location : https://mirrors.rit.edu/gnu/wget/wget-1.9.1.tar.gz
Published : 2003-11-14T00:00:00Z

Instead of:

Name : wget
Version : 1.20.1
Location : https://mirrors.rit.edu/gnu/wget/wget-1.20.1.tar.lz
Published : 2018-12-26T20:13:00Z

I think you should sort the list of release by date to fix this issue (and as a precaution do it for all other providers to avoid future problems).

Suggested additional providers

I'm not sure if you want such suggestions here. But if so, I would like to point out the "Free Software Directory" as an important one. In particular packages under gnu.org/software. Feel free to add to this list if deemed appropriate.

Any reason to require personal access token?

hey @DataDrake ! I'm trying this for the first time and it's a great idea, but I have a quick question. The responses that I get are basically 404 when I don't have an access token, even for public repos.

2020/10/03 10:17:44 Unable to find cuppa https://github.com/DataDrake/cuppa/archive/v1.1.0.tar.gz

but I've used the GitHub API before, and although the limit is lower, it should work without a token. What am I doing wrong?

cuppa q https://github.com/zsh-users/zsh-autosuggestions/archive/v0.5.0.tar.gz
 🕱  10:47:05  FATAL    No release found.