aurutils/aurutils

Helper tools for the AUR.

SYNOPSIS

aurutils is a collection of scripts to automate usage of the Arch User Repository, with different tasks such as package searching, update checks, or computing dependencies kept separate.

The chosen approach for managing packages is local pacman repositories, rather than foreign (installed by pacman -U) packages.

INSTALLATION

Install one of the following packages:

• aurutils for the release version (recommended).
• aurutils-git for the master branch.

Upgrade notices are posted to the Arch forums. (RSS)

USAGE

Documentation is included as man pages with groff typesetting. They provide a general overview of the various utilities with several examples. Detailed instructions on how to set up a local repository can be found in the section CREATING A LOCAL REPOSITORY of the aur(1) man page.

TROUBLESHOOTING

See ISSUE_TEMPLATE.md. For informal discussion, see the #aurutils channel on Libera Chat.

aurutils's Issues

Make xtrace more useful

http://wiki.bash-hackers.org/scripting/debuggingtips#making_xtrace_more_useful

export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'

aurclone

Script to fetch (and update?) sources with git clone. Example use: multireq ... | aurclone

Allow to exclude custom repository packages

aurbuild will use repose with a local repository. After packages are built and added, they are filtered alongside official repository packages with aurqueue. Possible workarounds:

• Don't filter command-line arguments
• Use pacsift --regex --repo=(core|extra|community)

aurbuild: preserve PKGDEST

Right now all the script does is check if the variable is set; it does not pass it on to makepkg, and values in /etc/makepkg.conf take priority (via source).

aursync: change the name (again)

This does (and will do) a whole lot more than just "sync". I've thought of aurmaid, as an allusion to the helper idea.

aursync: clone/pull destination

Right now it just dumps stuff to $PWD.

aurbuild: allow to skip packages on error

In particular "a package was already built" errors.

aurbuild: devtools support

Build in a clean environment against latest repo packages. Fits the idea of upgrading AUR packages via -Syu.

• makechrootpkg (250de7d)
• Disable by default (be0e863)
• Signature verification (172e7a3)

Credits file

Not sure on the format.

http://pear.accc.uic.edu/ojs/index.php/fm/article/view/1151/1071

PKGBUILD

To track aurutils via pacman.

repofind: ensure that write permission are set in [custom] path

After enabling the [custom] repo and configured its path in pacman.conf, aursync fails because the directory hasn't be created yet:

==> ERROR: You do not have write permission to store packages in /home/remy/aur/custompkgs.
    Aborting...

You probably want to ensure the configured directory actually exists before trying to store package there, and also check its permissions.

Edit: Also, I get a warning: database file for 'custom' does not exist on the very first use. You might want to add a check to run repose in case this situation occurs.

aursync: snapshot support

With 5e2b0c3, parallel support for git clone was removed. Add snapshot support with aria2c as an alternative.

aur-sync: use $EDITOR instead of forcing the use of vifm

It's probably not a good idea to force the use of a specific third party tool to view PKGBUILD. Instead, using the standardized $EDITOR variable might be preferable.

aursync: check before upgrading VCS

By first verifying SRCINFO, we could run the (costly) --verifysource only when a VCS source is available.

aurbuild: --rebuild flag

Allow to run makepkg -f e.g. for soname bumps, see #28. Doesn't seem necessary with makechrootpkg, as it builds the package either way.

• build (172e7a3)
• build_chroot (4f01afc)
• aursync (831d9f4)

aurchain: i686 support

I don't know if I should add a workaround for this; it's an upstream issue (#20) and most Arch installs are x64, see pkgstats. But if the code is clean enough, by all means. cf. rmarquis/pacaur@6e3cb3b

aursearch: improve tmp file check

This construct is terrible:

tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT

# Check if tmp is not empty, as cd '' returns 0
if ! { [[ $tmp ]] && cd "$tmp"; }; then
    exit 1
fi

aurbuild: default root/pool values

repose uses $PWD when -r or -p are not specified, however -r/-p without arguments fails.

aursync: fails on SSH remote

ssh_exchange_identification: Connection closed by remote host

aursearch: PCRE support

The basic idea is to have a list of all AUR packages, and run grep -P on it. The RPC has no regex support, and a query minimum of two characters.

Possible ways to tackle this:

• Create a giant wordlist which matches pkgname in PKGBUILD(5). Considering package names go up and beyond 60 characters, this is not feasible (66^60 entries)
• Do as cower, and query the first two non-regex characters. Will prevent inverse searches.
• Query all two-letter combinations, save the results to temporary files, remove duplicates, and concatenate. This can be done in parallel, and the final result cached for later queries (XDG_CACHE_HOME)

aurqueue: non-pkgbase depends

Example: libunity. Before 3ed7884, pkgbase and pkgname appeared both in the output, which was problematic for automatic building. Now pkgbase appears (here: libdbusmenu), but in the wrong order, as tsort only sees "libunity libdbusmenu-glib".

aurchain handles this by replacing pkgname entries with pkgbase.

aursync: download-only option

Use a common.bash library

To reduce code duplication.

aur-graph: support versioned dependencies

With #5 restricting to official repositories, this could actually work. How complex would it be to implement?

See also rmarquis/pacaur@d3f7d55

Multilanguage support

Maybe we could steal some translations from makepkg and pacaur.

aursearch: cache results

We already cache packages.gz (if curl -z works, at least), we could also cache info results (for the complete AUR this is about 19mb), to be parsed later. Arguments/patterns could be stored via cksum.

aurchain: split package support

Since for some reason you can query split packages via RPC, they are included in the aurchain output. aurweb includes PackageBase in JSON, so we could work with that.

aurqueue: split packages in graph

See for example python-nikola, which includes split packages in the global make/depends.

aursync: update support

Options:

• git pull for all arguments
• only check with git describe / vercmp

See also:

https://github.com/Earnestly/dotfiles/blob/fed2affcdcc974f3003b1d5f7b07d162e4aead2e/local/bin/retroarch-updater#L28

aurchain: versioned dependencies

error: target not found: python-yapsy>=1.11.223
error: target not found: python2-yapsy>=1.11.223

Apparently, aurchain misses one dependency aurqueue catched.

Upstream issues

Closed

• aria2c errors are output to stdout, instead of stderr aria2/aria2#638
• aria2c no ability to disable download results aria2/aria2#639
• aria2c returns 0 when standard input is empty (No files to download) aria2/aria2#637 (wontfix)
• expac does not support repositories defined with Include falconindy/expac#28
• expac prints all packages in the sync db if stdin is empty falconindy/expac#29
• makechrootpkg possible to run makepkg --asroot FS#43432
• makechrootpkg unclear usage text on importing of variables like PKGDEST FS#44827
• makechrootpkg does not respect GNUPGHOME (https://git.archlinux.org/devtools.git/commit/?id=ffb5003fdacaece3540ba167f7e965a122133af0)
• makepkg install_package always returns 0 pacman-dev
• makepkg gnupg fails due to running in fakeroot FS#49946
• pacman vercmp alpha '' results in -1 (alpha < "") FS#49093
• pacutils update a single repository andrewgregory/pacutils#4
• pacutils missing space in confirmation prompt andrewgregory/pacutils#8
• pacutils pacsift --repo misses results compared to pacman -Sl andrewgregory/pacutils#9 (wontfix)
• pacutils pacsift --sync | pacinfo --short segfaults andrewgregory/pacutils#10
• pacutils support for inverting pacconf configuration andrewgregory/pacutils#12
• pacutils floating point exception andrewgregory/pacutils#13
• pacutils pacinfo returns 0 on missing package andrewgregory/pacutils#14
• pacutils support for inverting prompts andrewgregory/pacutils#15 (wontfix)
• setconf Increment variables with spaces xyproto/setconf#6
• shellcheck Does not handle UTF-8 on non UTF-8 locales gracefully koalaman/shellcheck#324

Reported

• aurweb arch-specific make/depends aren't separated FS#48796
• aurweb allow using POST requests to prevent 414 FS#49089
• aurweb does not allow to query provide-only packages FS#49090
• aurweb misses a Last-Modified field for packages.gz/pkgbase.gz FS#49092
• aurweb no support for regex searches FS#49099
• aurweb no support for versioned queries FS#54906
• aurweb no support for querying pkgbase FS#57230
• aurweb no support for querying groups FS#57902
• libalpm Relies on the locale to parse database files FS#49342
• makechrootpkg required variables for pinentry not available in chroot arch-projects
• arch-nspawn replaces CacheDir unconditionally patch
• makepkg "Unknown error" when answering "n" to pacman -Rn prompt (patch: https://lists.archlinux.org/pipermail/pacman-dev/2019-January/023103.html, discussion: http://ix.io/1yLx)
• pacsift: no output if --sysroot is specified andrewgregory/pacutils#26

Not reported

• expac misses support for --sysroot
• libmakepkg message.sh misses PURPLE and CYAN
• mkarchroot Failed to copy /etc/resolv.conf to /root/etc/resolv.conf
• makechrootpkg rm: cannot remove '/logdest/logpipe.zJztfPcl': No such file or directory
  pacman-dev
• makechrootpkg default makepkg options cannot be overridden (in particular --noconfirm)
• makepkg hangs on INT (Ctrl+C, interactive) when run from a script
• repo-add relies on hardcoded list of extensions: mimetype should suffice

ABI compliance checker

Could be useful to indicate when rebuilds are needed.

https://github.com/lvc/abi-compliance-checker

aurbuild: INT traps

makepkg seems to freeze the parent when it gets INT inside an (interactive) script.

aurbuild: separate makepkg if statement

aursync: file inspection

This used to be part of aurbuild, but is likely better suited for a wrapper script.

aursync: "prepare" performance

This could arguably be done in parallel (as bauerbill), see:

http://kmkeen.com/multithreaded-bash/

aursync: output new/updated directories on stdout

#9

Check traps

http://mywiki.wooledge.org/SignalTrap
http://www.cons.org/cracauer/sigint.html

aurbuild: repo discrepancy makepkg.conf/pacman.conf

Default commented options: in pacman.conf /home/custompkgs, in makepkg.conf /etc/makepkg.conf /home/packages. Report upstream.
#1

multireq: change the name!

Other scripts start in aur, so this is the odd one out...

aurbuild: makepkg exit codes

There's a lot of funky business going on here, e.g. install_package() always returns 0.

aurbuild: provider selection

Does --noconfirm choose the default (1) when multiple providers are available?

aurchain: missing dependencies

This happens when a split package with pkgbase that doesn't match a given pkgname (exactly, with grep -x) . Introduced by switching to PackageBase in #27.

Example (libunity)

{
   "ID": 273568,
   "Name": "libdbusmenu-glib",
   "PackageBaseID": 107058,
   "PackageBase": "libdbusmenu",
   "Version": "12.10.3.15.04.20150410.2-2",
   "Description": "A library for passing menus over DBus",
   "URL": "https:\/\/launchpad.net\/libdbusmenu",
   "NumVotes": 386,
   "Popularity": 18.675602999999999,
   "OutOfDate": null,
   "Maintainer": "WoefulDerelict",
   "FirstSubmitted": 1454522692,
   "LastModified": 1455036962,
   "URLPath": "\/cgit\/aur.git\/snapshot\/libdbusmenu.tar.gz",
   "Depends": [
    "glib2"
   ],
   "MakeDepends": [
    "docbook-xsl",
    "gnome-common",
    "gnome-doc-utils",
    "gobject-introspection",
    "gtk2",
    "gtk3",
    "intltool",
    "vala"
   ],
   "License": [
    "GPL3"
   ],
   "Keywords": []
  },

See also #35

zsh/bash completion

Completion for command line options

aurbuild

Build script which wraps repose and makepkg.

• Take package names from stdin.
• andrewgregory/pacutils#4
• systemd-nspawn support?
• #17 #18

aurbuild: select repo interactively

Instead of hardcoding a repo name (or specifying it manually), could pacconf parse the pacman configuration, and present a list of file:// repos ?

The choice could then be stored in XDG_CONFIG_HOME/aurbuild.

aursearch: parse JSON

There are many bash implementations to search the AUR, none ideal:

aurget: Uses a large awk filter
wrapaur: Runs an awk filter sequentially (slow)
prm: Runs jshon sequentially (slow)
packer: Uses one jshon call per argument + a big sed/paste/sort filter

We can get colors from util.sh, as with the other scripts. jq may be more straightforward here.

aursync: command line options

aurchain: sequential curl usage

Right now this performs badly on slow internet connections, e.g. plasma-git-meta with 22 requests can take 30+ seconds. The reason for sequential requests is that dependency trees are not visible from the RPC. bauerbill performs better here; it appears to use a "seen" logic to save requests.

Documentation: comments, examples

• howto: create a local repository
• howto: migrate foreign packages
• add examples
• move to .org
• code comments
  • aurbuild
  • aurchain
  • aurqueue
  • aursift
  • aursync
  • repofind (5565e83)

See also: #76