sabotage-linux/butch

old C version of package and build manager for sabotage linux, current version is written in shell/awk and can be found in KEEP/bin in the `sabotage` repo.

About

Butch is a relatively small and simple package and build manager, in less than 1000 SLOC of portable C code.

It stands out among other build managers in that it allows to do more than one thing at once. in the default configuration, 16 parallel download jobs (if necessary) and one build job are started, however changing these numbers just requires overriding them via the env variables BUTCH_DL_THREADS and BUTCH_BUILD_THREADS.

as soon as one download finished (and the checksum is valid), the build of the package will be started, unless all build slots are currently in use, or other dependencies not yet downloaded or built.

for any job, butch writes a shell script based on a user-supplied template (see sabotage linux repo for examples) which is then started and its output redirected into a log file.

It uses a custom, ini-like package format which contains the following information:

• dependency information (section deps)
• information about a tarball (section vars) (can contain source or binaries)
  • filesize
  • sha512 hash
  • tardir, if the tarball doesnt extract to a dir of the same name
  • other variables
• mirror information (section mirrors)
• build section (required to be the last section in the file) the contents of the build section are copied verbatim into the generated build script.

all sections are optional. it's entirely possible to have a package that just contains a list of dependencies (a so-called metapackage), or one that just contains a build section, or one that just contains a mirror url for a tarball + a main section containing the metainformation for that tarball.

example package script:

[deps]
package-foo
package-bar

[vars]
filesize=1024
sha512=deadbeef
tardir=foobar-4.2

[mirrors]
http://foo.bar/foobar-4.2.src.tar.gz
ftp://ftp.foo.bar/foobar-4.2.src.tar.gz
http://mirror.qux.bar/foobar-4.2.src.tar.gz

[build]
./configure && make && make install

Installation

cd /tmp
mkdir butch
cd butch
git clone https://github.com/sabotage-linux/butch
git clone https://github.com/rofl0r/libulz lib
git clone https://github.com/rofl0r/rcb2
export PATH=$PATH:/tmp/butch/rcb2
ln -s /tmp/butch/rcb/rcb2.py /tmp/butch/rcb2/rcb2
cd butch
make

butch's Issues

butch update fails when a pkg was removed

stat("/src/pkg/root-perms", 0x7fffc9512f60) = -1 ENOENT (No such file or directory)

butch rebuildall should treat all deps as "not installed"

i.e. remove them from installed.dat list before starting to build

butch should strip path prefixes from package names

this would allow usage such as butch install pkg/gdb

add config file mode

it should be possible to have a config file in /etc/butch.conf or in a user-given path instead of relying on one-char environment vars.

move installed.dat to another location

it really doesnt have any business to lurk in pkg/

possible it would make sense to make the location overridable with some env var or config file setting.

[BUG] empy package with only [deps] section does'nt get added to installed.dat

as experienced when i added
pkg/readline
with only
[deps]
libedit

is_installed failed for that package, and the circular ref error was printed for gdb.
adding [build] fixed it

butch download PKG doesnt do anything if PKG is already installed

even if the tarball was deleted

butch prefetch gettext
package gettext is already installed, skipping download
*** downloadqueue ***
*** buildqueue ***
2014-11-04 19:52:25 done.

make it possible to pass build mode to the template script

for example if the package is rebuilding or building.
only in the former case it makes sense to remove symlinks before the builds, because they could come from a different package.

BUTCH_SKIPLIST works only partially

the intention is to completely ignore items in the skiplist.
this works with regard to adding them not to the buildqueue, however it does not work if they are dependencies of another package.
so for example BUTCH_SKIPLIST=gcc3:stage0_gcc:stage0_musl:patch butch install gdb (gdb depends on patch, which depends on some stage0 stuff) will silently fail to build if the patch entry is not found in butch.db, as its listed as dependency of gdb.
possible fix is to iterate over all pkgs in both queues before starting them and removing any from deps that are in the skiplist.

butch should resolve deps till the root

for example for butch rebuild kernel, butch should not only resolve the dependency kernel-patches (which may already be installed), but also the deps of it which may have changed since the last build.
that basically means that for every dependency found that is not already in the deps list, dive into the deps until all are resolved, whether or not it's already installed.