freight-team / freight Goto Github PK

View Code? Open in Web Editor NEW

This project forked from rcrowley/freight

106.0 106.0 37.0 405 KB

A modern take on the Debian archive.

License: Other

Makefile 6.46% Shell 93.14% Dockerfile 0.13% Emacs Lisp 0.27%

freight's People

Contributors

Stargazers

Watchers

freight's Issues

Installation on Ubuntu18.04 fails

after running:
sudo apt-get update
got an error:
The repository 'http://build.openvpn.net/debian/freight_team bionic Release' does not have a Release file.
and after running:
sudo apt-get -y install freight
error:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Package freight is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'freight' has no installation candidate

Adding a source package to a repo before a binary package results in error

Steps to repro:

Add a source package to a brand new (and empty) repo

$ FREIGHT_CONF=/etc/freight.test.conf freight-add /tmp/scamper_20140122-1.dsc apt/jessie
$ FREIGHT_CONF=/etc/freight.test.conf freight-add /tmp/scamper_20140122.orig.tar.gz apt/jessie
$ FREIGHT_CONF=/etc/freight.test.conf freight-add /tmp/scamper_20140122-1.debian.tar.xz apt/jessie

Run freight-cache

$ FREIGHT_CONF=/etc/freight.test.conf freight-cache
# [freight] adding scamper_20140122-1.dsc to pool
# [freight] adding scamper_20140122.orig.tar.gz to pool
# [freight] adding scamper_20140122-1.debian.tar.xz to pool
/usr/bin/freight-cache: 128: eval: cannot create /var/cache/freight.test/dists/jessie-20160324133929871040415/main/binary-i386/Release: Directory nonexistent

If you add a binary package after this and run freight-cache everything works fine. I'm not sure if a source-only repo is a realistic possibility, but perhaps a better error message might be better.

Multiple GPG support uses bashisms

The multiple GPG key support I added in #2 uses bashisms, which breaks under dash:

bin/freight-init: 20: bin/freight-init: Syntax error: "(" unexpected

I hadn't realised freight was using /bin/sh explicitly. There are some small tweaks possible to the option parsing to make all of freight run properly under regular bash, or I can rewrite the changes in a more POSIX compatible style if preferred.

I'd tend towards using bash as it has other useful features we can start depending on, and freight's a reasonably complex set of scripts.

freight-cache without TTY raises "gpg: invalid option "-abs --no-tty" error

When running freight-cache without a TTY (e.g. over SSH) since 0.3.7, the GPG signing step fails with:

gpg: invalid option "-abs --no-tty"

c112399 changed the GPG options passed in to be quoted such that -abs and --no-tty are a single argument.

Bad Installation Instructions

When attempting to install frieght on a fresh 16.04 Ubuntu server, the instructions are not functional.

wget -O - https://swupdate.openvpn.net/repos/repo-public.gpg|sudo apt-key add -
echo "deb http://build.openvpn.net/debian/freight_team $(lsb_release -sc) main" | sudo tee  /etc/apt/sources.list.d/freight.list
sudo apt-get update
sudo apt-get -y install freight

yields the following results

...
W: The repository 'http://packages.rcrowley.org xenial Release' does not have a Release file
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
...
Package freight is not available, but is referred to by another package.
This may mean the package is missing, has been obsoleted, or 
is only available from another source

E: Package 'freight' has no installation candidate

Additionally if I checkout the v0.3.5 tag from git and use these instructions:

sudo apt install make
make && sudo make install

yields

make: Nothing to be done for 'all'

and there are no binaries available, bash completion file, etc.

Additionally with the v0.3.5 tag from git and using these instructions:

sudo apt install build-essential
dpkg-buildpackage -uc -us -b

yields

tail: cannot open 'debian/changelog' for reading: No such file or directory
dpkg-buildpackage: error: tail of debian changelog gave error exit status 1

use dpkg-scanpackages

suggestion from rcrowley#53

When freight-clear-cache is necessary?

I've created a script to update public repositories/archives with many .deb packages, but it works increasingly slow with each new package.
My actions for each package file are:
freight-add
freight-clear-cache
freight-cache

but I suspect not each time freight-clear-cache and freight-cache are necessary.

freight does not generate Contents File

For main component of wheezy distribution for amd64 architecture, this file would normally be located at

/dists/wheezy/main/Contents-amd64.gz

"Contents" indices files are needed for example by the apt-file tool, it would be really nice to be able to use this tool with an repository created by freight.

Feature request: rpm support

It would be cool if freight add - also handled rpm repo's - and gpg signing of rpms.
It would need to "know to fail" - if one tried to add an rpm to a debian repo..

Either you could simply add a setting to a conf file (which is per repo) - simply had VARCACHE and VARLIB setting for rpm.. or if freight was made to prefix with type.. so you could set a config setting:
HANDLEYUM=true|false
HANDLEAPT=true|false
and prefixed repo with $VARCACHE/YUM/$usual path - as it already does with VARLIB dir.

What do you think of the idea? (then one could add other rpm-based systems, which does not support yum repo format as well).

I might hack a little on it if I get time.. if we agree on how it should be implemented (in helicopter view :)

Getting an active maintainer for the project

@freight-team/developers I see that there are lots of PRs open and I don't see much activity in getting them merged.

I currently maintain only one freight-based apt repository which works for my use-case. So I don't want to commit myself to managing this project.

What I would propose is adding @yegorich to the team - he's the most active contributor right now.

What do you think?

Add option to avoid re-computing hashes

Hi,

We have a repo that's slowly growing in size, and freight is starting to take a long time to run. By far most of the time seems to be spent here:

        # Finish the top-level `Release` file with references and
        # checksums for each sub-`Release` file and `Packages.gz` file.
        # In the future, `Sources` may find a place here, too.
        find "$DISTCACHE" -mindepth 2 -type f -printf %P\\n |
            grep -v ^\\. |
            while read -r FILE; do
                SIZE="$(apt_filesize "$DISTCACHE/$FILE")"
                echo " $(apt_md5 "$DISTCACHE/$FILE") $SIZE $FILE" >&3
                echo " $(apt_sha1 "$DISTCACHE/$FILE") $SIZE $FILE" >&4
                echo " $(apt_sha256 "$DISTCACHE/$FILE") $SIZE $FILE" >&5
                echo " $(apt_sha512 "$DISTCACHE/$FILE") $SIZE $FILE" >&6
            done 3>"$TMP/md5sums" 4>"$TMP/sha1sums" 5>"$TMP/sha256sums" 6>"$TMP/sha512sums"

Would it be acceptable to place a dot-file along with the repo files with the computed hashes, so that this piece can do a lookup in a file instead of computing the hash?

I've got a fork with a simple (and yet untested) implementation: master...runejuhl:avoid-rehashing . If you feel like this is a good way of doing it I'll be glad to do some testing and submit a PR.

CACHE and LIB dirs need to be absolute

All the freight scripts assume that CACHE and LIB are absolute paths.

It should either bail if the paths are not absolute, or use readlink -f to turn them absolute before using them.

Without this, I get a ton of weird errors from freight-add and freight-cache.

dist with only one package is skipped in cache ???

Hi, strange bug occurs on my http://repo.vitexsoftware.cz/ :

When generating all:

One package from natty and one package from sid was not added

But when generating only given distro the before missing package is added:

The /var/lib/freight/apt/ content:

vitex@vitexsoftware:~$ ls -la /var/lib/freight/apt/*/*
-rw-r--r-- 1 vitex vitex   1632324 říj  5 14:33 /var/lib/freight/apt/bionic/devilutionx_0.5.0_amd64.deb
-rw-r--r-- 1 vitex vitex   1174364 říj  5 14:33 /var/lib/freight/apt/bionic/devilutionx_0.5.0.tar.xz
-rw-r--r-- 1 vitex vitex   1631972 říj  7 03:39 /var/lib/freight/apt/bionic/devilutionx_0.5.0.2_amd64.deb
-rw-r--r-- 1 vitex vitex   1173980 říj  7 03:39 /var/lib/freight/apt/bionic/devilutionx_0.5.0.2.tar.xz
-rw-r--r-- 1 vitex vitex   1247764 říj  8 00:30 /var/lib/freight/apt/bionic/devilutionx_0.5.0.3_amd64.deb
-rw-r--r-- 2 vitex vitex      2040 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.2.5-38_all.deb
-rw-r--r-- 2 vitex vitex      2044 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.2.5-41_all.deb
-rw-r--r-- 2 vitex vitex      2032 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.3.0_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.3.0.1_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.3.0.2_all.deb
-rw-r--r-- 2 vitex vitex      2056 říj  3 07:53 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server-backup_2019.3.0.4_all.deb
-rw-r--r-- 2 vitex vitex      9628 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server_2019.2.5-38_all.deb
-rw-r--r-- 2 vitex vitex 123553700 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server_2019.2.5-41_all.deb
-rw-r--r-- 2 vitex vitex 123557496 říj  9 12:12 /var/lib/freight/apt/bionic/flexibee-server_2019.3.0.1_all.deb
-rw-r--r-- 2 vitex vitex 123552748 říj  9 12:13 /var/lib/freight/apt/bionic/flexibee-server_2019.3.0.2_all.deb
-rw-r--r-- 2 vitex vitex 123552772 říj  3 07:54 /var/lib/freight/apt/bionic/flexibee-server_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex 123549912 říj  9 12:13 /var/lib/freight/apt/bionic/flexibee-server_2019.3.0.4_all.deb
-rw-r--r-- 2 vitex vitex 131255442 říj  3 07:29 /var/lib/freight/apt/bionic/flexibee_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex 131253816 říj  9 12:01 /var/lib/freight/apt/bionic/flexibee_2019.3.0.4_all.deb
-rw-r--r-- 3 vitex vitex     65380 říj  4 01:36 /var/lib/freight/apt/bionic/nightly_71.0a1_all.deb
-rw-r--r-- 1 vitex vitex      3144 říj  1 00:12 /var/lib/freight/apt/buster/composer-global-update_1.0_all.deb
-rw-r--r-- 1 vitex vitex   6830712 říj  7 03:25 /var/lib/freight/apt/buster/devilutionx-dbgsym_0.5.0.2_amd64.deb
-rw-r--r-- 1 vitex vitex   1637052 říj  7 03:25 /var/lib/freight/apt/buster/devilutionx_0.5.0.2_amd64.deb
-rw-r--r-- 1 vitex vitex   2411776 říj  7 03:26 /var/lib/freight/apt/buster/devilutionx_0.5.0.2.tar.xz
-rw-r--r-- 1 vitex vitex   1250884 říj  8 00:23 /var/lib/freight/apt/buster/devilutionx_0.5.0.3_amd64.deb
-rw-r--r-- 1 vitex vitex      6352 říj  2 16:49 /var/lib/freight/apt/buster/flexibee-matcher_0.7.10_all.deb
-rw-r--r-- 1 vitex vitex      7828 zář 19 10:19 /var/lib/freight/apt/buster/flexibee-testing-tools_0.9_all.deb
-rw-r--r-- 1 vitex vitex  72883600 srp  1 16:56 /var/lib/freight/apt/buster/gcompris-qt-data_0.97_all.deb
-rw-r--r-- 1 vitex vitex   2425844 srp  1 16:56 /var/lib/freight/apt/buster/gcompris-qt-dbgsym_0.97_amd64.deb
-rw-r--r-- 1 vitex vitex    173212 srp  1 16:56 /var/lib/freight/apt/buster/gcompris-qt_0.97_amd64.deb
-rw-r--r-- 1 vitex vitex     39440 srp  1 16:56 /var/lib/freight/apt/buster/gcompris_0.97_all.deb
-rw-r--r-- 3 vitex vitex 177106982 čec 18 13:37 /var/lib/freight/apt/buster/netbeans_11.1-1beta_all.deb
-rw-r--r-- 3 vitex vitex     65380 říj  4 01:36 /var/lib/freight/apt/buster/nightly_71.0a1_all.deb
-rw-r--r-- 1 vitex vitex     24384 říj  2 22:46 /var/lib/freight/apt/buster/php-ease-core_0.11_all.deb
-rw-r--r-- 1 vitex vitex     26436 říj  2 22:41 /var/lib/freight/apt/buster/php-ease-html_0.5_all.deb
-rw-r--r-- 1 vitex vitex     23352 říj  2 23:11 /var/lib/freight/apt/buster/php-flexibee-bricks_0.27.1_all.deb
-rw-r--r-- 1 vitex vitex      5072 říj  1 00:47 /var/lib/freight/apt/buster/php-flexibee-config_0.14_all.deb
-rw-r--r-- 1 vitex vitex    149580 říj  2 02:21 /var/lib/freight/apt/buster/php-flexibee_1.33_all.deb
-rw-r--r-- 1 vitex vitex    770952 zář  6 01:14 /var/lib/freight/apt/buster/phpunit-skeleton-generator_2.0.2_all.deb
-rw-r--r-- 1 vitex vitex  14654172 zář  1 13:45 /var/lib/freight/apt/buster/synergy-dbgsym_1.10.3-rc1_amd64.deb
-rw-r--r-- 1 vitex vitex    652404 zář  1 13:45 /var/lib/freight/apt/buster/synergy_1.10.3-rc1_amd64.deb
-rw-r--r-- 3 vitex vitex 177106982 čec 18 13:37 /var/lib/freight/apt/natty/netbeans_11.1-1beta_all.deb
-rw-r--r-- 1 vitex vitex     65380 říj  7 03:23 /var/lib/freight/apt/sid/nightly_71.0a1_all.deb
-rw-r--r-- 2 vitex vitex      2040 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.2.5-38_all.deb
-rw-r--r-- 2 vitex vitex      2044 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.2.5-41_all.deb
-rw-r--r-- 2 vitex vitex      2032 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.3.0_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.3.0.1_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.3.0.2_all.deb
-rw-r--r-- 2 vitex vitex      2056 říj  3 07:53 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex      2060 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server-backup_2019.3.0.4_all.deb
-rw-r--r-- 2 vitex vitex      9628 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server_2019.2.5-38_all.deb
-rw-r--r-- 2 vitex vitex 123553700 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server_2019.2.5-41_all.deb
-rw-r--r-- 2 vitex vitex 123557496 říj  9 12:12 /var/lib/freight/apt/stretch/flexibee-server_2019.3.0.1_all.deb
-rw-r--r-- 2 vitex vitex 123552748 říj  9 12:13 /var/lib/freight/apt/stretch/flexibee-server_2019.3.0.2_all.deb
-rw-r--r-- 2 vitex vitex 123552772 říj  3 07:54 /var/lib/freight/apt/stretch/flexibee-server_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex 123549912 říj  9 12:13 /var/lib/freight/apt/stretch/flexibee-server_2019.3.0.4_all.deb
-rw-r--r-- 2 vitex vitex 131255442 říj  3 07:29 /var/lib/freight/apt/stretch/flexibee_2019.3.0.3_all.deb
-rw-r--r-- 2 vitex vitex 131253816 říj  9 12:01 /var/lib/freight/apt/stretch/flexibee_2019.3.0.4_all.deb
-rw-r--r-- 1 vitex vitex    149644 říj  3 15:41 /var/lib/freight/apt/stretch/flexipeehp_1.33_all.deb
-rw-r--r-- 1 vitex vitex   1415400 zář  6 14:16 /var/lib/freight/apt/stretch/netbeans-php-tools_0.10_all.deb
-rw-r--r-- 3 vitex vitex 177106982 čec 18 13:37 /var/lib/freight/apt/stretch/netbeans_11.1-1beta_all.deb
-rw-r--r-- 3 vitex vitex     65380 říj  4 01:36 /var/lib/freight/apt/stretch/nightly_71.0a1_all.deb
-rw-r--r-- 1 vitex vitex    840548 zář  6 12:37 /var/lib/freight/apt/stretch/phpunit-skelgen_2.0.2_all.deb

Another example:

Minor suggestion from shellcheck linting of freight-cache

https://github.com/freight-team/freight/blob/master/bin/freight-cache#L70 reads

[ -z "$GPG_PASSPHRASE_FILE" -o -r "$GPG_PASSPHRASE_FILE" ] || echo "# [freight] could not read passphrase file: $GPG_PASSPHRASE_FILE." >&2

This is reported as ambigious and should be changed to use the form [ something ] || [ something ] for better consistency. Like so:

[ -z "$GPG_PASSPHRASE_FILE" ] || [ -r "$GPG_PASSPHRASE_FILE" ] || echo "# [freight] could not read passphrase file: $GPG_PASSPHRASE_FILE." >&2

Thank you.

IRC channel?

Would there be interest in a #freight IRC channel on Freenode or something similar for realtime communication?

`find: command not found`, findutils missing dependency

In a minimal fedora docker container, I've run this

$ yum -y install freight
$ freight add my.deb apt/squeeze
$ freight cache
/usr/bin/freight-cache: line 95: find: command not found

yum install findutils fixes this issue

freight missing packages from pool

I have run into a strange problem. Some packages are in the Packages file, but the corresponding file is missing in the pool.
The strange thing about it that is that there are two sets of packages.
When I run freight-cache, in one run set A gets put to the pool, in the next run set B, then set A again and so on.
I tried to delete the pool directory and dists/bionic*
After that set A contains all the packages, and set B is empty to the extent that even the pool directory gets deleted.

it seems that the bionic-* directory does contain all the packages, but the Packages file refers to the pool.

my /etc/freight.conf

# Example Freight configuration.

# Directories for the Freight library and Freight cache.  Your web
# server's document root should be `$VARCACHE`.
VARLIB="/var/lib/freight"
VARCACHE="/var/www/repo/apt"

# Default `Origin`, `Label`, `NotAutomatic`, and
# `ButAutomaticUpgrades` fields for `Release` files.
ORIGIN="Kode Konveyor"
LABEL="KodeKonveyor"
NOT_AUTOMATIC="no"
BUT_AUTOMATIC_UPGRADES="no"

# Cache the control files after each run (on), or regenerate them every
# time (off).
CACHE="off"

# GPG key(s) to use to sign repositories.  This is required by the `apt`
# repository provider.  Use `gpg --gen-key` (see `gpg`(1) for more
# details) to generate a key and put its email address here.
#
# Multiple addresses can be given sign the repository with them all.
GPG="[email protected]"
# GPG="[email protected] [email protected]"

# Message digest algorithm that GPG should use to sign the repository.
# It is not recommended to use SHA1 as new versions of `apt` will report
# that the repository is half-broken due to weak digest.
#
# SHA512 is the default
GPG_DIGEST_ALGO="SHA512"

# Whether to follow symbolic links in `$VARLIB` to produce extra components
# in the cache directory (on) or not (off).
SYMLINKS="off"

Second install fails due to existing gzip files

# make install find bin -type f -printf %P\\0 | xargs -0r -I__ install -D bin/__ /usr/local/bin/__ find lib -type f -printf %P\\0 | xargs -0r -I__ install -m644 -D lib/__ /usr/local/lib/__ find man -type f -name \*.[12345678] -printf %P\\0 | xargs -0r -I__ install -m644 -D man/__ /usr/local/share/man/__ find man -type f -name \*.[12345678] -printf %P\\0 | xargs -0r -I__ gzip /usr/local/share/man/__ gzip: /usr/local/share/man/man1/freight-clear-cache.1.gz already exists; not overwritten gzip: /usr/local/share/man/man1/freight.1.gz already exists; not overwritten gzip: /usr/local/share/man/man1/freight-init.1.gz already exists; not overwritten gzip: /usr/local/share/man/man1/freight-add.1.gz already exists; not overwritten gzip: /usr/local/share/man/man1/freight-cache.1.gz already exists; not overwritten gzip: /usr/local/share/man/man5/freight.5.gz already exists; not overwritten Makefile:27: recipe for target 'install-man' failed make: *** [install-man] Error 123

It can be worked around by switching the gzip to forced mode in the Makefile but there is probably a better solution.
# make install find bin -type f -printf %P\\0 | xargs -0r -I__ install -D bin/__ /usr/local/bin/__ find lib -type f -printf %P\\0 | xargs -0r -I__ install -m644 -D lib/__ /usr/local/lib/__ find man -type f -name \*.[12345678] -printf %P\\0 | xargs -0r -I__ install -m644 -D man/__ /usr/local/share/man/__ find man -type f -name \*.[12345678] -printf %P\\0 | xargs -0r -I__ gzip -f /usr/local/share/man/__ find etc -type f -not -name freight.conf -printf %P\\0 | xargs -0r -I__ install -m644 -D etc/__ /usr/local/etc/__

Test suite required

A test suite is required to help us prevent regressions, something that can easily be enabled with Travis CI on this repository.

As to frameworks, bats would be quite an easy way to write tests in the same language as the project. It can be installed easily on Travis CI, and we just create a directory of .bats files containing individual tests to test different pieces of functionality.

Minor suggestion from shellcheck linting of freight-init

https://github.com/freight-team/freight/blob/master/bin/freight-init#L72 reads

[ -z "$GPG" -o -z "$DIRNAME" ] && usage 1

This should be changed to

[ -z "$GPG" ] || [ -z "$DIRNAME" ] && usage 1

Example

$ [ -z "$GPG" ] || [ -z "$DIRNAME" ] && echo foo 
foo
$ GPG=1
$ [ -z "$GPG" ] || [ -z "$DIRNAME" ] && echo foo
foo
$ DIRNAME=1
$ [ -z "$GPG" ] || [ -z "$DIRNAME" ] && echo foo
$ [ -z "$GPG" ] && echo foo
$ [ -z "$DIRNAME" ] && echo foo
$ unset GPG
$ [ -z "$GPG" ] && echo foo
foo
$ [ -z "$GPG" ] || [ -z "$DIRNAME" ] && echo foo
foo
$

Used the online tool at http://shellcheck.net to do the linting of this file.

freight-add: raising an error when adding an already existing file

We have automatic tools to add and check repository managed by freight.
It would be nice to create (if wanted) an error (exit code not 0) when adding a file in repository (freight-add) which has already been added.

We already do this by checking the file and if it does not exist send an error to the asking user, but it is not atomic like the ln command in freight-add command.

Attached is a patch which can do this:

By option -e or --add-error
By environment variable FREIGHT_ADD_ERROR if defined not empty

freight-error-add.patch

Add support for source packages in git format

So far only tarball packages are accepted.

2 words architecture?

[freight] adding packagename_0.9.8o-4_kfreebsd-amd64.deb to pool
tee: /.../public-archive/dists/codename-20170610172737818859675/mirror/binary-kfreebsd-amd64/Packages: No such file or directory

This seems some kind of error with the architecture name parsing.

Enhancement - Other managers (yum in particular)

HI guys,

Is there any documentation about what is required to generate repositories for other package mangers like yum?

Or does someone already have a script somewhere that can do this?

Thanks!

rcrowley/freight -> freight-team/freight

Have you guys talked to @rcrowley about officially taking over the project? There are no pointers to freight-team from rcrowley, and all the Google results for freight point to his now inactive repository. I had to find this fork by tracing back @domcleal's commits in the Fedora package to his blog.

freight is failing to host ubuntu 18.04 packages.

For the ubuntu 18.04 packages, the Packages.gz which gets generated is not proper. The parent package and dependent package versions are not listed in order, so the installation os parent package fails. For e.g python and python-minimal versions are not listed correctly. So when apt takes the first version that is listed for both, there'll be a mismatch.

Feature request: Keeping only X versions of a package

We use freight for software repos - where we build ourself - and hence we get a lot of new versions.. and currently freight has no support for even removing packages from repo.. so we have manually remove files and run freight cache..

It would be nice if freight handled freight-remove - and a bonus if freight could be configured to only keep max X instances of a certain package.

Reword manpage after #32

I don't think it's making sense to list each and every extension anyway.

How to replace a cache package file with same name

I have pushed this package to a Debian archive:
sltd_1.1.6_all.deb
I've fixed some detail to package scripts (preinst, postinst,...) and packed again with dpkg-buildpackage, but freight-add and freight-cache commands don't update library/cache because they say that sltd_1.1.6_all.deb already exist.

How can I replace a .deb file that has same name?

InRelease.gpg is missing

When trying to use this commit: 0a8c99a
InRelease.gpg file is not created so when running:
sudo apt-get update
we got an error:
bionic InRelease: The following signatures couldn't be verified because the public key is not available:

gpg2 incompatibilities

Hi,

I'm currently facing an issue while trying to sign a repository on Debian Stretch.
The error looks like:

$ freight cache apt/zi-stable
# [freight] adding burp-client_2.0.52-3_amd64.deb to pool
# [freight] adding burp-core_2.0.52-3_amd64.deb to pool
# [freight] adding burp-server_2.0.52-3_amd64.deb to pool
chmod: cannot access '/somewhere/repos/debian/stretch/cache/work.6178.8nLuU5zPkb/gpg/pubring.gpg': No such file or directory

I suppose the problematic code lies in lib/freight/apt.sh:233

    # Generate `pubkey.gpg` containing the plaintext public key and
    # `keyring.gpg` containing a complete GPG keyring containing only
    # the appropriate public keys.  `keyring.gpg` is appropriate for
    # copying directly to `/etc/apt/trusted.gpg.d`.
    mkdir -m700 -p "$TMP/gpg"
    # shellcheck disable=SC2086
    gpg -q --export -a $GPG |
    tee "$VARCACHE/pubkey.gpg" |
    gpg -q --homedir "$TMP/gpg" --import
    chmod 644 "$TMP/gpg/pubring.gpg"
    mv "$TMP/gpg/pubring.gpg" "$VARCACHE/keyring.gpg"

gpg --version returns gpg (GnuPG) 2.1.16 and indeed, the doc says:

gpg used to keep the public key pairs in two files: pubring.gpg and secring.gpg. The only difference is that secring stored in addition to the public part also the private part of the key pair. The secret keyring thus contained only the keys for which a private key is available, that is the user’s key. It required a lot of code to keep both versions of the key in sync and led to sometimes surprising inconsistencies.

source

And indeed, running some tests show there is no more pubring.gpg file:

$ mkdir -m700 /tmp/gpg2
$ gpg -q --export <mykey> | gpg -q --homedir /tmp/gpg2/ --import
$ ls /tmp/gpg2/
S.gpg-agent  S.gpg-agent.browser  S.gpg-agent.extra  S.gpg-agent.ssh  private-keys-v1.d  pubring.kbx  pubring.kbx~  trustdb.gpg

Test suite: problems with gpg keys

What are the exact requirements to run a test suite? All the tests (beginning with the freight-cache builds distro Release/InRelease file) that require a key fail with the following message:

gpg: skipped "[email protected]": No secret key
gpg: signing failed: No secret key

I have created such a key myself and it is listed via gpg --list-keys. Nevertheless, it will be ignored during the test.

freight-add ignores multiverse, universe options and adds the package in main only

While adding a package, even if apt/xenial/multiverse or universe is specified, the package is added in main itself. As per freight-add doc, main is default for apt but it should also support other options.

Request for a new release 03-2021

Could we either get a new release tagged or a rough roadmap of the next release?

Ability to remove packages

I needed to remove a package and so I followed the steps here:

https://oitibs.com/remove-package-from-freight/

Those worked fine, but I wonder is there any interest in this being a supported option? Possibly supporting this is more complicated than I naively imagine?

Let use keyring.gpg or pubkey.gpg ?

At README.md is documented to register Freight project's repository using repo-public.gpg , but it's indicated to people register new made repositories using keyring.gpg
Why the difference?

Can this be better for usage indication?
wget -O - http://example.com/pubkey.gpg | sudo apt-key add -

CentOS 8 EPEL

Could you publish it to CentOS 8 EPEL repository ?

Releasing freight 0.3.11?

@mmoll , @domcleal : what about releasing freight 0.3.11 soon? If we merge PR
#57 then 0.3.11 would include these changes:

David R. Bild (2):
      Add NotAutomatic and ButAutomaticUpgrades to conf
      Include NotAutomatic and ButAutomaticUpgrades in Release files

Jeremy Teale (1):
      Prevent GPG >= 2.1 from using .kbx pubring format

Stephen Ryan (1):
      Let install-man task work on second and subsequent attempts

Vasily Laur (2):
      Debianize the sources
      Change instruction to build deb package in README

no debian buster repo for installing freight from Debian archive

In the README there is a "Install from Debian archive" section:
https://github.com/freight-team/freight#from-a-debian-archive

but the debian repo does not contain buster distro:
http://build.openvpn.net/debian/freight_team/dists/

Though the stretch is there.

Need to add packages for buster distro.

Apt repository key expires soon (4th August 2017)

Freight itself is available in freight-team's apt repository. The apt signing key will, however, expire 4th August 2017, which will trigger complaints from apt when people try to use our repository.

list packages support

I'd really like to be able to switch to freight to support multiple versions however two features are missing:

removal of package: issue in original repo rcrowley#40
listing of packages.

In reprepro world I do a reprepro -b CACHE_FOLDER list DISTRIBUTION to list all the packages and then present this via the web server.

can something like that be supported?

GPG signing fails with GnuPG 2.x

Running freight-cache fails on Debian 9 which comes with GnuPG 2.1.18 installed:

$ freight-cache -c /etc/freight.conf
gpg: signing failed: Inappropriate ioctl for device
gpg: signing failed: Inappropriate ioctl for device
# [freight] couldn't sign the repository, perhaps you need to run
# [freight] gpg --gen-key and update the GPG setting in /etc/freight.conf
# [freight] (see freight(5) for more information)

When I run the same with "set -x" in /usr/bin/freight-cache the offending command-line is conveniently shown:

+ gpg -abs --use-agent [email protected] --batch --passphrase-fd 1 --passphrase-file /etc/freight.pass --personal-digest-preferences SHA512 -o/var/www/html/debian/repo/work.28782.fFKwq5V0mt/release_last_signature.gpg /var/www/html/debian/repo/dists/xenial-20170905111739900168706/Release
gpg: signing failed: Inappropriate ioctl for device
gpg: signing failed: Inappropriate ioctl for device

The problem seems to be that GnuPG 2.x by default requires gpg-agent and expects the user to enter the password via some pinentry mechanims (e.g. pinentry-curses, pinentry-gnome3). A fix / workaround is to append --pinentry-mode loopback to the gpg command-line in lib/freight/apt.sh. However, the --pinentry-mode parameter is unknown to GnuPG 1.x, so it would have to be added conditionally.

Another way to fix this would be to ensure that freight can actually use a running gpg-agent. In my case it does, except that it does not pop up any pinentry GUI. If I run an essentially identical command manually outside freight the pinentry GUI does show up:

$ gpg -abs --use-agent [email protected] --batch --passphrase-fd 1 --passphrase-file /etc/freight.pass --personal-digest-preferences SHA512 -otestfile.sig testfile

Once gpg-agent has cached the password entered via pinentry, freight-cache is able to use the cached credentials. So the only problem is that something in freight-cache / apt.sh prevents it from showing the pinentry prompt.

EDIT: fix a small typo.

unclear if freight is safe against parallel invocation

assume freight-cache is run concurrently - will things go haywire?

btw thanks for working on this - really worth it and very much appreciated!

Working directory not being cleaned after freight add when linking fails

I use freight to add new packages from the docker repos every hour.
The newest deb is downloaded, and added to freight and the downloaded file is then cleaned up.

but it seems like freight is not cleaning up the freight.$$.XXXXX working directory when the hard linking fails:

root@4b8294f34e3d:/archive/vpc-lib# find /freight*
find: '/freight*': No such file or directory
root@4b8294f34e3d:/archive# find freight*
freight.1051.jCEizAxtjB
freight.1051.jCEizAxtjB/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.1051.jCEizAxtjB/ln
freight.1079.FU3ArY99qd
freight.1079.FU3ArY99qd/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.1079.FU3ArY99qd/ln
freight.1120.LIqOUWKXz5
freight.1120.LIqOUWKXz5/ln
freight.1120.LIqOUWKXz5/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.11762.gdhFhhEr3O
freight.11762.gdhFhhEr3O/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.11762.gdhFhhEr3O/ln
freight.11789.ymFtKxHlBt
freight.11789.ymFtKxHlBt/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.11789.ymFtKxHlBt/ln
freight.11828.d6PsFQ3e8c
freight.11828.d6PsFQ3e8c/ln
freight.11828.d6PsFQ3e8c/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.12141.cBCo5pmm6M
freight.12141.cBCo5pmm6M/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.12141.cBCo5pmm6M/ln
freight.12170.CkA94cr3Zw
freight.12170.CkA94cr3Zw/ln
freight.12170.CkA94cr3Zw/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.14438.Nbq2wCFOhk
freight.14438.Nbq2wCFOhk/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.14438.Nbq2wCFOhk/ln
freight.14465.p2wJnYm9EU
freight.14465.p2wJnYm9EU/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.14465.p2wJnYm9EU/ln
freight.14505.52QdohvnQM
freight.14505.52QdohvnQM/ln
freight.14505.52QdohvnQM/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.14780.AIeRHEv4BS
freight.14780.AIeRHEv4BS/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.14780.AIeRHEv4BS/ln
freight.14806.50Cp75ouGx
freight.14806.50Cp75ouGx/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.14806.50Cp75ouGx/ln
freight.14846.aAoGTibI0u
freight.14846.aAoGTibI0u/ln
freight.14846.aAoGTibI0u/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.17113.2b8z2bYE7g
freight.17113.2b8z2bYE7g/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.17113.2b8z2bYE7g/ln
freight.17141.imlM2ErPV3
freight.17141.imlM2ErPV3/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.17141.imlM2ErPV3/ln
freight.17183.k8P4wr0Kqv
freight.17183.k8P4wr0Kqv/ln
freight.17183.k8P4wr0Kqv/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.17456.LwES6G5tbN
freight.17456.LwES6G5tbN/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.17456.LwES6G5tbN/ln
freight.17482.RShXZNIL3u
freight.17482.RShXZNIL3u/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.17482.RShXZNIL3u/ln
freight.17522.LeKlnePhWG
freight.17522.LeKlnePhWG/ln
freight.17522.LeKlnePhWG/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.19885.InGCSColiO
freight.19885.InGCSColiO/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.19885.InGCSColiO/ln
freight.19910.PQYPRhU8nm
freight.19910.PQYPRhU8nm/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.19910.PQYPRhU8nm/ln
freight.19952.QKju2HUJyH
freight.19952.QKju2HUJyH/ln
freight.19952.QKju2HUJyH/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.20135.4AX6TCg1VR
freight.20135.4AX6TCg1VR/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.20135.4AX6TCg1VR/ln
freight.20159.k9sBW8Tqby
freight.20159.k9sBW8Tqby/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.20159.k9sBW8Tqby/ln
freight.20199.XWjDCVYsnk
freight.20199.XWjDCVYsnk/ln
freight.20199.XWjDCVYsnk/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.22814.eX8qgFTLQZ
freight.22814.eX8qgFTLQZ/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.22814.eX8qgFTLQZ/ln
freight.22837.EnED3KMsCK
freight.22837.EnED3KMsCK/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.22837.EnED3KMsCK/ln
freight.22877.siHSFy2gse
freight.22877.siHSFy2gse/ln
freight.22877.siHSFy2gse/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.25489.w9u51Q4RJh
freight.25489.w9u51Q4RJh/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.25489.w9u51Q4RJh/ln
freight.25515.VlZumOkV8O
freight.25515.VlZumOkV8O/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.25515.VlZumOkV8O/ln
freight.25555.3SD5Y3Fdxl
freight.25555.3SD5Y3Fdxl/ln
freight.25555.3SD5Y3Fdxl/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.28163.V9etBEUbi9
freight.28163.V9etBEUbi9/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.28163.V9etBEUbi9/ln
freight.28189.3RNJHVlkzr
freight.28189.3RNJHVlkzr/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.28189.3RNJHVlkzr/ln
freight.28230.5J9uaJ2iy9
freight.28230.5J9uaJ2iy9/ln
freight.28230.5J9uaJ2iy9/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.30839.JXkpTSp5WO
freight.30839.JXkpTSp5WO/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.30839.JXkpTSp5WO/ln
freight.30868.9QmoGzZWZz
freight.30868.9QmoGzZWZz/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.30868.9QmoGzZWZz/ln
freight.30908.HmirAKBbt8
freight.30908.HmirAKBbt8/ln
freight.30908.HmirAKBbt8/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.3731.sa0n0oUmcq
freight.3731.sa0n0oUmcq/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.3731.sa0n0oUmcq/ln
freight.3756.H4SPWcxrM6
freight.3756.H4SPWcxrM6/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.3756.H4SPWcxrM6/ln
freight.3797.2cOI8LI7bI
freight.3797.2cOI8LI7bI/ln
freight.3797.2cOI8LI7bI/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.6406.6R3soNWUw1
freight.6406.6R3soNWUw1/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.6406.6R3soNWUw1/ln
freight.6436.G5jXMdAtFx
freight.6436.G5jXMdAtFx/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.6436.G5jXMdAtFx/ln
freight.6475.TouCZAszdx
freight.6475.TouCZAszdx/ln
freight.6475.TouCZAszdx/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.9082.0b8Mo7YTKs
freight.9082.0b8Mo7YTKs/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.9082.0b8Mo7YTKs/ln
freight.9110.MLe5y84Tzs
freight.9110.MLe5y84Tzs/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.9110.MLe5y84Tzs/ln
freight.9151.pMOBC5LI6q
freight.9151.pMOBC5LI6q/ln
freight.9151.pMOBC5LI6q/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb
freight.9424.CHWrtvxyYU
freight.9424.CHWrtvxyYU/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb
freight.9424.CHWrtvxyYU/ln
freight.9449.pRoSou3kkm
freight.9449.pRoSou3kkm/docker-engine_17.03.0~ce-0~ubuntu-xenial_amd64.deb
freight.9449.pRoSou3kkm/ln
freight.9490.Qy8kSOhARp
freight.9490.Qy8kSOhARp/ln
freight.9490.Qy8kSOhARp/docker-engine_17.03.1~ce-0~ubuntu-xenial_amd64.deb

root@4b8294f34e3d:/archive# cat freight.1051.jCEizAxtjB/ln
ln: failed to create hard link 'apt/mlswarm/docker-engine_17.04.0~ce-0~ubuntu-xenial_amd64.deb': File exists

This problem however, only occurs when freight-add is called useing a subprocess in a python script.

It seems the trap statement in freight-add is not firing correctly?

Upgraded Apt requests the InRelease file

Hi,

A new version of Apt appears to be looking for the InRelease file in the freight repo.

Reading package lists...
E: Failed to fetch InRelease
E: Some index files failed to download. They have been ignored, or old ones used instead.
Ran apt-get update returned 100

I've created a basic example PR as a guide for what I believe the PGP command should be.

$ bin/freight-add test/fixtures/source_1.0.orig.tar.xz apt/example
# [freight] added test/fixtures/source_1.0.orig.tar.xz to apt/example
$ bin/freight-add test/fixtures/source_1.0-1.dsc apt/example
# [freight] added test/fixtures/source_1.0-1.dsc to apt/example
$ bin/freight-add test/fixtures/source_1.0-1.tar.gz apt/example
# [freight] added test/fixtures/source_1.0-1.tar.gz to apt/example
$ bin/freight-cache -v
# [freight] adding source_1.0-1.dsc to pool
# [freight] adding source_1.0-1.tar.gz to pool
$ find var/cache/pool/ -type f
var/cache/pool/example/main/s/source/source_1.0-1.dsc
var/cache/pool/example/main/s/source/source_1.0-1.tar.gz
$ find var/lib -type f
var/lib/apt/example/source_1.0.orig.tar.xz
var/lib/apt/example/source_1.0-1.dsc
var/lib/apt/example/source_1.0-1.tar.gz

It does get added to var/lib/ correctly, but only .orig.tar.gz files are copied to the pool itself.

This seems to stem from https://github.com/freight-team/freight/blob/v0.3.6/lib/freight/apt.sh#L355, and is linked to PR #4.

Batch unlock GPG

Is there some way to automatically answer the "You need a passphrase to unlock the secret key for user" prompt?

freight-team / freight Goto Github PK

freight's People

Contributors

Stargazers

Watchers

Forkers

freight's Issues

Recommend Projects

Recommend Topics

Recommend Org