Giter Site home page Giter Site logo

freebsd / poudriere Goto Github PK

View Code? Open in Web Editor NEW
379.0 43.0 160.0 21.83 MB

Port/Package build and test system

Home Page: https://github.com/freebsd/poudriere/wiki

License: BSD 2-Clause "Simplified" License

Shell 43.85% C 5.18% Awk 1.37% JavaScript 2.70% CSS 3.26% HTML 1.18% Makefile 37.72% M4 0.08% Roff 2.03% Perl 1.31% Python 0.56% Tcl 0.13% Ruby 0.15% Assembly 0.03% Raku 0.02% Lua 0.42%
freebsd shell freebsd-ports build-automation

poudriere's Introduction

Welcome to poudriere!

poudriere is a tool primarily designed to test package production on FreeBSD. However, most people will find it useful to bulk build ports for FreeBSD.

Its goals are to use modern facilities present in FreeBSD (such as ZFS, jails), to be easy to use and to depend only on base.

Where is the documentation?

The canonical documentation is located at:

https://github.com/freebsd/poudriere/wiki

A copy of this documentation could be found in the doc directory.

You can also open the poudriere's manpage, located in the 8th section.

Support

IRC: #poudriere on EFNet Mailing list: [email protected] (lists.freebsd.org)

Getting started with poudriere

  1. Install it by typing ./configure, make and make install at the top-level directory
  2. Copy /usr/local/etc/poudriere.conf.sample to /usr/local/etc/poudriere.conf
  3. Edit it to suit your needs
  4. man poudriere, search for EXAMPLES, read the wiki
  5. Follow "bulk build of binary packages" steps
  6. Enjoy your new fresh binary packages!

Build status

  • Build Status

poudriere's People

Contributors

0mp avatar 5u623l20 avatar aklv avatar allanjude avatar ant1 avatar bapt avatar bdrewery avatar brd avatar brooksdavis avatar dankm avatar driesmp avatar egypcio avatar evadot avatar fel1x-developer avatar igalic avatar jadawin avatar jdmulloy avatar jlaffaye avatar jlduran avatar jrmarino avatar kevans91 avatar linimon avatar madpilot78 avatar mat813 avatar ocochard avatar pizzamig avatar smortex avatar swills avatar vishwin avatar wca avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

poudriere's Issues

poudriere fails on a IPV4-only server

In an IPV4-only environment (kernel source without INET6), some jail commands fail with "unknown parameter: ip6". I wrote a PR on this topic (bin/169028).


bapt added on 2012-06-18 14:49:56 UTC:
I think the best here is to add a new configuration entry in poudriere.conf:

NOIPV4
NOIPV6

I'll do that

zfs data record cannot hold poudriere_data

It would be nice to be able to configure poudriere_data as a zfs data record. If I try it now, logging fails due to mkfifo error.
sh -x output:
====>> Building devel/libtool

  • injail make -C /usr/ports/devel/libtool -VPKGNAME
  • jexec -U root pkg864 make -C /usr/ports/devel/libtool -VPKGNAME
  • PKGNAME=libtool-2.4.2
  • log_start /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log
  • exec
  • [ ! -e /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log.pipe ]
  • mkfifo /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log.pipe
    mkfifo: /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log.pipe: No such file or directory
  • tee /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log
  • export tpid=63454
  • exec
    /usr/local/share/poudriere/test_ports.sh: cannot create /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log.pipe: No such file or directory
    /usr/local/share/poudriere/test_ports.sh: cannot open /mpool/poudriere/poudriere_data/logs/pkg864-DSS-libtool-2.4.2.log.pipe: No such file or directory
  • sig_handler
  • [ 1 -eq 1 ]
  • msg 'Signal caught, cleaning up and exiting'
  • echo '====>> Signal caught, cleaning up and exiting'
    ====>> Signal caught, cleaning up and exiting

bapt added on 2012-06-19 11:54:07 UTC:
I don't get what you mean/try here?

Can you tell more more about what you did?


anonymous added on 2012-06-19 15:45:52 UTC:
Sorry, if I was not clear enough.
I get this error after simply editing poudriere.conf: POUDRIERE_DATA=/mpool/poudriere/poudriere_data

Unlike the portstrees or jails, poudriere doesn't automatically create a data record for poudriere_data via "zfs create".


anonymous added on 2012-06-19 16:13:27 UTC:
even after manually creating "poudriere_data" via zfs create, this error persists. But poudriere uses this new data record somehow:
# find /mpool/poudriere/poudriere_data

/mpool/poudriere/poudriere_data/

/mpool/poudriere/poudriere_data/packages

/mpool/poudriere/poudriere_data/packages/pkg832-default

/mpool/poudriere/poudriere_data/packages/pkg832-default/All


bapt added on 2012-07-28 14:19:03 UTC:
in trunk poudriere_data is not optionnal (and not recommanded anymore)

By default the data are now simple zfs FS treadted like ports and jails.

Make editions

Currently all is building for combination of jail and portstree.
You can specify options, make.conf or for jail, or for jail+portstree.

How about add one more thing - "edition"
So you can have one jail and one ports tree, and still building different things from it (as make.conf could be different, also options tree could be different per edition).

This could spare a lot of space when you need make few package sets with some differences, as no plain jail/portstree copying happening.


bdrewery added on 2012-09-19 18:27:23 UTC:
This will be handled by [d3bdeb6599]

Ability to use a local ports tree

I maintain a local copy of the ports tree with various patches and files
for our software. I did not see an option to poudriere ports to use a
local ports tree (e.g., tarball or other directory). Another option to -m
such as 'tarball' with an accompanying flag to specify the tarball would be nice.

It seems like poudriere ports -c -F with a manual copy would work but it
wasn't obvious where to manually copy until after running the full
poudriere ports -c once (which was a waste of time since I didn't want a
ports tree from the outside).


bapt added on 2012-09-03 13:17:46 UTC:
Wouldn't just a bit a documentation be able to save the situation?

poudriere needs some special metadata on the ports files system and a ports subdirectory (because of portsnap and csup requiring a place for data)

So yes it is not obvious but I do think we can fix this by documentation, no?


bapt added on 2012-10-04 14:50:25 UTC:
This is now possible and documented here http://fossil.etoilebsd.net/poudriere/doc/trunk/doc/use_system_ports_tree.wiki

and using earlier version than [80e88655a3]

autoconf build failure

devel/autoconf was updated.

The old file remained in my PACKAGEDIR.

When the new one was built, and then the pkg repo was ran, it detected that multiple packages were for the devel/autoconf origin and bombed out.

Generating repo.sqlite in /usr/ports/packages/: |pkg-static: Another package already provides devel/autoconf

bdrewery added on 2012-05-15 14:52:35 UTC:
Another example:

I built devel/git.
I then removed the git package in the packages/All directory and rebuilt again.

The following error occurred:

===>  Building package for git-1.7.9.6
Creating package for git-1.7.9.6
====>> Packaging all installed ports
Generating repo.sqlite in /usr/ports/packages/: \pkg-static: sqlite: columns package_id, origin are not unique 591
can not create repository====>> Signal caught, cleaning up and exiting
====>> Stopping jail
====>> Umounting file systems

bapt added on 2012-05-16 05:59:59 UTC:
This is because sanity check still needs some cleanup, I'll refactor it.

Make poudriere states more proper

Would be nice to have poudriere state "done", for not mistaken "idle" between builds and "idle" when all is done.

Or remove intermediate "idle" state, it would automate build end detection


bapt added on 2012-07-16 17:02:45 UTC:
Now the status is next: if not finished for real (in trunk)

poudriere builds packages with incomplete dependencies

poudriere seems to build packages with incomplete dependencies.
Example:
the python27 package created by poudriere shows no dependencies at all (pkg_info -qr). Running 'make run-depends-list' in "/usr/ports/lang/python" shows "/usr/ports/devel/gettext".
Running "pkg_info -qr python27.tbz" (downloaded from ftp.freebsd.org") shows:
@pkgdep libiconv-1.14
@pkgdep gettext-0.18.1.1

Many packages (built by poudriere) which depend on python are missing those dependencies consequently. This issue was revealed after installing these packages into an empty jail and comparing the +CONTENTS files with a locally built INDEX with identical make.conf, port options and portstree as poudriere had used while executing "bulk ..."


bapt added on 2012-07-04 13:38:00 UTC:
Can you confirm this is still happening with newer version of poudriere? (like 1.5.4?)
anonymous added on 2012-07-04 15:40:42 UTC:
From where can I get a newer version?
anonymous added on 2012-07-05 09:23:13 UTC:
after updating the portstree and reinstalling poudriere (version 1.5.4) I rebuilt lang/python27 and the dependencies are ok now. BTW. Is there a way of rebuilding an existent package without deleting it first? (poudriere removed all my python related packages prior to build, even with -k set)
bapt added on 2012-07-07 00:05:26 UTC:
This is a feature of poudriere, because we can't 100% trust the bump (which are often forgotten) of revision so with -k all the ports that depends on the port which have a newer version are rebuilt.

For the other question, currently force rebuilding a port is not possible, but could be a feature request :)

Do not hesitate to open a new ticket "feature request" if you desire this feature.

Not clear comments regarding RESOLV_CONF in poudriere.conf.sample

The comments regarding RESOLV_CONF is not clear.
uncommenting and configuring this variable is mandatory for downloading sources.
But the current comments says "If you want to use a custom resolv.conf in your jails, set the path to this file on the host system below.".
=> I didn't want a custom resolv.conf because I'm happy with my host system resolv.conf Then I didn't un-comment this line and jail's poudriere failed to download sources.
By reading the comments, I didn't understand that it's mandatory to configure RESOLV_CONF to "/etc/resolv.conf" at minimum.


bapt added on 2012-05-06 23:08:39 UTC:
Well RESOLV_CONF is not mandatory if you use a proxy or something like that, but still I'll update the comment

sparc64: linprocfs doesn't exist

[root@sparc]~#poudriere bulk -f /root/pkglist_srv -j 90sparc64srv
kldload: can't load linprocfs: No such file or directory

On sparc64 there is none linux related modules (linuxprocfs.ko, linux.ko).

Should add an ugly "if uname -m" somewhere on poudriere ?


olivier added on 2012-08-28 22:39:10 UTC:
Here is a little patch for linux modules: http://pastebin.com/0Vt1TTGW

But I've got still a problem to solve:

[root@sparc]#poudriere bulk -f /root/pkglist_srv -j 90sparc64srv
====>> Mounting system devices for 90sparc64srv
/etc/resolv.conf -> /usr/local/poudriere/jails/90sparc64srv/etc/resolv.conf
====>> Starting jail 90sparc64srv
====>> Mounting ports filesystems for 90sparc64srv
mount: /usr/local/poudriere/jails/90sparc64srv/usr/ports/distfiles: No such file
or directory
====>> Umounting file systems
Failed to mount the distfile directory

stable/9 jail breaks when building security/nss

I created a jail using svn mode to track stable/9: mostly this works fine,
but security/nss fails because of the '/' in stable/9 -- somehow this text replaces the usual value of ${OPSYS} in make(1)

[...]
===> Configuring for nss-3.13.5
===> Building for nss-3.13.5
cd ../coreconf ; gmake
gmake[1]: Entering directory /wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf' cd nsinstall; gmake export gmake[2]: Entering directory/wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf/nsinstall'
gmake[2]: Nothing to be done for export'. gmake[2]: Leaving directory/wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf/nsinstall'
cd nsinstall; gmake libs
gmake[2]: Entering directory /wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf/nsinstall' mkdir: FreeBSDstable: No such file or directory gmake[2]: *** [FreeBSDstable/9_OPT.OBJ/nsinstall.o] Error 1 ^^^^^^^^ gmake[2]: Leaving directory/wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf/nsinstall'
gmake[1]: *** [libs] Error 2
gmake[1]: Leaving directory `/wrkdirs/usr/ports/security/nss/work/nss-3.13.5/mozilla/security/coreconf'
gmake: *** [build_coreconf] Error 2
*** [do-build] Error code 1

Stop in /usr/ports/security/nss.

Haven't tried poudriere-devel yet.


bapt added on 2012-08-19 13:29:10 UTC:
This is now really fixed in poudriere-devel

allbsd.org snapshots

(re@ did provide "official" monthly snapshots in the past, but they were usually a few months old, not for every month and only in ISO)

allbsd.org provides daily snapshots for

HEAD
RELENG_9
RELENG_8

An example URL looks like this:

https://pub.allbsd.org/FreeBSD-snapshots/i386-i386/10.0-HEAD-20120729-JPSNAP/ftp/base.txz

and a list of available snapshots:

https://pub.allbsd.org/FreeBSD-snapshots/i386-i386/

Issues:

  1. There is currently no way to specify complete URL prefix in config, not just host/protocol. A few examples:

    FREEBSD_URL_PREFIX='https://pub.allbsd.org/FreeBSD-snapshots/${ARCH}-${ARCH}/${VERSION}-JPSNAP/ftp/'

    FREEBSD_URL_PREFIX='ftp://ftp.freebsd.org/pub/FreeBSD/releases/$(
    case ${ARCH} in
    powerpc64)
    echo powerpc ;;
    *)
    echo ${ARCH} ;;
    esac)/${ARCH}/${VERSION}/'

  2. poudriere could try locating latest snapshot, e.g. use date(1), decrement a day until fetch(1) succeeds and stop, say, after 5 retries as specified in config. It could be implemented by allowing user to override common functions, e.g.

    my_fetch_file() {
    local i
    case ${VERSION} in
    -latest)
    until [ $((i+=1)) -ge ${RETRY_LIMIT:-6} ]; do
    VERSION="${VERSION+${VERSION%-
    }-$(date -v-$((i-1))d +%Y%m%d)}"
    eval fetch -p -o $1 $2 && break
    done
    ;;
    *)
    eval fetch -p -o $1 $2
    ;;
    esac
    }

    debug

    FREEBSD_URL_PREFIX='https://pub.allbsd.org/FreeBSD-snapshots/${ARCH}-${ARCH}/${VERSION}-JPSNAP/ftp/'
    VERSION=10.0-HEAD-latest
    ARCH=i386
    my_fetch_file /tmp/ "${FREEBSD_URL_PREFIX}/doc.txz"

Workaround:

FREEBSD_HOST=https://pub.allbsd.org/

Apply the following patch and specify version explicitly, e.g.

$ poudriere jail -c -j 10i386 -v 10.0-HEAD-20120729 -a i386 -m ftp

diff --git a/src/poudriere.d/jail.sh b/src/poudriere.d/jail.sh
index c306d34..7c6f66e 100644
--- a/src/poudriere.d/jail.sh
+++ b/src/poudriere.d/jail.sh
@@ -1,4 +1,5 @@
#!/bin/sh
+# -- tab-width: 4 --

usage() {
echo "poudriere jail [parameters] [options]
@@ -192,7 +193,14 @@ install_from_ftp() {

if [ ${VERSION%%.*} -lt 9 ]; then
    msg "Fetching sets for FreeBSD ${VERSION} ${ARCH}"
  •   case ${FREEBSD_BASE} in
    
  •       _allbsd.org_)
    
  •           URL="${FREEBSD_HOST}/pub/FreeBSD-snapshots/${ARCH}-${ARCH}/${VERSION}-JPSNAP/ftp/"
    
  •           ;;
    
  •       *)
            URL="${FREEBSD_BASE}/pub/FreeBSD/releases/${ARCH}/${VERSION}"
    
  •           ;;
    
  •   esac
    DISTS="base dict src"
    [ ${ARCH} = "amd64" ] && DISTS="${DISTS} lib32"
    for dist in ${DISTS}; do
    
    @@ -223,7 +231,14 @@ install_from_ftp() {
    tar --unlink -xpf - -C ${JAILBASE}/${APPEND} || err 1 " Fail" && echo " done"
    done
    else
  •   case ${FREEBSD_BASE} in
    
  •       _allbsd.org_)
    
  •           URL="${FREEBSD_HOST}/pub/FreeBSD-snapshots/${ARCH}-${ARCH}/${VERSION}-JPSNAP/ftp/"
    
  •           ;;
    
  •       *)
            URL="${FREEBSD_BASE}/pub/FreeBSD/releases/${ARCH}/${VERSION}"
    
  •           ;;
    
  •   esac
    DISTS="base.txz src.txz"
    [ ${ARCH} = "amd64" ] && DISTS="${DISTS} lib32.txz"
    for dist in ${DISTS}; do
    

bapt added on 2012-08-16 14:09:30 UTC:
implemented another way: poudriere jail -c -m allbsd -v HEAD poudriere jail -c -m allbsd -v RELENG_9 poudriere jail -c -m allbsd -v RELENG_8 etc

man page typo

Here is an extract of poudriere man page:

Create a flat text file in which you put the ports you want to see built
by poudriere.

       echo  sysutils/screen" > ~/pkglist"
       echo  editors/vim" >> ~/pkglist"

=> I beleive there are some missplaced quote characters.

SET options dir is not mounted

There is a discrepancy between man page and implementation:

Man page says options dir are mounted in the following order
/usr/local/etc/poudriere.d/$jailname-$setname-options
/usr/local/etc/poudriere.d/$setname-options
/usr/local/etc/poudriere.d/$jailname-options
/usr/local/etc/poudriere.d/options
( could be made more clear if they are additive or exclusive )
but /usr/local/etc/poudriere.d/$jailname-options is mounted even though /usr/local/etc/poudriere.d/$jailname-$setname-options and usr/local/etc/poudriere.d/$setname-options exists.


bdrewery added on 2012-10-16 17:50:30 UTC:
It uses the first found.

I just tested this and it seems ok.

The order that is checked:

# poudriere testport -j 90i386 -p dev -z test -o ports-mgmt/portmaster
...
+ [ -n -test -a -d /usr/local/share/poudriere/../../etc/poudriere.d/90i386-test-options ]
+ [ -d /usr/local/share/poudriere/../../etc/poudriere.d/test-options ]
+ [ -d /usr/local/share/poudriere/../../etc/poudriere.d/90i386-options ]
+ [ -d /usr/local/share/poudriere/../../etc/poudriere.d/options ]
# cd /usr/local/etc/poudriere.d
# mkdir 74i386-options 90i386-options 90i386-test-options options test-options
# echo *options*
74i386-options 90i386-options 90i386-test-options options test-options
# poudriere testport -j 90i386 -p dev -z test -o ports-mgmt/portmaster
...
====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/90i386-test-options
# rmdir /usr/local/etc/poudriere.d/90i386-test-options
# poudriere testport -j 90i386 -p dev -z test -o ports-mgmt/portmaster
...
====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/test-options
# rmdir /usr/local/etc/poudriere.d/test-options
# poudriere testport -j 90i386 -p dev -z test -o ports-mgmt/portmaster
...
====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/90i386-options
# rmdir /usr/local/etc/poudriere.d/90i386-options
# poudriere testport -j 90i386 -p dev -z test -o ports-mgmt/portmaster
...
====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/options

bapt added on 2013-01-06 14:03:45 UTC:
This is what is expected

poudriere-2.2 always rebuilds mail/mailman

Every single time I use 'poudriere bulk -f /root/pkg-buildlist -j 90amd64' and if mail/mailman is in my pkg-buildlist, it rebuilds mailman. Thankfully mailman is small so it finishes quickly, this would be much more annoying if it happened on a large port.
====>> Options changed, deleting: mailman-2.1.14_6.txz

My /usr/local/etc/poudriere.d/90amd64-make.conf contains:
WITH_PKGNG=yes
WITH_MYSQL_VER=50
WANT_PGSQL_VER=84
DEFAULT_PHP_VER=53
PERL_VERSION=5.16.0
WITHOUT="CUPS"
WITHOUT_CUPS=yes
.if ${.CURDIR:M*/mail/mailman}
MM_USERNAME=postlocal
MM_USERID=2012
MM_GROUPNAME=postlocal
MM_GROUPID=2012
MAIL_GID=postlocal
GID_FILES=/usr/ports/distfiles/decs/GIDs-postlocal
UID_FILES=/usr/ports/distfiles/decs/UIDs-postlocal
.endif

cat /usr/ports/distfiles/decs/GIDs-postlocal

postlocal:*:2012:

cat /usr/ports/distfiles/decs/UIDs-postlocal

postlocal:*:2012:2012::0:0:Postfix Local Delivery:/home/postlocal:/bin/false

more /usr/local/etc/poudriere.d/options/mailman/options

This file is auto-generated by 'make config'.

Options for mailman-2.1.14_6

_OPTIONS_READ=mailman-2.1.14_6
_FILE_COMPLETE_OPTIONS_LIST=DOCS HTDIG INTEGRATION MTA NAMAZU2 NLS SENDMAIL EXIM3 EXIM4 POSTFIX COURIER
OPTIONS_FILE_SET+=DOCS
OPTIONS_FILE_UNSET+=HTDIG
OPTIONS_FILE_UNSET+=INTEGRATION
OPTIONS_FILE_UNSET+=MTA
OPTIONS_FILE_UNSET+=NAMAZU2
OPTIONS_FILE_SET+=NLS
OPTIONS_FILE_UNSET+=SENDMAIL
OPTIONS_FILE_UNSET+=EXIM3
OPTIONS_FILE_UNSET+=EXIM4
OPTIONS_FILE_SET+=POSTFIX
OPTIONS_FILE_UNSET+=COURIER


bdrewery added on 2012-11-03 05:33:53 UTC:
The problem is that POSTFIX is selected, but it depends on the MTA option to be enabled. Run 'options' again, enable MTA as well, then also select POSTFIX.

This is a usability bug with the new options framework. It should not allow you to select the child option, without the parent MTA in this case. It's something I keep meaning to add a warning for.

You can see this by running 'make config' in /usr/ports/mail/mailman, selecting only POSTFIX (not MTA), then running pretty-print-config / showconfig:

$ make config
$ make pretty-print-config
+DOCS -HTDIG -INTEGRATION -MTA -NAMAZU2 +NLS MTA( -SENDMAIL -EXIM3 -EXIM4 -POSTFIX -COURIER )
$ make showconfig|egrep "(MTA|POSTFIX)"
     MTA=off: integrate with a MTA
====> Options available for the single MTA: you have to select exactly one of them
     POSTFIX=off: for use with postfix

Then selecting both:

$ make config
$ make pretty-print-config
+DOCS -HTDIG -INTEGRATION +MTA -NAMAZU2 +NLS MTA( -SENDMAIL -EXIM3 -EXIM4 +POSTFIX -COURIER )
$ make showconfig|egrep "(MTA|POSTFIX)"
     MTA=on: integrate with a MTA
====> Options available for the single MTA: you have to select exactly one of them
     POSTFIX=on: for use with postfix

poudriere misses left over file that tinderbox reports.

When building databases/glom with poudriere-devel and tinderbox, only tinderbox sees a leftover file. Please note that this is with glom 1.12.2 from ports. I got a new version lined up 1.20.2 that doesn't exhibit this issue. I have no idea why though.

It might be good to look at why poudriere doesn't report this.

Tinderbox:
=== Checking filesystem state
list of extra files and directories in / (not present before this port was installed but present after it was deinstalled)
5394553 1 -rw-r--r-- 1 root wheel 281208 Oct 23 12:58 usr/local/share/icons/hicolor/icon-theme.cache
Deleting py27-gnome-extras-2.25.3_13

poudriere bulk -j 90amd64 -w -f /root/poudriere-short.txt
content of poudriere-short.txt is "databases/glom"


bdrewery added on 2013-03-11 12:07:11 UTC:
I think this is a tinderbox/pointyhat bug. I had similar frustrations with ports/170490 with this file.

IPv4/IPv6 detection on pre-8.3 versions

IPv4/v6 detection via kern.features does not work on pre-8.3 versions, leaving jails without network access.

FYI, kern.features.inet and kern.features.inet6 were added with this commit :
http://svnweb.freebsd.org/base?view=revision&revision=222272

and then MFC'ed with the following one :
http://svnweb.freebsd.org/base?view=revision&revision=223272

so, they start being available with 8.3-RELEASE.

The following patch to src/poudriere.d/common.sh is an attempt to fix the problem (anyway, I am not sure whether testing the presence of net.inet and net.inet6 is the right way to go) :


--- src/poudriere.d/common.sh
+++ src/poudriere.d/common.sh
@@ -1,10 +1,10 @@
#!/bin/sh

zfs namespace

NS="poudriere"
-IPS="$(sysctl -n kern.features.inet 2>/dev/null || echo 0)$(sysctl -n kern.features.inet6 2>/dev/null || echo 0)"
+IPS="$(sysctl -n kern.features.inet 2>/dev/null || (sysctl -n net.inet 1>/dev/null 2>&1 && echo 1) || echo 0)$(sysctl
-n kern.features.inet6 2>/dev/null || (sysctl -n net.inet6 1>/dev/null 2>&1 && echo 1) || echo 0)"

err() {
if [ $# -ne 2 ]; then
err 1 "err expects 2 arguments: exit_number "message""
fi



bdrewery added on 2012-08-12 14:55:58 UTC:
I discussed this with bz and it seems it should be ok. However, 8.1 and 8.2 are now EoL (www.freebsd.org/security), which leaves 7.4 to need this. The advertised minimum version required is 8.3.
martymac added on 2012-08-13 11:23:25 UTC:
Patch applied in commit [0a211066a3]

bulk.sh doesn't catch error from pkg-static repo command

I'm running a poudriere build for some selfmade ports. I had a problem with one port and when poudriere started to create the pkgng repository via 'injail pkg-static repo /usr/ports/packages/' it failed and the returncode was ignored it seems:

====>> Creating pkgng repository
tar: Removing leading '/' from member names
Generating repo.sqlite in /usr/ports/packages/: pkg-static: fread(/usr/ports/packages/All/rrdtool-minimal-1.4.7_2.txz): |/-|/-|/Bad file descriptor
-|/-|/done!
====>> Cleaning up
====>> Umounting file systems

poudriere 2.2, pkg 1.0.2


bdrewery added on 2012-11-16 19:57:54 UTC:
It wasn't ignored, it immediately cleaned up and exited instead of display build stats.

What were you expecting to see?

some minor update of man page

I found some minor typos in poudriere.8. Here is the diff output:

=== root@dsssrvt4 (pts/0) /home/poudriere 65(2) -> diff poudriere.8.orig poudriere.8
64c64

< This command allows you to manage different portstree which will be used

This command allows you to manage different portstrees which will be used
76c76

< This command does not take arguments and run jobs from the queue

This command does not take arguments and runs jobs from the queue
116a117,118
.It Fl t
add some testing
261d262
< .El
277d277
< .El
351,352c351,352
< package dir (pourdriere_data/packages/jailname), then it mounts the

< ~/ports-cvs/mybeautifulporttotest (nullfs) it builds all the dependency

package dir (poudriere_data/packages/jailname), then it mounts the
~/ports-cvs/mybeautifulporttotest (nullfs) it builds all the dependencies

Besides this i have a question concerning jail sizes:
Line #293 reads: A jail will take approximately 3GB of space.
Line #302 reads: This command will fetch and install a minimal jail, small (~400MB) ...

So which jail size is to be expected? In my environment one jail uses about 720 (32-bit) to 800 MBytes (64-bit)


bapt added on 2012-06-18 16:08:27 UTC:
diff -u is often preferable because more readable.

Still thanks

untitled

Do you have WITH_NEW_XORG activated?


bapt added on 2012-09-20 09:58:17 UTC:
erg

[patch] The -J flag is not recognized

I have just installed ports-mgmt/poudriere-devel from the Ports Collection and
tried to use the parallel build option in bulk mode. However I got this:

% poudriere bulk -f bulkfile -t -J 2 -j 90i386 -p portstee
Illegal option -J

It seems the 'J' flag is not recognized when parsing the options. Thus I would
suggest the following patch (which works for me):

--- bulk.sh.orig 2012-08-16 22:54:52.000000000 +0200
+++ bulk.sh 2012-08-16 23:25:19.000000000 +0200
@@ -103,7 +103,7 @@
CLEAN=0
. ${SCRIPTPREFIX}/common.sh

-while getopts "f:j:cn:p:ts" FLAG; do
+while getopts "f:j:J:cn:p:ts" FLAG; do
case "${FLAG}" in
t)
export PORTTESTING=1


bdrewery added on 2012-08-17 13:28:17 UTC:
Merged, thanks!

Make a bit faster build in specific conditions

All builds starts from clean jail.
After that we install pkg;
After that real build starts.
After each package creation, jail goes to clean state.

I propose at this point, don't rollback to clean state, but instead check if there in queue exist port, which depends only on all installed packages , if so

  • then continue with new one, without rollback to @clean snapshot.

It's like:

-find list of not-built packages which have exactly same list of depends packages
-if list not empty, create temp snapshot (with name probably = hash from sorted list of packages)
-one by one, begin build next package (note that after this one built, same procedure with checking "what's next" will be done)
-after each package building -rollback to this temp snapshot.


anonymous added on 2012-07-14 18:00:19 UTC:

Description & Comments:
All builds starts from clean jail.
After that we install pkg;
After that real build starts.
After each package creation, jail goes to clean state.

I propose at this point, don't rollback to clean state, but instead check if there in queue exist port, which depends only on all installed packages , if so

  • then continue with new one, without rollback to @clean snapshot.

It's like:

-find list of not-built packages which have exactly same list of depends packages
-if list not empty, create temp snapshot (with name probably = hash from sorted list of packages)
-one by one, begin build next package (note that after this one built, same procedure with checking "what's next" will be done)
-after each package building -rollback to this temp snapshot.

As I thought more - this way would be mostly efficient:

  1. Before all build, build dependency graph;
  2. Find all nodes (pacakges) which have more than one dependants
  3. For all such nodes build some unique hash ( for example, sort all package pathes, concat them and take md5)
  4. for such packages, create intermediate snapshot after install.

This will be most efficient way, there wouldn't be useless snapshots.

Poudriere launches several build_pkg per builder at the same time

Poudriere launches several build_pkg per builder because of pgrep failing to find the process of a pidfile.

This error can be reproduced by setting PARALLEL_JOBS=1 in poudriere.conf ; you'll see that poudriere will launch several builds in builder #1 instead of building ports sequentially. It mostly results in machine lockups as well as build failures because several jexec are launched in the same jail while the jail would eventually be dying.

This problem has been encountered on FreeBSD 8.1-RELEASE, but should probably appear on FreeBSD 9.0 too as I could not manage to get pgrep display a process from a pid file on 9.0 :

$ echo $$ > test.pid && pgrep -F test.pid
$ echo $?
1

Here is a simple patch for common.sh :

--- common.sh.orig 2012-09-26 14:01:15.000000000 +0200
+++ common.sh 2012-09-27 16:09:24.000000000 +0200
@@ -804,7 +804,7 @@
fs="${JAILFS}/build/${j}"
name="${JAILNAME}-job-${j}"
if [ -f "${JAILMNT}/poudriere/var/run/${j}.pid" ]; then

  •                           if pgrep -qF "${JAILMNT}/poudriere/var/run/${j}.pid" >/dev/null 2>&1; then
    
  •                           if ps $(cat "${JAILMNT}/poudriere/var/run/${j}.pid") >/dev/null 2>&1; then
                                    continue
                            fi
                            build_stats
    


bdrewery added on 2012-09-27 19:09:57 UTC:
the -a option to pgrep(1) will fix your test case. It seems to work in this test script fine though without -a.
#! /bin/sh

build_pkg() {
        sleep 500
}

build_pkg &
pid=$!
echo $pid > /tmp/test.pid
if pgrep -F /tmp/test.pid; then
        echo "RUNNING!"
fi
kill $pid

Output:

RUNNING!

And another:

#! /bin/sh

build_pkg() {
        sleep 5
}

build_pkg &
pid=$!
echo $pid > /tmp/test.pid
while true; do
        if pgrep -qF /tmp/test.pid; then
                echo "RUNNING!"
                continue
        fi
        break
done


bdrewery added on 2012-09-27 22:25:07 UTC:
I tested this on 8.3 with success. I don't have any 8.1. 7.4 seems OK.

I would not recommend using 8.1 as it is EoL now and below the advertised >=8.3 poudriere requirement.


martymac added on 2012-09-28 09:19:36 UTC:

the -a option to pgrep(1) will fix your test case

Thanks ! You're right : my test case was using the parent shell of pgrep as the test PID, but without -a, pgrep excludes its own PID and ancestors (the shell) from being displayed. We are not in the same situation in the common.sh script ; my test was not appropriate, sorry.

Anyway, I've found the key problem, which is very much simpler as I thought at the beginning : the -q option of pgrep only appeared on 8.3 :

http://svnweb.freebsd.org/base?view=revision&revision=216814

As I kept the stderr redirection during my tests, I could not detect it. Removing the -q option fixes the build on 8.1 (anyway, why do we need it since stderr and stdout are redirected to /dev/null ?).

I know 8.1 is EOL, but we are not ready yet to switch to 8.3 at work, that's why I keep trying to make poudriere work with that version.

Shall we remove the -q flag ?


bdrewery added on 2012-09-28 15:02:25 UTC:
I noticed the -q issue as well when checking 7.4.

Yes I will remove the -q :)

$BASEFS under $ZPOOL breaks poudriere after bulk

You might want to put this in your docs, or perhaps there is a way you can code to handle it.

After a bulk run, the build file systems would not be unmounted after the bulk finished. This would leave things in a state where poudriere could not run another bulk or do a ports tree update until things were manually cleaned up.

I found the problem to be due to BASEFS living in ZPOOL on the file system.

I had this in poudriere.conf:

ZPOOL=zfs-pool
ZROOTFS=/poudriere
FREEBSD_HOST=http://ftp.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere

On the OS, /usr/local was a symlink to /zfs-pool/local

That caused the build filesystems to be mounted (displayed by df) as:

/zfs-pool/local/poudriere/jails/90amd64_p5_8/build/01/dev

Instead of

/usr/local/poudriere/jails/90amd64_p5_8/build/01/dev that poudriere expected

That made this line (#391) in jail_stop() of common.sh

 mount | awk -v mnt="${MASTERMNT:-${JAILMNT}}/" 'BEGIN{ gsub(/\//, "\\\/", mnt); } { if ($3 ~ mnt && $1 !~ /\/dev\/md/ ) { print $3 }}' |  sort -r | xargs umount -f || :

Not unmount anything because all the mounts were under /zfs-pool/local/.... Instead of /usr/local/...

Once I moved /usr/local out from underneath /zfs-pool, everything works as expected.

Like I said up top, it is a show stopper, but not really poudriere's fault.


bapt added on 2012-10-04 14:08:51 UTC:
Should be fixed can you confirm? (I'll update the -devel port soon
bdrewery added on 2012-10-16 17:43:14 UTC:
This fix has been released to ports in 2.2

csup cvstag and FreeBSD version

Hello,

I've created a poudriere jail using the csup method. As far I can see, poudriere use the -v version as the CVS TAG so I've built the jail vith RELENG_9 as version. That works fine (csup + buildworld):

poudriere jail -i -j 90amd64

Jailname: 90amd64
FreeBSD Version: RELENG_9
FreeBSD arch: amd64
Status: 0

The problem is that this FreeBSD Version (RELENG_9 here) is used by some port. For example sysutils/lsof fails to build in the jail with :

===> Returning to build of lsof-4.87.a,7
===> Cleaning for lsof-4.87.a,7
...
=> SHA256 Checksum OK for lsof_4.87A.freebsd.tar.bz2.
===> Patching for lsof-4.87.a,7
===> Configuring for lsof-4.87.a,7
Unknown FreeBSD release: RELENG_9

So I think we need something else to specify the CVS tag to use with csup, the CVS tag is not related to the FreeBSD version (9.1-PRERELEASE => RELENG_9 and so on)

Thanks, regards.

add svn+file and svn+https protos

svn+file: useful to save traffic if you have a local svn mirror
svn+https: useful if you don't have svn+ssh access but want to fetch sources securely

All packages deleted if SIGINTing early

I hit ^C when it was "Calculating ports order and dependencies" and had all my packages deleted.

====>> Umounting file systems
egrep: /usr/jails/poudriere/jails/82i386/tmp/orderport.5qFjoJ: No such file or directory
jexec: jail "82i386" is dying
jexec: jail "82i386" is dying
jexec: jail "82i386" is dying
... snip ...

jexec: jail "82i386" is dying
jexec: jail "82i386" is dying
jexec: jail "82i386" is dying
tsort: /usr/jails/poudriere/jails/82i386/tmp/orderport1.lYee9: No such file or directory
====>> Sanity checking the repository
====>> net/6tunnel does exists anymore, delete staled 6tunnel-0.11.r2_2.txz
====>> irc/bitchx does exists anymore, delete staled BitchX-1.1.0.1_4.txz
====>> net/GeoIP does exists anymore, delete staled GeoIP-1.4.8_2.txz
====>> www/mod_evasive does exists anymore, delete staled ap22-mod_evasive-1.10.1_1.txz
====>> www/apache22 does exists anymore, delete staled apache-worker-2.2.22_5.txz
====>> devel/apr1 does exists anymore, delete staled apr-ipv6-devrandom-gdbm-db42-1.4.5.1.3.12_1.txz
====>> archivers/arc does exists anymore, delete staled arc-5.21p.txz
====>> archivers/arj does exists anymore, delete staled arj-3.10.22_4.txz
====>> devel/autoconf213 does exists anymore, delete staled autoconf-2.13.000227_6.txz
====>> devel/autoconf does exists anymore, delete staled autoconf-2.69.txz
====>> devel/autoconf-wrapper does exists anymore, delete staled autoconf-wrapper-20101119.txz
====>> devel/automake does exists anymore, delete staled automake-1.12.txz
====>> devel/automake14 does exists anymore, delete staled automake-1.4.6_6.txz
====>> devel/automake-wrapper does exists anymore, delete staled automake-wrapper-20101119.txz
====>> shells/bash does exists anymore, delete staled bash-static-4.2.28.txz
====>> x11/bigreqsproto does exists anymore, delete staled bigreqsproto-1.1.1.txz
====>> devel/binutils does exists anymore, delete staled binutils-2.22_1.txz
====>> devel/bison does exists anymore, delete staled bison-2.5,1.txz
====>> irc/bitlbee does exists anymore, delete staled bitlbee-3.0.5.txz
====>> irc/bnc does exists anymore, delete staled bnc-2.9.4.txz
====>> sysutils/bsdadminscripts does exists anymore, delete staled bsdadminscripts-6.1.1_1.txz
====>> security/bsmtrace does exists anymore, delete staled bsmtrace-1.3_1.txz
====>> devel/buildbot does exists anymore, delete staled buildbot-0.8.3p1.txz
... snip ...
many more

^C

bdrewery added on 2012-06-22 20:04:58 UTC:
Fixed

Auto options rebuilding

Have poudriere be smart about options. If the existing package has a set of OPTIONS and the make.conf or port defaults has new OPTIONS, then rebuild that package.


bapt added on 2012-05-06 14:54:55 UTC:
That would be a nice feature, just be careful that this can only be doable with pkgng.

So embed in some if [ $PKGNG -eq 1 ]; then ... fi stuff


bdrewery added on 2012-06-18 15:49:44 UTC:
This is done, just waiting on ports/168936
bdrewery added on 2012-06-20 18:07:15 UTC:
Implemented in 042aa0bd36

poudriere cannot create packages from some ports

poudriere is not able to build a package in some cases (build in "normal" environment works ok).
One example is "textproc/odt2txt". Here is the build log:

=== root@dsssrvt4 (pts/0) /home/poudriere 93(2) -> echo "textproc/odt2doc" > tt.pkgs
=== root@dsssrvt4 (pts/0) /home/poudriere 94(2) -> poudriere bulk -k -f tt.pkgs -p DSS -j pkg864
====>> Mounting devfs
====>> Mounting /proc
====>> Mounting linuxfs
/etc/resolv.conf -> /mpool/poudriere/jails/pkg864/etc/resolv.conf
====>> Starting jail pkg864
====>> Populating LOCALBASE
====>> Calculating ports order and dependencies
make: chdir textproc/odt2doc: No such file or directory
====>> Caching missing port versions
make: chdir /usr/ports/textproc/odt2doc: No such file or directory
====>> Signal caught, cleaning up and exiting
====>> Stopping jail
====>> Umounting file systems
=== root@dsssrvt4 (pts/0) /home/poudriere 95(2) -> echo "textproc/odt2txt" > tt.pkgs
=== root@dsssrvt4 (pts/0) /home/poudriere 96(2) -> poudriere bulk -k -f tt.pkgs -p DSS -j pkg864
====>> Mounting devfs
====>> Mounting /proc
====>> Mounting linuxfs
/etc/resolv.conf -> /mpool/poudriere/jails/pkg864/etc/resolv.conf
====>> Starting jail pkg864
====>> Populating LOCALBASE
====>> Calculating ports order and dependencies
====>> Caching missing port versions
====>> Sanity checking the repository
====>> Deleting stale symlinks
====>> Cleaning the build queue
====>> Cleaning up wrkdir
====>> Building textproc/odt2txt
===> odt2txt-0.4_1 depends on executable: gmake - not found
===> Verifying install for gmake in /usr/ports/devel/gmake
===> Installing existing package /usr/ports/packages/All/gmake-3.82.tbz
===> Returning to build of odt2txt-0.4_1
===> odt2txt-0.4_1 depends on shared library: iconv.3 - found
===> Cleaning for odt2txt-0.4_1
===> License GPLv2 accepted by the user
===> License GPLv2 accepted by the user
=> SHA256 Checksum OK for odt2txt-0.4.tar.gz.
===> License GPLv2 accepted by the user
===> Extracting for odt2txt-0.4_1
=> SHA256 Checksum OK for odt2txt-0.4.tar.gz.
===> Patching for odt2txt-0.4_1
/usr/bin/sed -i.bak -e 's#/usr/local#/usr/local#g' /wrkdirs/usr/ports/textproc/odt2txt/work/odt2txt-0.4/Makefile
===> odt2txt-0.4_1 depends on executable: gmake - found
===> odt2txt-0.4_1 depends on shared library: iconv.3 - found
===> Configuring for odt2txt-0.4_1
===> Building for odt2txt-0.4_1
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o odt2txt.o odt2txt.c
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o regex.o regex.c
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o mem.o mem.c
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o strbuf.o strbuf.c
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o kunzip/fileio.o kunzip/fileio.c
cc -O2 -DICONV_CHAR="const char" -I/usr/local/include -I/usr/local/include -c -o kunzip/zipfile.o kunzip/zipfile.c
cc -o odt2txttbz -L/usr/local/lib -L/usr/local/lib odt2txt.o regex.o mem.o strbuf.o kunzip/fileio.o kunzip/zipfile.o -lz -liconv
===> Installing for odt2txt-0.4_1
===> odt2txt-0.4_1 depends on shared library: iconv.3 - found
===> Generating temporary packing list
===> Checking if textproc/odt2txt already installed
install -s -o root -g wheel -m 555 /wrkdirs/usr/ports/textproc/odt2txt/work/odt2txt-0.4/odt2txt /usr/local/bin/odt2txt
install: /wrkdirs/usr/ports/textproc/odt2txt/work/odt2txt-0.4/odt2txt: No such file or directory
*** Error code 71

Stop in /usr/ports/textproc/odt2txt.
====>> No package built, no need to update INDEX
====>> Stopping jail
====>> Umounting file systems
====>> 0 packages built, 1 failures, 0 ignored
====>> Failed ports: install:textproc/odt2txt


bapt added on 2012-07-07 00:02:15 UTC:
Sorry this is not a poudriere bug, but a port bug, should be reported to the port maintainer.

Thanks for reporting

Support for proxy variables in environment

We have our *BSD boxes on internal network space. To get outside (portsnap, pkg) we have HTTP_PROXY & FTP_PROXY defined in our user environment.

You are sanitizing the shell ENV and knocking that out. Perhaps a config variable?

In order to get poudriere to be able to reach portsnap servers, the following:

HTTP_PROXY=http://yyy.xxxx.net:3128
FTP_PROXY=http://yyy.xxxx.net:3128

export HTTP_PROXY
export FTP_PROXY

was added to ${SCRIPTPREFIX}/common.sh

tia,

ben


bapt added on 2012-09-29 02:07:00 UTC:
Imho this is more a documentation problem, if you add the two export line to poudriere.conf directly, it will work.

If you comfirm that it fixes the problem for you I ll update the documentation.

Thanks for reporting.
Bapt


bloyall added on 2012-10-02 22:01:31 UTC:
Adding the export lines to poudriere.conf does indeed work, thank you.

ben

can't run build

If I run a build, I get an error about
mtree: /usr/ports/Templates/BSD.local.dist: No such file or directory

debug output

f2d169d8-20d2-41d4-8e43-8a9fc5a2b509 ) 0 # poudriere bulk -f portlist-current-php53-mysql55 -j 90amd64 [20:55]

  • set -e
  • realpath /usr/local/share/poudriere/bulk.sh
  • SCRIPTPATH=/usr/local/share/poudriere/bulk.sh
  • dirname /usr/local/share/poudriere/bulk.sh
  • SCRIPTPREFIX=/usr/local/share/poudriere
  • PTNAME=default
  • SKIPSANITY=0
  • SETNAME=''
  • CLEAN=0
  • . /usr/local/share/poudriere/common.sh
  • NS=poudriere
  • sysctl -n kern.features.inet
  • sysctl -n kern.features.inet6
  • IPS=11
  • RESOLV_CONF=''
  • STATUS=0
  • test -f /usr/local/share/poudriere/../../etc/poudriere.conf
  • . /usr/local/share/poudriere/../../etc/poudriere.conf
  • ZPOOL=datapool
  • FTPHOST=ftp.ch.freebsd.org
  • FREEBSD_HOST=http://ftp.ch.freebsd.org
  • RESOLV_CONF=/etc/resolv.conf
  • BASEFS=/usr/local/poudriere
  • USE_PORTLINT=no
  • USE_TMPFS=yes
  • DISTFILES_CACHE=/usr/ports/distfiles
  • CSUP_HOST=localhost
  • CHECK_CHANGED_OPTIONS=yes
  • PKG_REPO_SIGNING_KEY=/etc/ssl/keys/pkgng.everyware.ch.key
  • CCACHE_DIR=/data/cache/ccache
  • [ -z datapool ]
  • [ -z /usr/local/poudriere ]
  • trap sig_handler SIGINT SIGTERM SIGKILL
  • trap exit_handler EXIT
  • trap siginfo_handler SIGINFO
  • zpool list datapool
  • zpool list -H -oversion datapool
  • ZVERSION=28
  • [ 28 = - ]
  • : svn.FreeBSD.org
  • : git://git.freebsd.org/freebsd-ports.git
  • : http://ftp.ch.freebsd.org
  • : /poudriere
  • : /cron
  • get_data_dir
  • local data
  • [ -n '' ]
  • awk '$1 == "data" { print $2 }'
  • head -n 1
  • zfs list -rt filesystem -H -o poudriere:type,mountpoint datapool/poudriere
  • data=/usr/local/poudriere/data
  • [ -n /usr/local/poudriere/data ]
  • echo /usr/local/poudriere/data
  • return
  • POUDRIERE_DATA=/usr/local/poudriere/data
  • : tbz
  • sysctl -n hw.ncpu
  • PARALLEL_JOBS=1
  • getopts Df:j:J:cn:p:tswz: FLAG
  • LISTPKGS=portlist-current-php53-mysql55
  • getopts Df:j:J:cn:p:tswz: FLAG
  • jail_exists 90amd64
  • [ 1 -ne 1 ]
  • zfs list -rt filesystem -H -o poudriere:type,poudriere:name datapool/poudriere
  • awk -v n=90amd64 'BEGIN { ret = 1 } $1 == "rootfs" && $2 == n { ret = 0; } END { exit ret }'
  • return 0
  • JAILNAME=90amd64
  • getopts Df:j:J:cn:p:tswz: FLAG
  • shift 4
  • [ 0 -eq 0 ]
  • [ -n portlist-current-php53-mysql55 ]
  • [ -f portlist-current-php53-mysql55 ]
  • export SKIPSANITY
  • STATUS=0
  • test -z 90amd64
  • PKGDIR=/usr/local/poudriere/data/packages/90amd64-default
  • [ 0 -eq 1 ]
  • jail_get_fs 90amd64
  • [ 1 -ne 1 ]
  • zfs list -rt filesystem -s name -H -o poudriere:type,poudriere:name,name datapool/poudriere
  • awk -v n=90amd64 '$1 == "rootfs" && $2 == n { print $3 }'
  • head -n 1
  • JAILFS=datapool/poudriere/jails/90amd64
  • jail_get_base 90amd64
  • [ 1 -ne 1 ]
  • zfs list -rt filesystem -s name -H -o poudriere:type,poudriere:name,mountpoint datapool/poudriere
  • awk -v n=90amd64 '$1 == "rootfs" && $2 == n { print $3 }'
  • head -n 1
  • JAILMNT=/usr/local/poudriere/jails/90amd64
  • export POUDRIERE_BUILD_TYPE=bulk
  • jail_start
  • [ 0 -ne 0 ]
  • zget arch
  • [ 1 -ne 1 ]
  • zfs get -H -o value poudriere:arch datapool/poudriere/jails/90amd64
  • local arch=amd64
  • local 'NEEDFS=nullfs procfs'
  • [ -z '' ]
  • [ amd64 = i386 -o amd64 = amd64 ]
  • NEEDFS='nullfs procfs linprocfs linsysfs'
  • sysctl -n compat.linux.osrelease
  • [ -n yes ]
  • NEEDFS='nullfs procfs linprocfs linsysfs tmpfs'
  • lsvfs nullfs
  • lsvfs procfs
  • lsvfs linprocfs
  • lsvfs linsysfs
  • lsvfs tmpfs
  • jail_exists 90amd64
  • [ 1 -ne 1 ]
  • zfs list -rt filesystem -H -o poudriere:type,poudriere:name datapool/poudriere
  • awk -v n=90amd64 'BEGIN { ret = 1 } $1 == "rootfs" && $2 == n { ret = 0; } END { exit ret }'
  • return 0
  • jail_runs
  • [ 0 -ne 0 ]
  • jls -qj 90amd64 name
  • return 1
  • zset status start:
  • [ 2 -ne 2 ]
  • zfs set poudriere:status=start: datapool/poudriere/jails/90amd64
  • zfs destroy -r datapool/poudriere/jails/90amd64/build
  • :
  • zfs rollback -R datapool/poudriere/jails/90amd64@clean
  • msg 'Mounting system devices for 90amd64'
  • echo '====>> Mounting system devices for 90amd64'
    ====>> Mounting system devices for 90amd64
  • do_jail_mounts 1
  • [ 1 -ne 1 ]
  • local should_mkdir=1
  • zget arch
  • [ 1 -ne 1 ]
  • zfs get -H -o value poudriere:arch datapool/poudriere/jails/90amd64
  • local arch=amd64
  • [ 1 -eq 1 ]
  • mkdir -p /usr/local/poudriere/jails/90amd64/proc
  • mount -t devfs devfs /usr/local/poudriere/jails/90amd64/dev
  • mount -t procfs proc /usr/local/poudriere/jails/90amd64/proc
  • [ -z '' ]
  • [ amd64 = i386 -o amd64 = amd64 ]
  • [ 1 -eq 1 ]
  • mkdir -p /usr/local/poudriere/jails/90amd64/compat/linux/proc
  • mkdir -p /usr/local/poudriere/jails/90amd64/compat/linux/sys
  • mount -t linprocfs linprocfs /usr/local/poudriere/jails/90amd64/compat/linux/proc
  • mount -t linsysfs linsysfs /usr/local/poudriere/jails/90amd64/compat/linux/sys
  • test -n /etc/resolv.conf
  • cp -v /etc/resolv.conf /usr/local/poudriere/jails/90amd64/etc/
    /etc/resolv.conf -> /usr/local/poudriere/jails/90amd64/etc/resolv.conf
  • msg 'Starting jail 90amd64'
  • echo '====>> Starting jail 90amd64'
    ====>> Starting jail 90amd64
  • jrun 0
  • [ 1 -ne 1 ]
  • local network=0
  • local ipargs
  • [ 0 -eq 0 ]
  • ipargs='ip4.addr=127.0.0.1 ip6.addr=::1'
  • jail -c persist name=90amd64 ip4.addr=127.0.0.1 ip6.addr=::1 path=/usr/local/poudriere/jails/90amd64 host.hostname=90amd64 allow.sysvipc allow.mount allow.socket_af allow.raw_sockets allow.chflags
  • [ 1 -eq 1 ]
  • export STATUS=1
  • prepare_jail
  • [ -z '' ]
  • export PACKAGE_BUILDING=yes
  • export FORCE_PACKAGE=yes
  • export USER=root
  • export HOME=/root
  • port_get_base default
  • [ 1 -ne 1 ]
  • awk -v n=default '$1 == "ports" && $2 == n { print $3 }'
  • zfs list -rt filesystem -H -o poudriere:type,poudriere:name,mountpoint datapool/poudriere
  • PORTSDIR=/ports
  • POUDRIERED=/usr/local/share/poudriere/../../etc/poudriere.d
  • [ -z /usr/local/poudriere/jails/90amd64 ]
  • [ -z /ports ]
  • [ -z /usr/local/poudriere/data/packages/90amd64-default ]
  • [ -n '' -a -n yes ]
  • msg 'Mounting ports filesystems for 90amd64'
  • echo '====>> Mounting ports filesystems for 90amd64'
    ====>> Mounting ports filesystems for 90amd64
  • do_portbuild_mounts 1
  • [ 1 -ne 1 ]
  • local should_mkdir=1
  • [ 1 -eq 1 ]
  • mkdir -p /ports/packages
  • mkdir -p /usr/local/poudriere/data/packages/90amd64-default/All
  • mkdir -p /ports/distfiles
  • [ -d /data/cache/ccache ]
  • mkdir -p /usr/local/poudriere/jails/90amd64/data/cache/ccache
  • msg 'Mounting ccache from /data/cache/ccache'
  • echo '====>> Mounting ccache from /data/cache/ccache'
    ====>> Mounting ccache from /data/cache/ccache
  • export CCACHE_DIR
  • export WITH_CCACHE_BUILD=yes
  • [ -d /usr/local/share/poudriere/../../etc/poudriere.d/options-90amd64 ]
  • mount -t nullfs /ports /usr/local/poudriere/jails/90amd64/usr/ports
  • mount -t nullfs /usr/local/poudriere/data/packages/90amd64-default /usr/local/poudriere/jails/90amd64/usr/ports/packages
  • [ -d /usr/ports/distfiles ]
  • mount -t nullfs /usr/ports/distfiles /usr/local/poudriere/jails/90amd64/usr/ports/distfiles
  • [ -n '' ]
  • [ -n yes ]
  • mount -t tmpfs tmpfs /usr/local/poudriere/jails/90amd64/wrkdirs
  • [ -n '' -a -d /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-options ]
  • [ -d /usr/local/share/poudriere/../../etc/poudriere.d/-options ]
  • [ -d /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-options ]
  • mount -t nullfs /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-options /usr/local/poudriere/jails/90amd64/var/db/ports
  • [ -d /data/cache/ccache ]
  • mount -t nullfs /data/cache/ccache /usr/local/poudriere/jails/90amd64/data/cache/ccache
  • [ ! -d /usr/ports/distfiles ]
  • [ -f /usr/local/share/poudriere/../../etc/poudriere.d/make.conf ]
  • cat /usr/local/share/poudriere/../../etc/poudriere.d/make.conf
  • [ -f /usr/local/share/poudriere/../../etc/poudriere.d/-make.conf ]
  • [ -f /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-make.conf ]
  • cat /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-make.conf
  • [ -f /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-default-make.conf ]
  • [ -f /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-make.conf ]
  • cat /usr/local/share/poudriere/../../etc/poudriere.d/90amd64-make.conf
  • [ -z '' ]
  • echo PACKAGE_BUILDING=yes
  • msg 'Populating LOCALBASE'
  • echo '====>> Populating LOCALBASE'
    ====>> Populating LOCALBASE
  • mkdir -p /usr/local/poudriere/jails/90amd64//usr/local
  • injail /usr/sbin/mtree -q -U -f /usr/ports/Templates/BSD.local.dist -d -e -p /usr/local
  • jexec -U root 90amd64 /usr/sbin/mtree -q -U -f /usr/ports/Templates/BSD.local.dist -d -e -p /usr/local
    mtree: /usr/ports/Templates/BSD.local.dist: No such file or directory
  • exit_handler
  • trap - EXIT SIGTERM SIGKILL
  • trap '' SIGINT
  • [ 1 -eq 1 ]
  • cleanup
  • [ -n '' ]
  • msg 'Cleaning up'
  • echo '====>> Cleaning up'
  • [ -n '' ]
  • [ -n '' ]
  • export CLEANING_UP=1
  • [ -z 90amd64 ]
  • log_stop
  • [ -n '' ]
  • [ -d /usr/local/poudriere/jails/90amd64/poudriere/var/run ]
  • zfs destroy -r datapool/poudriere/jails/90amd64/build
  • :
  • zfs destroy -r datapool/poudriere/jails/90amd64@prepkg
  • :
  • zfs destroy -r datapool/poudriere/jails/90amd64@prebuild
  • :
  • jail_stop
  • [ 0 -ne 0 ]
  • jail_runs
  • [ 0 -ne 0 ]
  • jls -qj 90amd64 name
  • return 0
  • zset status stop:
  • [ 2 -ne 2 ]
  • zfs set poudriere:status=stop: datapool/poudriere/jails/90amd64
  • jail -r 90amd64
  • [ 1 -ne 0 ]
  • jot -w %02d 1
  • jail -r 90amd64-job-01
  • :
  • msg 'Umounting file systems'
  • echo '====>> Umounting file systems'
  • mount
  • awk -v mnt=/usr/local/poudriere/jails/90amd64/ 'BEGIN{ gsub(///, "\/", mnt); } { if ($3 ~ mnt && $1 !~ //dev/md/ ) { print $3 }}'
  • sort -r
  • xargs umount -f
  • [ -n '' ]
  • zfs rollback -R datapool/poudriere/jails/90amd64@clean
  • zset status idle:
  • [ 2 -ne 2 ]
  • zfs set poudriere:status=idle: datapool/poudriere/jails/90amd64
  • export STATUS=0
  • export CLEANED_UP=1
  • [ -n ]
    +

bdrewery added on 2012-11-22 00:03:26 UTC:
This should be fixed now.

checking local package repository very time-consuming

The local packages generated by poudriere are rechecked every time I run a poudriere command (either testport or bulk).
After having successfully built some packages (here about 1300), this step takes a very long time (several minutes) to complete, before the actual poudriere command is executed.
I wonder if a complete check is really necessary, at least in case of "testport".
In this case it would IMHO be sufficient to only check the actual build dependencies of the port to be tested.

distfiles mountpoint missing while making nullfs mount, because of previous nullfs

On Mon, Sep 24, 2012 at 01:09:44PM -0400, Adam McDougall wrote:

First time user of poudriere, I was getting an error while
it tried to nullfs mount distfiles into jails. I augmented
the right part of the script with sh -x and some added ls -ld:

  • mkdir -p /poudriere/ports/default/ports/packages
  • mkdir -p /poudriere/data/packages/90amd64-default/All
  • [ -n /usr/ports/distfiles -a -d /usr/ports/distfiles ]

If missing(?) it creates it: (I didn't read that logic early in the morning)

  • mkdir -p /poudriere/jails/90amd64/usr/ports/distfiles
  • ls -ld /poudriere/jails/90amd64/usr/ports/distfiles
    drwxr-xr-x 2 root wheel 2 Sep 24 09:24 /poudriere/jails/90amd64/usr/ports/distfiles

Creation worked

  • [ -n '' -a -d '' ]
  • mount -t nullfs /poudriere/ports/default/ports /poudriere/jails/90amd64/usr/ports

Then it mounts the default ports over top of it:

  • mount -t nullfs /poudriere/data/packages/90amd64-default /poudriere/jails/90amd64/usr/ports/packages
  • ls -ld /poudriere/jails/90amd64/usr/ports/distfiles
    ls: /poudriere/jails/90amd64/usr/ports/distfiles: No such file or directory

Then the upcoming nullfs mount fails because the directory it created is covered
by the default ports which didn't have distfiles.

  • sig_handler

The logic is a bit confusing because why should it bother creating a directory
if something is going to mount over top afterwards? I worked around it by creating
a distfiles dir in the default ports. Should this step be listed on:
http://fossil.etoilebsd.net/poudriere/doc/trunk/doc/pkgng_repos.wiki

It seems like it only needs to be used as a mountpoint if I am using
DISTFILES_CACHE=/usr/ports/distfiles in the conf. Thanks.

Thanks for reporting I'll have a look soon.

Can you open an ticket on http://fossil.etoilebsd.net/poudriere ?

so that I don't forget :)

regards,
Bapt


mcdouga9 added on 2012-09-25 01:46:17 UTC:
I setup a new system with poudriere today and realized it had a newer version without the problem, sorry. On the other system where I saw the issue, I'm pretty sure it had 2.0.0. Maybe I got it from an old ports tree or the pkgbeta repo for 10amd. Closing ticket.

bulk failed

current# poudriere bulk -f ~tiger/poudriere/portlist -j 90amd64
====>> Mounting system devices for 90amd64
/etc/resolv.conf -> /usr/local/poudriere/jails/90amd64/etc/resolv.conf
====>> Starting jail 90amd64
====>> Mounting ports filesystems for 90amd64
====>> Mounting ccache from /var/cache/ccache
====>> Populating LOCALBASE
====>> Calculating ports order and dependencies
/usr/local/share/poudriere/bulk.sh: cannot create /usr/local/poudriere_data/logs/bulk/90amd64/default/index.html: No such file or directory
====>> Cleaning up
====>> Umounting file systems
current#

fix:
current# diff -Nura common.sh.orig common.sh
--- common.sh.orig 2012-09-12 09:06:29.000000000 +0300
+++ common.sh 2012-09-12 10:20:03.427118329 +0300
@@ -731,6 +731,7 @@
# Discard test stats page for now
html_path="/dev/null"
else

  •           [ -d ${logdir} ] || mkdir -p ${logdir}
            html_path="${logdir}/index.html"
    fi
    

bdrewery added on 2012-09-13 04:13:30 UTC:
Thank you, merged in.

Duplicate RESOLV_CONF directive in poudriere.conf.sample

RESOLV_CONF is mentioned twice in poudriere.conf.sample.

The following patch fixes that cosmetic problem :

8<----------------------------->8
--- poudriere.conf.sample.orig 2012-05-24 09:24:45.000000000 +0200
+++ poudriere.conf.sample 2012-05-24 09:25:22.000000000 +0200
@@ -10,10 +10,11 @@

replace CHANGE_THIS by your 2-letter country-code.

FTPHOST=ftp.CHANGE_THIS.freebsd.org

-# If you want to use a custom resolv.conf in your jails, set the path to this
-# file on the host system below. If unset, the jail's resolv.conf file will
-# not be overwritten.
-#RESOLV_CONF="/usr/local/etc/poudriere.d/resolv.conf"
+# By default the jails have no /etc/resolv.conf, you will need to set
+# REVOLV_CONF to a file on your hosts system that will be copied has
+# /etc/resolv.conf for the jail, except if you don't need it (using an http
+# proxy for example)
+RESOLV_CONF=/etc/resolv.conf

The directory where the jail will store the packages and logs

POUDRIERE_DATA=/usr/local/poudriere_data
@@ -38,12 +39,6 @@

it sticky and give a+rwx.

CRONDIR=${BASEFS}/cron

-# By default the jails have no /etc/resolv.conf, you will need to set
-# REVOLV_CONF to a file on your hosts system that will be copied has
-# /etc/resolv.conf for the jail, except if you don't need it (using an http
-# proxy for example)

-RESOLV_CONF=/etc/resolv.conf

if set the ports tree will be synced using csup instead of portsnap using the

given cvsup mirror.

CSUP_HOST=cvsup.CHANGE_THIS.freebsd.org
8<----------------------------->8

Bulk kills parent script on exit

This is pretty minor as everything does properly cleanup. The calling process just sees 'bulk' get killed, instead of bulk returning a proper exit status. It can affect scripted uses of 'bulk' that depend on the exit status though. Notice the 'Terminated:15' and it dropping back to my shell, while the cleanup continues to run in the background.

====>> Mounting system devices for 74i386
/etc/resolv.conf -> /usr/jails/poudriere/jails/74i386/etc/resolv.conf
====>> Starting jail 74i386
====>> Mounting ports filesystems for 74i386
====>> Mounting ccache from /usr/ccache
====>> Populating LOCALBASE
====>> Cleaning up old logs
====>> Calculating ports order and dependencies
====>> Sanity checking the repository
====>> Deleting stale symlinks
====>> Cleaning the build queue
====>> Starting using 1 builders
====>> [01] Starting build of sysutils/beadm
====>> [01] Finished build of sysutils/beadm: Ignored: not supported on 7.x
====>> Stopping 1 builders
====>> No package built, no need to update INDEX

  • [ -n '' ]
  • msg 'Cleaning up'
  • echo '====>> Cleaning up'
    ====>> Cleaning up
  • [ -n '' ]
  • [ -n '' ]
  • export CLEANING_UP=1
  • [ -z 74i386 ]
  • log_stop
  • [ -n '' ]
  • [ -d /usr/jails/poudriere/jails/74i386/poudriere/var/run ]
  • [ '/usr/jails/poudriere/jails/74i386/poudriere/var/run/.pid' = '/usr/jails/poudriere/jails/74i386/poudriere/var/run/.pid' ]
  • break
  • wait
  • trap '' SIGTERM
  • kill 0
  • trap sig_handler SIGTERM
    Terminated: 15
  • wait
  • zfs destroy -r zroot/poudriere/jails/74i386/build
  • [/usr/local/etc/poudriere.d] # + :
  • zfs destroy -r zroot/poudriere/jails/74i386@prepkg
  • zfs destroy -r zroot/poudriere/jails/74i386@prebuild
  • :
  • jail_stop
  • [ 0 -ne 0 ]
  • [ 0 -ne 0 ]
  • jail_runs
  • [ 0 -ne 0 ]
  • jls -qj 74i386 name
  • return 0
  • zset status stop:
  • [ 2 -ne 2 ]
  • zfs set poudriere:status=stop: zroot/poudriere/jails/74i386
  • jail -r 74i386
  • [ 1 -ne 0 ]
  • msg 'Umounting file systems'
  • echo '====>> Umounting file systems'
    ====>> Umounting file systems
  • mount
  • awk -v mnt=/usr/jails/poudriere/jails/74i386/ 'BEGIN{ gsub(///, "\/", mnt); } { if ($3 ~ mnt && $1 !~ //dev/md/ ) { print $3 }}'
  • xargs umount -f
  • sort -r
  • [ -n '' ]
  • zfs rollback -R zroot/poudriere/jails/74i386@clean
  • zset status idle:
  • [ 2 -ne 2 ]
  • zfs set poudriere:status=idle: zroot/poudriere/jails/74i386
  • export STATUS=0
  • export CLEANED_UP=1
  • msg '0 packages built, 0 failures, 1 ignored, 0 skipped'
  • echo '====>> 0 packages built, 0 failures, 1 ignored, 0 skipped'
    ====>> 0 packages built, 0 failures, 1 ignored, 0 skipped
  • [ 0 -gt 0 ]
  • [ 0 -gt 0 ]
  • [ 1 -gt 0 ]
  • msg_n 'Ignored ports: '
  • echo -n '====>> Ignored ports: '
    ====>> Ignored ports: + echo sysutils/beadm
    sysutils/beadm
  • [ 0 -gt 0 ]
  • set +e
  • exit 0
  • exit_handler
  • trap - EXIT SIGTERM SIGKILL
  • trap '' SIGINT
  • [ 0 -eq 1 ]
  • [ -n ]
    +

bdrewery added on 2012-09-10 12:46:20 UTC:
I actually run bulk from a parent script such as the one below. When 74i386 finishes, the script is killed. # ! /bin/sh

[ ${_FLOCKED:-0} -eq 0 ] && _FLOCKED=1 exec lockf -s -t0 /tmp/${0##*/}.lock $0 "$@"

/root/cvs

set -e

for jail in 74i386 82i386 90amd64 90i386; do
/usr/bin/time poudriere bulk -f /root/poudriere.list -j ${jail}
done

untitled

Hi - I am making very difficult progress with poudriere and would appreciate your help:

  1. My immediate aim is to build packages for my own hos environment (amd64).

  2. Because of (1) I would like to create the poudriere jail from already built world and not fetch, but poudriere syntax does not seem to allow this?

  3. To get around (2) I did # make installworld DESTDIR=zpool/data/poudriere and created world there. I then tried:

    poudriere jail -c -j pa64 -a amd64 -J 2 -m ftp

    This command signals a syntax error but does not say why.
    In poudriere.conf I have disabled FREEBSD_HOST (because it states that by default: ftp://ftp.freebsd.org). But if I set
    FREEBSD_HOST= ftp://ftp.freebsd.org
    this time it claims: /usr/local/share/poudriere/../../etc/poudriere.conf: ftp://ftp.freebsd.org: not found

  4. I do not understand the difference between settings in poudriere.conf for ZROOTFS and BASEFS. Are config options for ports saved in BASEFS? They could easily be saved in the build jail in my opinion. What is the reasoning that poudriere needs 2 areas?

  5. This confuses me:
    http://fossil.etoilebsd.net/poudriere/doc/trunk/doc/poudriere_in_jail.wiki
    This is obviously to run poudriere in a jail, but by following this guide I can set up a build environment on host which does not require ZFS (unless I am missing something?)

  6. Not all jails and chroots require ZFS. I would like to get a better understanding of why ZFS is a "must have" for poudriere.

Thanks and Regards.


bapt added on 2012-12-07 08:12:20 UTC:
1/ this is what poudriere is for :)

2/ poudriere jail -c ... -f yourfs with yourfs being your jail (the zfs fs containing your jail)

3/ This is perhaps a real bug that needs to be investigated, but strange anyway, I'll have a deeper look

4/ BASEFS == real point on your filesystem where the poudriere stuff will mounted into (/usr/local/poudriere by default)
ZROOTFS == path on top of the ZFS pool by default /poudriere, meaning if your pool is tank by default the ports trees FS will be created in tank/poudriere/ports/name with ZROOTFS it will be in tank/${ZROOTFS}/ports/name

5/ no zfs will still be used inside that jail

6/ zfs is mandatory because we use a lot zfs clone/rollback/snapshot/diff for which we cannot find any equivalent with UFS and a low cost in term of performance. If someone comes with a smart idea about how to make it with UFS then poudriere will no more hard depend on zfs.

poudriere ports -c error: No such file or directory

During the attempts to create the ports tree, I get the following error:
sed: /usr/local/share/poudriere/../../etc/poudriere.d/portstrees: No such file or directory

root@test91:/ # poudriere ports -c
====>> Creating default fs... done
====>> Extracting portstree "default"...
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching public key from ec2-eu-west-1.portsnap.freebsd.org... done.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Wed Dec 12 00:09:10 UTC 2012:
9b65d262bd803f780611328198a9464bb6b2eab32d5664 18% of 67 MB 724 kBps 01m18s
...
/usr/local/poudriere/ports/default/ports/x11/yelp/
/usr/local/poudriere/ports/default/ports/x11/zenity/
Building new INDEX files... done.
sed: /usr/local/share/poudriere/../../etc/poudriere.d/portstrees: No such file or directory

My poudriere.conf config file:
root@test91:/ # cat /usr/local/etc/poudriere.conf
ZPOOL=tank
FREEBSD_HOST=ftp://ftp.ru.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
POUDRIERE_DATA=/usr/local/poudriere_data
DISTFILES_CACHE=/usr/ports/distfiles
USE_PORTLINT=no

root@test91:/ # pkg info
pkg-1.0.2 New generation package manager
poudriere-devel-2.3.99.20121130 Port build and test system

root@test91:/ # uname -a
FreeBSD test91 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r244121: Tue Dec 11 22:15:03 YEKT 2012 i386

After "poudriere ports -c":
root@test91:/ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank 1,36G 123G 31K none
tank/poudriere 535M 123G 31K none
tank/poudriere/ports 535M 123G 31K none
tank/poudriere/ports/default 535M 123G 535M /usr/local/poudriere/ports/default
tank/root 854M 123G 768M /
tank/root/tmp 36K 123G 36K /tmp
tank/root/var 86,1M 123G 86,1M /var

Then I installed FreeBSD 9.0 (using mfsbsd-se-9.0-RELEASE-amd64.iso) and tested again with the same result:
fbsd90# poudriere ports -c
...
/usr/local/poudriere/ports/default/ports/x11/yeahconsole/
/usr/local/poudriere/ports/default/ports/x11/yelp/
/usr/local/poudriere/ports/default/ports/x11/zenity/
Building new INDEX files... done.
sed: /usr/local/share/poudriere/../../etc/poudriere.d/portstrees: No such file or directory

fbsd90# pkg info
pkg-1.0.3_1 New generation package manager
poudriere-devel-2.3.99.20121130 Port build and test system

fbsd# uname -a
FreeBSD fbsd 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 amd64

fbsd# zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank 1,38G 123G 31K none
tank/poudriere 535M 123G 31K none
tank/poudriere/ports 535M 123G 31K none
tank/poudriere/ports/default 535M 123G 535M /usr/local/poudriere/ports/default
tank/root 877M 123G 791M /
tank/root/tmp 35K 123G 35K /tmp
tank/root/var 85,9M 123G 85,9M /var


bdrewery added on 2012-12-20 20:55:25 UTC:
Fixed in trunk

Support mdmfs/tmpfs for LOCALBASE

This should help speedup all of the install/deinstall during big builds.


bdrewery added on 2012-05-09 01:59:11 UTC:
I've been playing with this and not seeing as much of a performance boost as I was hoping for. Going to continue messing with it.
bdrewery added on 2012-08-22 20:11:48 UTC:
Closing for now.

Build failure for cairomm-1.10.0_1 ( and others depending on cairo )

cairomm fails to find/read/use cairo.pc files (as far as I can tell ) and so does not add relevant -I flags.

Build host is running 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE amd64, zpool v28.

Hw is dell vostro 460 with 8gb ram and corei7-2600.

(Same thing happens for a few other packages as well:
graphics/gstreamer-plugins-cairo
x11-toolkits/pango
graphics/poppler-glib
graphics/py-cairo
graphics/cairomm
sysutils/fusefs-kmod
print/libspectre
graphics/opencv
x11-drivers/xf86-video-vesa
x11-drivers/xf86-input-keyboard
x11-drivers/xf86-input-mouse)

I made a tarball containing my options-dir, my make.conf the logs and the list I used, which could probably be shortened somewhat just to produce the error. The tarball is available at http://benriach.widell.net/~andrnils/poudriere-pc.tar.gz

Best regards
Andreas


andrnils added on 2012-09-18 10:41:26 UTC:
I think I spotted what might be the problem: On my desktop a config.log contains

pkg_cv_CAIROMM_CFLAGS='-I/usr/local/include/sigc++-2.0 -I/usr/local/lib/sigc++-2.0/include -I/usr/local/include/cairo -D_REENTRANT -I/usr/l
ocal/include/glib-2.0 -I/usr/local/include/pixman-1 -I/usr/local/include/freetype2 -I/usr/local/include/libdrm -I/usr/local/include/libpng1
5 -D_THREAD_SAFE -I/usr/local/include '
pkg_cv_CAIROMM_LIBS='-lsigc-2.0 -lfreetype -lpng15 -lz -lcairo -pthread -lXrender -L/usr/local/lib -lX11 '

and when build with poudriere

pkg_cv_CAIROMM_CFLAGS=''
pkg_cv_CAIROMM_LIBS='-lsigc-2.0 -lfreetype -lpng15 -lz -lcairo -pthread -lXrender -L/usr/local/lib -lX11 '

I believe this is due to Makefile.in being patched to have pkgconfigdir = $(prefix)/libdata/pkgconfig instead of pkgconfigdir = $(libdir)/pkgconfig

And then noticing that poudriere ( in a testport run ) calls configure as
$ ./configure --prefix=/tmp/cairomm-1.10.0_1 --mandir=/tmp/cairomm-1.10.0_1/man --infodir=/tmp/cairomm-1.10.0_1/info/ --build=amd64-portbld-freebsd9.1 so $(prefix)/libdata/pkgconfig will not exist.

For bulk build ./configure is called
$ ./configure --prefix=/usr/local --mandir=/usr/local/man --infodir=/usr/local/info/ --build=amd64-portbld-freebsd9.1 so why pkgconf fails to set pkg_cv_CAIROMM_CFLAGS in a bulk build I dont know.

Hope it helps.


andrnils added on 2012-09-18 21:43:28 UTC:
Well, I spent some more time on this.

There seems to be different categories:
1.) Port is broken, ie does not specify actual requirments.
2.) Port seems ok, but pkgconfig fails to do its thing. It works for libs but not cflags, so digging a bit in configure-scripts seems to indicate that an indication of "Package found" and no cflags being set is that ${PACKAGE}_CFLAGS is non-null.

Does this ring any bells?


bapt added on 2012-09-20 09:58:52 UTC:
do you have WITH_NEW_XORG activated?
andrnils added on 2012-09-21 14:52:04 UTC:
Yes, that I do.

If ones does modify the Makefile for the ports in question to specify the correct CPPFLAGS ( or often packate_CFLAGS in autotools ) builds does work.


andrnils added on 2012-09-21 19:43:38 UTC:
Update: without WITH_NEW_XORG=YES and WITH_KMS=YES in make.conf the build happily churned along past several ports that gave me troubles before. But the machine seems to have hanged ( a dedicated build machine ). I'll get back with more info when I reset it.

Main takeaway: WITH_NEW_XORG and WITH_KMS seems to break something related to pkg-config. Interesting.


andrnils added on 2012-09-21 20:01:18 UTC:
Another data-point, even though this seems not to be a poudriere bug:

I actually use xorg-dev as well, so not only WITH_NEW_XORG, but really new xorg ;)


andrnils added on 2012-09-23 21:35:29 UTC:
After doing another run without WITH_NEW_XORG that resulted in a hang ( out-of-swap ), I realized that I had enough.

I started a new build with WITH_NEW_XORG ( but no xorg-dev ports tree ) which fails on the same list of packages. This time I started looking at py-cairo. Just as ./configure was being started I did ^Z and then proceded to break into the specfic builder jail. I tried to run pkgconf --cflags cairo. Results: It wouldn't run as glproto was missing. After adding glproto (and deps ) I exited the jail and un-suspended the build. It completed!

Does WITH_NEW_XORG bring in pkgconf instead of pkgconfig? Why would ( from configure )

($PKG_CONFIG --exists --print-errors "cairo >= $cairo_required") 2>&5
ac_status=$?
$as_echo "$as_me:${as_lineno-$LINENO}: $? = $ac_status" >&5
test $ac_status = 0; }; then
pkg_cv_CAIRO_CFLAGS=$PKG_CONFIG --cflags "cairo >= $cairo_required" 2>/dev/null

suddenly stop working. When I run just pkgconf --cflags cairo I got exit status 1.

Adding USE_XORG= glproto makes at least cairomm build.


bapt added on 2012-09-24 21:30:54 UTC:
Thanks for tracking this down, it them seems to be a bug, in the cairomm port :), I'll check with x11@ people

Thanks.
Bapt


andrnils added on 2012-09-24 22:05:17 UTC:
Thanks for not just writing it off as not a poudriere bug :)

I think it goes deeper than just cairomm. It affects at least ( off the top of my head, as I accidentally removed the fixed makefile :/ ): gtk-update-icon-cache pangomm cairomm py-cairo py-gobject firefox dconf gstreamer-plugin-{cairo,pango} gtk20 gtk-engines2 graphviz opencv

I think the underlying problem is that glproto should be list in LIB_PC_DEPENDS in bsd.xorg.mk and hopefully actually get picked up as a dependency in cairo. I just started a run in such a config, I'll get back with the results :)


andrnils added on 2012-09-25 06:04:41 UTC:
So, this patch fixes the problem:

--- Mk/bsd.xorg.mk.orig 2012-05-23 10:17:49.000000000 +0200
+++ Mk/bsd.xorg.mk 2012-09-24 23:53:03.254020147 +0200
@@ -166,7 +166,7 @@
fontenc_LIB_PC_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/fontenc.pc:${PORTSDIR}/x11-fonts/libfontenc
fontsproto_BUILD_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/fontsproto.pc:${PORTSDIR}/x11-fonts/fontsproto
fontutil_LIB_PC_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/fontutil.pc:${PORTSDIR}/x11-fonts/font-util
-glproto_BUILD_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/glproto.pc:${PORTSDIR}/x11/glproto
+glproto_LIB_PC_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/glproto.pc:${PORTSDIR}/x11/glproto
ice_LIB_PC_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/ice.pc:${PORTSDIR}/x11/libICE
inputproto_BUILD_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/inputproto.pc:${PORTSDIR}/x11/inputproto
kbproto_BUILD_DEPENDS= ${LOCALBASE}/libdata/pkgconfig/kbproto.pc:${PORTSDIR}/x11/kbproto

The build finished correctly :)


andrnils added on 2012-09-25 06:40:00 UTC:
I filed a pr for this :)
bapt added on 2012-09-25 17:22:25 UTC:
Nope :), the right fix is to use:

USE_XORG= glproto:both

In the cairo ports in case the GL option is activated


andrnils added on 2012-09-25 20:11:26 UTC:
As side note: I ran a new build using xorg-dev tree as well. For the patch with glproto is not enough there, I think same modification is needed for dri2proto.

Build running.


bapt added on 2012-09-30 20:24:08 UTC:
Fixed in the ports tree

checking local package repository very time-consuming

The local packages generated by poudriere are rechecked every time I run a poudriere command (either testport or bulk).
After having successfully built some packages (here about 1300), this step takes a very long time (several minutes) to complete, before the actual poudriere command is executed.
I wonder if a complete check is really necessary, at least in case of "testport".
In this case it would IMHO be sufficient to check the actual build dependencies of the port to be tested.


bapt added on 2012-06-19 06:58:53 UTC:
We can't just drop that feature, but we can certainly improve its speed. I'll try to look at it.
bapt added on 2012-06-19 11:58:00 UTC:
[851c0b25ba] and [2745af1739] should have improved the situation, I keep this report open, for further improvements.
bapt added on 2012-07-28 14:17:06 UTC:
using -s with bulk allow you to skip that part.

Create log files as <jail-name>/<port>

Currently poudriere creates log files in the form of ${POUDRIERE_DATA}/logs/- using only a dash between the and

Would be more convenient if log files were created similar to the packages in the form of ${POUDRIERE_DATA}/logs//

Thanks, bapt :)


bdrewery added on 2012-08-16 00:21:58 UTC:
This is now implemented.

add SETS concept to poudriere

I wish to suggest some enhancements to poudriere: a concept of SETS, SET
is a make.conf + ports option combination for some list of software. By
now its possible to have per-jail make.conf+options and I suggest to add
possibility to have as much make.conf+options combinations as needed.
For example I use 90amd64 jail to build software for my server and wish
to use the same 90amd64 jail for building desktop software with very
different options and make.conf parameters of server version. So here is
my dirty but working for me patches to implement it. Do please consider
it, fix it and may be even include it in poudriere.

--- bulk.sh.orig 2012-07-05 16:07:35.769269169 +0300
+++ bulk.sh 2012-07-05 17:15:35.336505923 +0300
@@ -11,7 +11,8 @@
-k -- Keep the previous built binary packages
-t -- Add some testings to package building
-j name -- Run only on the given jail

  • -p tree -- Specify on which ports tree the bulk will be done"
  • -p tree -- Specify on which ports tree the bulk will be done
  • -z name -- Specify which SET name to use"
exit 1

}
@@ -21,9 +22,10 @@
PTNAME="default"
SKIPSANITY=0
KEEP=0
+SETNAME=""
. ${SCRIPTPREFIX}/common.sh

-while getopts "f:j:kp:ts" FLAG; do
+while getopts "f:j:kp:tsz:" FLAG; do
case "${FLAG}" in
t)
export PORTTESTING=1
@@ -44,6 +46,11 @@
s)
SKIPSANITY=1
;;

  •           z)
    
  •                   [ ! -z ${OPTARG} ] && SETNAME="-"${OPTARG} || err 1 "Empty set name!"
    
  •                   ;;
    
    •       *)
                    usage
                    ;;
      

    @@ -59,7 +66,7 @@
    test -z "${JAILNAMES}" && JAILNAMES=jail_ls

    for JAILNAME in ${JAILNAMES}; do

  •   PKGDIR=${POUDRIERE_DATA}/packages/${JAILNAME}-${PTNAME}
    
  •   PKGDIR=${POUDRIERE_DATA}/packages/${JAILNAME}${SETNAME}-${PTNAME}
    jail_start ${JAILNAME}
    
    if [ ${KEEP} -ne 1 ]; then
    

    --- common.sh.orig 2012-07-05 16:04:04.858537137 +0300
    +++ common.sh 2012-07-05 17:49:02.468130306 +0300
    @@ -455,7 +455,7 @@

    msg "Building ${port}"
    PKGNAME=$(injail make -C ${portdir} -VPKGNAME)
    
  •   log_start ${LOGS}/${JAILNAME}-${PTNAME}-${PKGNAME}.log
    
  •   log_start ${LOGS}/${JAILNAME}${SETNAME}-${PTNAME}-${PKGNAME}.log
    
    zfs_set "poudriere:status" "depends:${port}"
    if ! injail make -C ${portdir} pkg-depends fetch-depends extract-depends \
    

    @@ -484,7 +484,7 @@
    export failed="${failed} ${state}"
    fi
    jail_status "idle:"

  •   log_stop ${LOGS}/${JAILNAME}-${PTNAME}-${PKGNAME}.log
    
  •   log_stop ${LOGS}/${JAILNAME}${SETNAME}-${PTNAME}-${PKGNAME}.log
    

    }

    list_deps() {
    @@ -683,10 +683,10 @@
    [ -n "${USE_TMPFS}" ] && mount -t tmpfs tmpfs ${JAILBASE}/wrkdirs

    [ -f ${POUDRIERED}/make.conf ] && cat ${POUDRIERED}/make.conf >> ${JAILBASE}/etc/make.conf
    
  •   [ -f ${POUDRIERED}/${JAILNAME}-make.conf ] && cat ${POUDRIERED}/${JAILNAME}-make.conf >> ${JAILBASE}/etc/make.conf
    
  •   [ -f ${POUDRIERED}/${JAILNAME}${SETNAME}-make.conf ] && cat ${POUDRIERED}/${JAILNAME}${SETNAME}-make.conf >> ${JAILBASE}/etc/make.conf
    
  •   if [ -d ${POUDRIERED}/${JAILNAME}-options ]; then
    
  •           mount -t nullfs ${POUDRIERED}/${JAILNAME}-options ${JAILBASE}/var/db/ports || err 1 "Failed to mount OPTIONS directory"
    
  •   if [ -d ${POUDRIERED}/${JAILNAME}${SETNAME}-options ]; then
    
  •           mount -t nullfs ${POUDRIERED}/${JAILNAME}${SETNAME}-options ${JAILBASE}/var/db/ports || err 1 "Failed to mount OPTIONS directory"
    elif [ -d ${POUDRIERED}/options ]; then
            mount -t nullfs ${POUDRIERED}/options ${JAILBASE}/var/db/ports || err 1 "Failed to mount OPTIONS directory"
    fi
    

    --- jail.sh.orig 2012-07-06 14:15:22.238129967 +0300
    +++ jail.sh 2012-07-06 14:23:54.779128681 +0300
    @@ -25,7 +25,9 @@
    that with svn and csup the world will be built. note
    that building from sources can use src.conf and
    jail-src.conf from localbase/etc/poudriere.d

  • -t version -- version to upgrade to"

  • -t version -- version to upgrade to

  • -p tree -- Specify on which ports tree the bulk will be done

  • -z name -- Specify which SET name to use"
    exit 1
    }

@@ -320,6 +322,8 @@
QUIET=0
INFO=0
UPDATE=0
+SETNAME=""
+PTNAME="default"

SCRIPTPATH=realpath $0
SCRIPTPREFIX=dirname ${SCRIPTPATH}
@@ -375,6 +379,12 @@
t)
TORELEASE=${OPTARG}
;;

  •           p)
    
  •                   PTNAME=${OPTARG}
    
  •                   ;;
    
  •           z)
    
  •                   [ ! -z ${OPTARG} ] && SETNAME="-"${OPTARG} || err 1 "Empty set name!"
    
  •                   ;;
            *)
                    usage
                    ;;
    
    @@ -397,7 +407,11 @@
    ;;
    0001000)
    export SET_STATUS_ON_START=0
  •           JAILBASE=`jail_get_base ${JAILNAME}`
    
  •           JAILFS=`jail_get_fs ${JAILNAME}`
    
  •           PKGDIR=${POUDRIERE_DATA}/packages/${JAILNAME}${SETNAME}-${PTNAME}
            jail_start ${JAILNAME}
    
  •           prepare_jail
            ;;
    0000100)
            delete_jail
    

bdrewery added on 2012-09-20 04:28:32 UTC:
Committed, thanks!

firefox-15.0.1 built with poudriere fails with XError: (X_ShmAttach|X_ShmPutImage): BadAccess

When building firefox with options

This file is auto-generated by 'make config'.

Options for firefox-15.0.1,1

_OPTIONS_READ=firefox-15.0.1,1
_FILE_COMPLETE_OPTIONS_LIST=DBUS DEBUG GNOME GSTREAMER LIBPROXY LOGGING OPTIMIZED_CFLAGS PGO ALSA OSS PULSEAUDIO GTK2 QT4 GCONF GIO GNOMEUI GNOMEVFS2
OPTIONS_FILE_SET+=DBUS
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_UNSET+=GNOME
OPTIONS_FILE_UNSET+=GSTREAMER
OPTIONS_FILE_UNSET+=LIBPROXY
OPTIONS_FILE_SET+=LOGGING
OPTIONS_FILE_SET+=OPTIMIZED_CFLAGS
OPTIONS_FILE_UNSET+=PGO
OPTIONS_FILE_UNSET+=ALSA
OPTIONS_FILE_SET+=OSS
OPTIONS_FILE_UNSET+=PULSEAUDIO
OPTIONS_FILE_UNSET+=GTK2
OPTIONS_FILE_SET+=QT4
OPTIONS_FILE_UNSET+=GCONF
OPTIONS_FILE_UNSET+=GIO
OPTIONS_FILE_UNSET+=GNOMEUI
OPTIONS_FILE_UNSET+=GNOMEVFS2

The build completes, but the resulting binary wont work. I get a fully transparent window, and error-messages like

[andrnils@caolila ~]$ firefox
XError: X_ShmAttach: BadAccess (attempt to access private resource denied); 8 requests ago
XError: X_ShmPutImage: BadShmSeg (invalid shared segment parameter); 6 requests ago
XError: X_ShmAttach: BadAccess (attempt to access private resource denied); 2 requests ago
XError: X_ShmDetach: BadShmSeg (invalid shared segment parameter); 8 requests ago
XError: X_ShmPutImage: BadShmSeg (invalid shared segment parameter); 6 requests ago
XError: X_ShmPutImage: BadShmSeg (invalid shared segment parameter); 5 requests ago
XError: X_ShmPutImage: BadShmSeg (invalid shared segment parameter); 23 requests ago
XError: X_ShmPutImage: BadShmSeg (invalid shared segment parameter); 7 requests ago

in terminal. If I run with MOZ_X_SYNC=1 I get a coredump:

[andrnils@caolila ~]$ MOZ_X_SYNC=1 firefox
QCoreApplication::postEvent: Unexpected null receiver

!!! ABORT: X_ShmAttach: BadAccess (attempt to access private resource denied); sync: file /wrkdirs/usr/ports/www/firefox/work/mozilla-release/toolkit/xre/nsX11ErrorHandler.cpp, line 157

!!! ABORT: X_ShmAttach: BadAccess (attempt to access private resource denied); sync: file /wrkdirs/usr/ports/www/firefox/work/mozilla-release/toolkit/xre/nsX11ErrorHandler.cpp, line 157

Segmentation fault: 11 (core dumped)


bapt added on 2012-09-30 20:22:52 UTC:
Looks like a port bug more than a poudriere bug imho.
andrnils added on 2012-10-01 13:30:07 UTC:
Yes, perhaps.

I'll take it to gecko@ then.

assertion fail on pkg create

if the manifest is malformed and has :

name: foo
version:
origin: bar/foo

then pkg create fails quite badly :

[vty-admin1:~/pkgng/collectd-plugins] pkg create -r ./fakeroot -m . collectd-plugins
Assertion failed: (buf != NULL), function sbuf_get, file utils.c, line 70.
[1] 73238 abort pkg create -r ./fakeroot -m . collectd-plugins


bapt added on 2012-09-25 17:20:06 UTC:
Not a poudriere bug :)

Reduce disk-io

I would like to propose a change in building strategy for poudriere, which would considerably lessen the disk-io burden ( which with 8 builders gets pretty severe ).

Let there be some packages a b c d e f g, where e depends on d and d on a, and g depends on all.
Now:
B : 1 2 3 4
01: a d e g
02: b
03: c
04: f

( One possibility, I guess )
Where for every builder and every pkg there is install all deps from scratch, build, rollback. But in the above example, why not just take a snapshot in-between the builds in builder 01 and do the diff on the snapshots?

Early on current solution is good, but when things like DEs gets build, every subpackage installs xorg+toolkits(gtk/qt)+perl and so on. When there is so clear chains, a complete reset is very costly IMHO.

I would also suggest that as the build progresses and deps becomes more pronounced and packages to build larger, BUILD_JOBS_DISABLE should not be used.

The build machine I use now ( core-i7 2600 ) has an average load like 2 for a whole build including libreoffice, chromium, firefox and kde ( and thats with BUILD_JOBS_DISABLE unset ), but the machine is almost non-responsive due to disk-io.


andrnils added on 2012-09-25 07:30:30 UTC:
Further data-points: I notice on average between 200 and 300 tps and ~10mb/s disk-io in systat :iostat

Doing diffs on snapshots in a chain build would also allow you do the package-building in the background, I think.


bdrewery added on 2012-09-25 16:12:16 UTC:
What is BUILD_JOBS_DISABLE ?
andrnils added on 2012-09-25 18:54:00 UTC:
It is my bad memory ;) DISABLE_MAKE_JOBS was the one. Sorry
bdrewery added on 2013-03-11 12:00:35 UTC:
Related to [008a6a6eb2]

Incorrect ports tree detection

If you have portstree which name is substring of other portstree, build is failing, since can't be found portstree:

root@:/root # zfs list -t filesystem -Hd1 -o poudriere:type,poudriere:name,mountpoint zpl/poudriere | awk '/^ports[[:space:]]'new-xorg'/ { print $3 }'
/zpl/pdr/ports/new-xorg
/zpl/pdr/ports/new-xorg-kms

Fixed function (common.sh:128):

port_get_base() {
[ $# -ne 1 ] && err 1 "Fail: wrong number of arguments"
zfs list -t filesystem -Hd1 -o poudriere:type,poudriere:name,mountpoint ${ZPOOL}/poudriere | \
awk '/^ports[[:space:]]'$1'[[:space:]]/ { print $3 }'
}

Cannot update portstree from cron

Launching "poudriere ports -u" from cron fails with the following error :

====>> Updating portstree "default"
portsnap fetch should not be run non-interactively.
Run portsnap cron instead.

The following patch can be applied to allow poudriere to update its porstree from cron :

8<------------------------>8
--- ports.sh.orig 2012-05-24 09:17:55.000000000 +0200
+++ ports.sh 2012-05-24 09:20:25.000000000 +0200
@@ -132,6 +132,8 @@
ports-all" > ${PTBASE}/csup
csup -z -h ${CSUP_HOST} ${PTBASE}/csup
else

  •           /usr/sbin/portsnap -d ${PTBASE}/snap -p ${PTBASE}/ports fetch update
    
  •           PSCOMMAND=fetch
    
  •           [ ! -t 0 ] && PSCOMMAND=cron
    
  •           /usr/sbin/portsnap -d ${PTBASE}/snap -p ${PTBASE}/ports ${PSCOMMAND} update
    fi
    
    fi
    8<------------------------>8

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.