osinside/kiwi-boxed-plugin

KIWI plugin to provide self contained build support

kiwi-boxed-plugin's Issues

Nested VM build may crash using smp default CPU count

When using nested VM CPU is limited. Adding a smp option could avoid build crash. Just like adding a parameters like memory setting.

Boxbuild stuck at `zypper` & runs into an interactive shell within VM

I am trying to build an appliance, using the boxbuild feature. The box has been pulled from the internet and everything seems supercool! :) 👍 Sadly, i am not able to build an image:

• when building as root, the box is actually booting and starting the build process. But it gets stuck on installing packages with zypper. Something weird i experienced on every build: I actually get a command-prompt in the VM.
• as an unprivileged user. It seems it has some issues with tar. This happened, because i first ran the boxbuild as root and the boxes belonged to root in /var/tmp/kiwi. After deleting /var/lib/kiwi, i could run it with low privileges. The images were downloaded again. But i ran into the same issue as root.

Problem description

I am using the following command to trigger the build. Note: Instead of using --type i use --profile, changing it made no difference for me.

kiwi-ng --profile Live system boxbuild --box leap -- \
    --description /home/${USER}/kiwi-descriptions/suse/x86_64/suse-leap-15.2 \
    --target-dir /tmp/myimage

In the same CLI i triggerd the build, i receive the command prompt on the VM and the process is stuck in Processing:.

[ INFO    ]: 15:02:24 | --> package: glibc-locale
[ INFO    ]: 15:02:24 | --> package: openSUSE-release
[ INFO    ]: 15:02:24 | --> package: udev
[ INFO    ]: 15:02:24 | --> package: zypper
[ INFO    ]: Processing: [                                        ] 0%
localhost:~ # 

End of the result.log:

INFO: 14:02:52 | --> package: zypper
DEBUG: 14:02:52 | EXEC: [zypper --non-interactive --pkg-cache-dir /result/build/image-root/var/cache/kiwi/packages --reposd-dir /result/build/image-root/var/cache/kiwi/zypper/repos --solv-cache-dir /result/build/image-root/var/cache/kiwi/zypper/solv --cache-dir /result/build/image-root/var/cache/kiwi/zypper --raw-cache-dir /result/build/image-root/var/cache/kiwi/zypper/raw --config /result/build/image-root/tmplzno8jm3 --root /result/build/image-root install --auto-agree-with-licenses --no-recommends -- ca-certificates ca-certificates-mozilla cracklib-dict-full filesystem glibc-locale openSUSE-release udev zypper]

Expected behaviour

• The build runs successfully.
  • It runs zypper successfully.
• The boxbuild documentation some insights to the local boxes:
  • E.g. where they are located /var/tmp/kiwi/boxes

Steps to reproduce

1. Run an Opensuse 15.2 as a host.
2. Add the appliance-builder repostiory (https://osinside.github.io/kiwi/installation.html#installation-from-obs)
3. Install kiwi-ng from the vendor repo. (https://osinside.github.io/kiwi/installation.html#installation-from-obs)
4. Install python3-kiwi_boxed_plugin from the vendor repo: (https://osinside.github.io/kiwi/self_contained.html)
5. Fetch the example kiwi-descriptions: git clone https://github.com/OSInside/kiwi-descriptions
6. Trigger the build:

   kiwi-ng --profile Live system boxbuild --box leap -- \
   --description /home/${USER}/kiwi-descriptions/suse/x86_64/suse-leap-15.2 \
   --target-dir /tmp/myimage

OS and Software information

• KIWI version:

  Name           : python3-kiwi
  Version        : 9.23.25-lp152.1.1
  Arch           : x86_64
  Vendor         : obs://build.opensuse.org/Virtualization:Appliances
  

• KIWI Boxed plugin version:

  Name           : python3-kiwi_boxed_plugin
  Version        : 0.2.8-lp152.1.1
  Arch           : noarch
  Vendor         : obs://build.opensuse.org/Virtualization:Appliances
  

• Operating system: openSUSE Leap 15.2
• OBS version: ?

Returncode is always `0`

I got two pennies and can build faster now. But i ran into another issue. Our scripts depend on kiwi-ngs return code. But it always returns 0:

Steps to reproduce

1. Setup as described in the documentation.
2. Clone kiwi-descriptions.
3. Create an error, two options exist:
   3a. echo "exit 42" >> kiwi-descriptions/suse/x86_64/suse-leap-15.2/config.sh
   3b. Syntax Error in config.xml: echo "<<<<<" >> kiwi-descriptions/suse/x86_64/suse-leap-15.2/config.xml
4. Trigger build
5. echo $?

Expected behavior

Step 5 must not print 0. Build has been triggered with:

kiwi-ng --type iso system boxbuild --box leap -- --description ~/kiwi-descriptions/suse/x86_64/suse-leap-15.2 --target-dir /tmp/myimage

It should work the same way as the classical build, which does not print 0.

Actual behavior

Step 5 prints 0. Which contradictively indicates that the build was successful.

Error early if the box VM would overcommit the host memory

I was testing how boxed works on low memory systems, and it will readily overcommit memory and get quickly reaped by the OoM killer.

It would be useful to add a quick check that the requested box memory is larger than the free/available memory instead of just crashing a host.

Came here from rockstor/rockstor-installer#73 btw

Allow boxroot for overlay files into the box

Like with the root/.. overlay files for the actual image build it would be handy to allow a boxroot/... for data that should be copied into the box environment prior building. Use cases:

• Provide custom kiwi.yml
• Provide security keys/credentials temporarily to the box
• etc...

Add support for HTTP PROXY

If dumped in the debug shell,  HTTP_PROXY=, is not set.

As mentioned run_kiwi fails.

If  HTTP_PROXY= is set manually run_kiwi works.

We should find a way to import proxy settings properly

argument parsing fails when using `--foo=bar` instead of `--foo bar`

when using = in arguments the argument parsing breaks:

does not work:
kiwi --profile=vmware system boxbuild --box leap -- --description=descriptions/sles-15sp2-amd64-vmware --target-dir=out/sles-15sp2-amd64-vmware

does work:
kiwi --profile=vmware system boxbuild --box leap -- --description descriptions/sles-15sp2-amd64-vmware --target-dir out/sles-15sp2-amd64-vmware

error message:

[ INFO    ]: 13:49:38 | Building with:
    ['--type', 'oem', '--profile', 'vmware', 'system', 'build', '--description=descriptions/sles-15sp2-amd64-vmware', '--target-dir=out/sles-15sp2-amd64-vmware']
[ ERROR   ]: 13:49:38 | Unexpected error:
Traceback (most recent call last):
  File "/usr/local/bin/kiwi", line 8, in <module>
    sys.exit(main())
  File "/opt/venv/lib/python3.9/site-packages/kiwi/kiwi.py", line 68, in main
    App()
  File "/opt/venv/lib/python3.9/site-packages/kiwi/app.py", line 35, in __init__
    task_class().process()
  File "/opt/venv/lib/python3.9/site-packages/kiwi_boxed_plugin/tasks/system_boxbuild.py", line 160, in process
    box_build.run(
  File "/opt/venv/lib/python3.9/site-packages/kiwi_boxed_plugin/box_build.py", line 76, in run
    desc = self._pop_arg_param(
  File "/opt/venv/lib/python3.9/site-packages/kiwi_boxed_plugin/box_build.py", line 155, in _pop_arg_param
    arg_index = self.kiwi_build_command.index(arg)
ValueError: '--description' is not in list

Some boxes are no longer available

It looks like the leap and universal boxes got removed by automated cleanup of repositories in OBS.

https://build.opensuse.org/request/show/1050474
https://build.opensuse.org/request/show/1053653

So now the boxbuild command gives error
KiwiUriOpenError: http://download.opensuse.org/repositories/Virtualization:/Appliances:/SelfContained:/leap/images: 404 Client Error: Not Found

Enable multiple cpus on the build box

There are several procedures that are costly in terms of CPU that are a parallelized by default if the host includes multiple CPUs. This is the case for squashfs creation and other compression procedures.

I would be nice if the kiwi-boxed-plugin could expose more than one cpu to the build box.

Permissions issues with the boxed-VMs storage

The storage location of the boxed-VMs is currrently /var/tmp/kiwi/boxes. All users on a system share these boxes. This leads to conflicting permissions between users, when a box-VM is outdated and another user tries to update it.

First noted in #26.

Expected behaviour

Either...

• Tell the user to delete the boxes manually rm -rf /var/tmp/kiwi/boxes
• Use individual storage paths for every user.

Steps to reproduce

1. Build a box as root.
2. Wait sometime for a box update. Inspect the date for e.g. leap:
   https://build.opensuse.org/project/show/Virtualization:Appliances:SelfContained:leap
3. Try a new build as another user, with lower privileges.

CentOS 8 qemu-system-x86_64 replace as qmeu-kvm

qemu-system-x86_64 doesn't exist in CentOS 8 anymore.
It is replaced by /usr/libexec/qemu-kvm in qemu-kvm-core.rpm.
We either need to adopt that and do qemu-system-x86_64 runtime detection script or testing to see if new qemu-kvm existed.

Allow specific kiwi version to be used in the box

User Story:

I just talked to my coworker about version pinning of the boxed vm. Would it be possible to pass a
version (9.19.7) to the plugin and the vm then runs pip install kiwi==9.19.7 before it executes kiwi?
And if no version is passed to the plugin, it uses the included kiwi version.

overlay tree as directory tree has issues with 9p shared folders

When using the box plugin there was a report on the kiwi mailing list that the rsync operation which transfers the overlay root tree (root/) from the image description into the image fails with an error message like this

cannot read symbolic link 'test99': Too many levels of symbolic links

The overlay tree just contains a symbolic link setup like this

acme:spec/root/home # ls -l
   total 4
   drwxr-xr-x 1 roger users 30 Oct 29 09:37 rst
   lrwxrwxrwx 1 roger users  4 Oct 30 11:39 test99 -> rst/

It is not expected that this causes any error as there are no circular references.

The issue can be solved if the root overlay is packed into a tarball root.tar.gz and used in that way. kiwi untars the contents into the image root tree instead of rsync a directory.

The reason for the issue must be related to the 9p shared folder characteristics of the image description directory. When using the box plugin the directory which contains the image description is shared with the virtual machine using the 9p filesystem. The rsync operation which transfers the root/ overlay directory into the image description is therefore an operation that runs through 9p and it seems depending on the structure of the tree this could cause problems.

I currently have not other solution than tarring up the root/ overlay. For the future we want to replace 9p with virtiofs but this has not yet landed on all distributions

`--config` option is not passed to the Kiwi instance inside the box

I'm trying to make use of the boxed plugin to build Ubuntu images, but I'm running into a problem with trying to pass through the Kiwi configuration file. Building Ubuntu images requires the following configuration passed to Kiwi using --config:

runtime_checks:
  - disable:
      - "check_dracut_module_for_disk_overlay_in_package_list"

However, this configuration does not seem to be passed to the Kiwi instance running inside of the box (note that I added the --config flag before system boxbuild, but it is not a part of the argument list shown by "Building with"):

$ kiwi  --debug --profile development --config nemos-images-reference-lunar/kiwi.yaml  system boxbuild --box ubuntu  --box-memory=$(( 6 * 1024))  --box-smp-cpus=2 -- --description nemos-images-reference-lunar/qemu-amd64
/ --target-dir ~/tmp/kiwi-amd64
[ INFO    ]: 14:01:02 | Reading runtime config file: 'nemos-images-reference-lunar/kiwi.yaml'
[ INFO    ]: 14:01:02 | Reading box plugin config file: /usr/lib/python3.11/dist-packages/kiwi_boxed_plugin/config/kiwi_boxed_plugin.yml
[ INFO    ]: 14:01:02 | Validating kiwi_build_command_args:
    ['system', 'build', '--description', 'nemos-images-reference-lunar/qemu-amd64/', '--target-dir', '/home/itrue/tmp/kiwi-amd64']
[ INFO    ]: 14:01:02 | Building with:
    ['--debug', '--profile', 'development', 'system', 'build', '--description', 'nemos-images-reference-lunar/qemu-amd64/', '--target-dir', '/home/itrue/tmp/kiwi-amd64']

This results in the following build error:

[ DEBUG   ]: 11:58:18 | EXEC: [mkdir -p /result/build]
[ INFO    ]: 11:58:18 | Loading XML description
[ INFO    ]: 11:58:18 | Support for XML markup available
[ INFO    ]: 11:58:32 | --> loaded /description/appliance.kiwi
[ INFO    ]: 11:58:32 | --> Selected build type: oem
[ INFO    ]: 11:58:32 | --> Selected profiles: development
[ ERROR   ]: 11:58:32 | KiwiRuntimeError:

Required dracut module package missing in package list

The package 'dracut-kiwi-overlay' is required for the selected
overlayroot activated image type. Please add the
following in your <packages type="image"> section to
your system XML description:

<package name="dracut-kiwi-overlay"/>

Add support for sshfs

In https://github.com/davidcassany/kiwi-boxed-plugin/tree/attempt_macos_support I tried to run the boxed pluing in macOS. However I realized that 9p or virtiofs support in QEMU for macOS is simply missing. The alternative is using sshfs under macOS.

sshfs seams to be available on most distros. My only doubt is how hard is to configure a macOS host to make use of it, it is unclear to me which ssh requirements encompasses for the host (ideally only an ssh client).

I believe this is the only missing feature to run boxed kiwi under macOS.

Allow for box repo signature verification

At the moment the plugin provides a pre configured box config via kiwi_boxed_plugin/config/kiwi_boxed_plugin.yml.
The used download source points to Virtualization:Appliances:SelfContained and is considered a trusted source.

However it would be more flexible and secure if the config file would be a system wide setup e.g

/etc/kiwi_boxed_plugin.yml

which allows for an additional setting like:

box:
  -
    name: ...
    verify: signature_key

The provided signature_key file should be used to verify the signature of the box source

Not able to download leap box image

This worked 2 days ago, but not working today and I have had no changes to my config. I can only assume something changed on the openSUSE downloads side.

When I try to build an iso based on openSUSE leap 15.4, I am now getting a 403 from the download server when trying to download the self contained image. I get the same 403 when I try to directly browse to that URL from my browser.

Is there something I can change in my config files or is this something I cannot resolve on my end:

user@host:~$ pip show kiwi
Name: kiwi
Version: 9.25.16
Summary: KIWI - Appliance Builder (next generation)
Home-page: https://osinside.github.io/kiwi
Author: Marcus Schaefer
Author-email: [email protected]
License: GPLv3+
Location: /usr/local/lib/python3.10/dist-packages
Requires: docopt, lxml, PyYAML, requests, simplejson
Required-by: kiwi-boxed-plugin

user@host:~$ pip show kiwi_boxed_plugin
Name: kiwi-boxed-plugin
Version: 0.2.30
Summary: KIWI - Boxed Build Plugin
Home-page: https://github.com/OSInside/kiwi-boxed-plugin
Author: Marcus Schaefer
Author-email: [email protected]
License: GPLv3+
Location: /usr/local/lib/python3.10/dist-packages
Requires: cerberus, docopt, kiwi, progressbar2, PyYAML, requests
Required-by:

user@host:~$ kiwi-ng --type iso system boxbuild --box leap -- --description kiwi --target-dir output
[ INFO    ]: 13:14:23 | Reading box plugin config file: /usr/local/lib/python3.10/dist-packages/kiwi_boxed_plugin/config/kiwi_boxed_plugin.yml
[ INFO    ]: 13:14:23 | Validating kiwi_build_command_args:
    ['system', 'build', '--description', 'kiwi', '--target-dir', 'output']
[ INFO    ]: 13:14:23 | Building with:
    ['--type', 'iso', 'system', 'build', '--description', 'kiwi', '--target-dir', 'output']
[ ERROR   ]: 13:14:35 | KiwiUriOpenError: KiwiUriOpenError: http://download.opensuse.org/repositories/Virtualization:/Appliances:/SelfContained:/leap/images: 403 Client Error: Forbidden for url: http://downloadcontentcdn.opensuse.org/repositories/Virtualization:/Appliances:/SelfContained:/leap/images/ None

Allow to specify a custom directory to be shared with the box

For accessing local content e.g a local repo it would be useful to allow an option to share a given host directory with the box such that a reference to it from inside the box becomes possible

Man pages are missing from PyPI source tarball

Trying to build the latest release in Fedora leads to this error:

+ rm -rfv /builddir/build/BUILDROOT/kiwi-boxed-plugin-0.2.12-1.fc35.noarch/usr/bin/__pycache__
+ make buildroot=/builddir/build/BUILDROOT/kiwi-boxed-plugin-0.2.12-1.fc35.noarch/ docdir=/usr/share/doc/ install
# install plugin manual page and license/readme
# NOTE: this file is not handled through pip because on system level
install -d -m 755 /builddir/build/BUILDROOT/kiwi-boxed-plugin-0.2.12-1.fc35.noarch/usr/share/man/man8
gzip -f doc/build/man/kiwi::system::boxbuild.8
gzip: doc/build/man/kiwi::system::boxbuild.8: No such file or directory
make: *** [Makefile:19: install] Error 1

The tarball doesn't include the files, so it looks like we're not generating them before making the sdist tarball to upload.