devcontainers-contrib / nanolayer Goto Github PK

View Code? Open in Web Editor NEW

37.0 1.0 5.0 83.51 MB

Shell 0.21% Python 88.18% Starlark 11.61%

nanolayer's Introduction

Nanolayer CLI

nanolayer helps keep container layers as small as possible.

It does so by automatically deleting any installation leftovers (such as apt-get update lists, ppas, etc)

Installation

pipx install nanolayer

GH Release installation:

Usage:

nanolayer install gh-release cli/cli gh

Example

FROM python:3.10

RUN apt-get -y update && apt-get install -y htop

layer size: 22MB

FROM python:3.10

RUN curl -sfL https://github.com/devcontainers-contrib/nanolayer/releases/download/v0.4.0/nanolayer-x86_64-unknown-linux-gnu.tgz | tar fxvz - -C / && \
    chmod 755 /tmp/nanolayer && ls /tmp && \
    /nanolayer install apt-get htop && \
    rm /nanolayer

Layer size: 1.6MB

nanolayer's People

Contributors

Stargazers

Watchers

Forkers

ndbrew devcontainers-contrib chriscunningham-cts fightingsleep sevetseh28

nanolayer's Issues

dcontainer is compiled to linux only binary

Erlang feature works on only some base images, fails on others

ok images:

mcr.microsoft.com/devcontainers/base:bullseye

bad images:

debian:bullseye (unknown reason)
debian:bookworm (different apt-get packages required)

devcontainer-contrib cli python dependencies reduce portability

Currently devcontainer-contrib cli is a python package which depends on python tooling in order to get installed. This prevents it from being used to install features as require an external bash script for that ("featmake")

ppas flag fail installation on debian

currently adding --ppas flag to nanolayer install apt-get failing on debian based (without the force ppas on debian flag)
required behavior:
ppas will be ignored in debian, and a warning will be printed

Allow installing a package from a github release

I'd like to install a github release that only has platform packages (eg, deb,rpm), and no binaries/zipped binaries.
(The particular repository is emqx/MQTTX, but there are possibly more. The repo does have cli-only binaries, but I want the GUI package)

Obviously there are other ways to get the package installed, but having the version resolving feature is what makes it more handy. Specifying the binary does get it downloaded, but that sticks it in the wrong location, and obviously doesn't unpack it.

Failing with ..AssetResolverError: Too many matches found for trivy

While using https://github.com/devcontainers-contrib/features/blob/main/src/trivy/README.md devcontainer feature.

It fails with "nanolayer.installers.gh_release.resolvers.asset_resolver.AssetResolver.AssetResolverError: Too many matches found: ['trivy_0.45.0_Linux-ARM.tar.gz', 'trivy_0.45.0_Linux-ARM64.tar.gz']"

I have tried to use specific version of trivy but that did not help either

Full Logs are below

2023-09-25T12:34:13.775Z] [2/2] STEP 9/20: RUN --mount=type=bind,from=dev_containers_feature_content_source,source=trivy_3,target=/tmp/build-features-src/trivy_3     cp -ar /tmp/build-features-src/trivy_3 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/trivy_3  && cd /tmp/dev-container-features/trivy_3  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/trivy_3
[2023-09-25T12:34:13.892Z] ===========================================================================
Feature       : Trivy (via Github Releases)
Description   : Trivy is a comprehensive and versatile security scanner. It finds vulnerabilities, misconfigurations, secrets, SBOM in containers, Kubernetes, code repositories, clouds and more.
Id            : ghcr.io/devcontainers-contrib/features/trivy
Version       : 1.0.8
Documentation : http://github.com/devcontainers-contrib/features/tree/main/src/trivy
Options       :
    VERSION="0.45.0"
===========================================================================
[2023-09-25T12:34:15.318Z] nanolayer
[2023-09-25T12:34:18.479Z] cd /tmp/tmpji0jzszo && chmod +x -R . && _REMOTE_USER="vscode" _REMOTE_USER_HOME="/home/vscode" REPO="aquasecurity/trivy" BINARYNAMES="trivy" VERSION="0.45.0" ASSETREGEX="" RELEASETAGREGEX="" BINLOCATION="/usr/local/bin" LIBNAME="" LIBLOCATION="/usr/local/lib" NANOLAYER_VERBOSE="" NANOLAYER_FORCE_CLI_INSTALLATION="" NANOLAYER_PROPAGATE_CLI_LOCATION="1" NANOLAYER_CLI_LOCATION="/tmp/nanolayer-BFFuv88Cq4/nanolayer" bash  -i  +H ./install.sh
Found a pre-existing nanolayer which were given in env variable: /tmp/nanolayer-BFFuv88Cq4/nanolayer
[2023-09-25T12:34:18.591Z] skipping usage of pre-existing nanolayer. (required version v0.5.3 does not match existing version 0.5.3)
[2023-09-25T12:34:20.231Z] nanolayer
[2023-09-25T12:34:21.534Z] 'bom.json' filtered by '(\.json$|\.sbom$|\.blockmap$)' (negative)
[2023-09-25T12:34:21.534Z] 
[2023-09-25T12:34:21.535Z] 'trivy_0.45.0_checksums.txt' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)
[2023-09-25T12:34:21.536Z] 'trivy_0.45.0_checksums.txt.pem' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)
[2023-09-25T12:34:21.538Z] 
[2023-09-25T12:34:21.541Z] 'trivy_0.45.0_checksums.txt.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)
[2023-09-25T12:34:21.542Z] 'trivy_0.45.0_FreeBSD-32bit.tar.gz' filtered by '(32[Bb]it|32\-[Bb]it)' (negative)

'trivy_0.45.0_FreeBSD-32bit.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_FreeBSD-32bit.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_FreeBSD-64bit.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)
[2023-09-25T12:34:21.542Z] 'trivy_0.45.0_FreeBSD-64bit.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_Linux-32bit.deb' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-32bit.deb.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)
[2023-09-25T12:34:21.544Z] 'trivy_0.45.0_Linux-32bit.deb.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-32bit.rpm' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-32bit.rpm.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-32bit.rpm.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-32bit.tar.gz' filtered by '(32[Bb]it|32\-[Bb]it)' (negative)
[2023-09-25T12:34:21.544Z] 

'trivy_0.45.0_Linux-32bit.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_Linux-32bit.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_Linux-64bit.deb' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-64bit.deb.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-64bit.deb.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-64bit.rpm' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-64bit.rpm.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)
[2023-09-25T12:34:21.545Z] 
[2023-09-25T12:34:21.545Z] 'trivy_0.45.0_Linux-64bit.rpm.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)
[2023-09-25T12:34:21.546Z] 'trivy_0.45.0_Linux-64bit.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_Linux-64bit.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_Linux-ARM.deb' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM.deb.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM.deb.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM.rpm' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM.rpm.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM.rpm.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)
[2023-09-25T12:34:21.546Z] 'trivy_0.45.0_Linux-ARM.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)
[2023-09-25T12:34:21.547Z] 'trivy_0.45.0_Linux-ARM.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_Linux-ARM64.deb' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM64.deb.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM64.deb.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM64.rpm' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)
[2023-09-25T12:34:21.547Z] 'trivy_0.45.0_Linux-ARM64.rpm.pem' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM64.rpm.sig' filtered by '(\.deb|\.rpm|\.pkg|\.apk|\.[Aa]ppImage|\.snap)' (negative)

'trivy_0.45.0_Linux-ARM64.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_Linux-ARM64.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_Linux-PPC64LE.deb' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.deb.pem' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.deb.sig' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.rpm' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.rpm.pem' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.rpm.sig' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.tar.gz' filtered
[2023-09-25T12:34:21.548Z]  by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.tar.gz.pem' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-PPC64LE.tar.gz.sig' filtered by '(\-ppc|ppc64|PPC64|_ppc)' (negative)

'trivy_0.45.0_Linux-s390x.deb' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.deb.pem' filtered by '(s390x|s390)' (negative)
[2023-09-25T12:34:21.548Z] 'trivy_0.45.0_Linux-s390x.deb.sig' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.rpm' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.rpm.pem' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.rpm.sig' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.tar.gz' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.tar.gz.pem' filtered by '(s390x|s390)' (negative)

'trivy_0.45.0_Linux-s390x.tar.gz.sig' filtered by '(s390x|s390)' (negative)
[2023-09-25T12:34:21.549Z] 'trivy_0.45.0_macOS-64bit.tar.gz' filtered by '([Mm]ac[Oo][Ss]|[Mm]ac\-[Oo][Ss]|\-osx\-|_osx_|[Dd]arwin|\.dmg)' (negative)

'trivy_0.45.0_macOS-64bit.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_macOS-64bit.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_macOS-ARM64.tar.gz' filtered by '([Mm]ac[Oo][Ss]|[Mm]ac\-[Oo][Ss]|\-osx\-|_osx_|[Dd]arwin|\.dmg)' (negative)

'trivy_0.45.0_macOS-ARM64.tar.gz.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_macOS-ARM64.tar.gz.sig' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)

'trivy_0.45.0_windows-64bit.zip' filtered by '(windows|Windows|WINDOWS|win32|\-win\-|\.msi$|.msixbundle$|\.exe$)' (negative)

'trivy_0.45.0_windows-64bit.zip.pem' filtered by '(\.pub$|\.pem$|\.crt$|\.asc$|pivkey|pkcs11key)' (negative)

'trivy_0.45.0_windows-64bit.zip.sig' filtered by '(\.sig$|\.text
[2023-09-25T12:34:21.549Z] $|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)
[2023-09-25T12:34:21.550Z] 'trivy_0.45.0_FreeBSD-64bit.tar.gz' filtered by '([Aa]rm64|ARM64|\-ARM|\-arm\-)' 

'trivy_0.45.0_Linux-64bit.tar.gz' filtered by '([Aa]rm64|ARM64|\-ARM|\-arm\-)' 
[2023-09-25T12:34:21.551Z] 'trivy_0.45.0_Linux-ARM.tar.gz' filtered by '(?i)(ubuntu)' 

'trivy_0.45.0_Linux-ARM64.tar.gz' filtered by '(?i)(ubuntu)' 
[2023-09-25T12:34:21.551Z] 'trivy_0.45.0_Linux-ARM.tar.gz' filtered by '.*static.*' 

'trivy_0.45.0_Linux-ARM64.tar.gz' filtered by '.*static.*' 
[2023-09-25T12:34:21.552Z] 'trivy_0.45.0_Linux-ARM.tar.gz' filtered by '(?i)(debian)' 

'trivy_0.45.0_Linux-ARM64.tar.gz' filtered by '(?i)(debian)' 
[2023-09-25T12:34:21.578Z] Traceback (most recent call last):
[2023-09-25T12:34:21.579Z]   File "<string>", line 1, in <module>
[2023-09-25T12:34:21.579Z]   File "nanolayer.__main__", line 50, in main
[2023-09-25T12:34:21.580Z]   File "typer.main", line 328, in __call__
[2023-09-25T12:34:21.581Z]   File "typer.main", line 311, in __call__
[2023-09-25T12:34:21.581Z]   File "click.core", line 1157, in __call__

  File "typer.core", line 778, in main

  File "typer.core", line 216, in _main

  File "click.core", line 1688, in invoke

  File "click.core", line 1688, in invoke

  File "click.core", line 1434, in invoke

  File "click.core", line 783, in invoke

  File "typer.main", line 683, in wrapper

  File "nanolayer.cli.install", line 153, in install_gh_release_binary

  File "nanolayer.installers.gh_release.gh_release_installer", line 167, in install

  File "nanolayer.installers.gh_release.resolvers.asset_resolver", line 304, in resolve

nanolayer.installers.gh_release.resolvers.asset_resolver.AssetResolver.AssetResolverError: Too many matches found: ['trivy_0.45.0_Linux-ARM.tar.gz', 'trivy_0.45.0_Linux-ARM64.tar.gz']
[2023-09-25T12:34:21.692Z] Traceback (most recent call last):
[2023-09-25T12:34:21.692Z]   File "<string>", line 1, in <module>
[2023-09-25T12:34:21.692Z]   File "nanolayer.__main__", line 50, in main
[2023-09-25T12:34:21.693Z]   File "typer.main", line 328, in __call__
[2023-09-25T12:34:21.693Z]   File "typer.main", line 311, in __call__
[2023-09-25T12:34:21.693Z]   File "click.core", line 1157, in __call__
[2023-09-25T12:34:21.693Z]   File "typer.core", line 778, in main
[2023-09-25T12:34:21.693Z]   File "typer.core", line 216, in _main
[2023-09-25T12:34:21.693Z]   File "click.core", line 1688, in invoke
[2023-09-25T12:34:21.693Z]   File "click.core", line 1688, in invoke
  File "click.core", line 1434, in invoke
[2023-09-25T12:34:21.693Z]   File "click.core", line 783, in invoke
[2023-09-25T12:34:21.694Z]   File "typer.main", line 683, in wrapper
[2023-09-25T12:34:21.694Z]   File "nanolayer.cli.install", line 66, in install_devcontainer_feature
[2023-09-25T12:34:21.694Z]   File "nanolayer.installers.devcontainer_feature.oci_feature_installer", line 135, in install
[2023-09-25T12:34:21.694Z]   File "nanolayer.utils.invoker", line 59, in invoke
[2023-09-25T12:34:21.695Z] nanolayer.utils.invoker.Invoker.InvokerException: The command 'cd /tmp/tmpji0jzszo && chmod +x -R . && _REMOTE_USER="vscode" _REMOTE_USER_HOME="/home/vscode" REPO="aquasecurity/trivy" BINARYNAMES="trivy" VERSION="0.45.0" ASSETREGEX="" RELEASETAGREGEX="" BINLOCATION="/usr/local/bin" LIBNAME="" LIBLOCATION="/usr/local/lib" NANOLAYER_VERBOSE="" NANOLAYER_FORCE_CLI_INSTALLATION="" NANOLAYER_PROPAGATE_CLI_LOCATION="1" NANOLAYER_CLI_LOCATION="/tmp/nanolayer-BFFuv88Cq4/nanolayer" bash  -i  +H ./install.sh' failed. error: Return Code: 1. see logs for details.
[2023-09-25T12:34:21.770Z] ERROR: Feature "Trivy (via Github Releases)" (ghcr.io/devcontainers-contrib/features/trivy) failed to install! Look at the documentation at http://github.com/devcontainers-contrib/features/tree/main/src/trivy for help troubleshooting this error.
[2023-09-25T12:34:21.910Z] Error: building at STEP "RUN --mount=type=bind,from=dev_containers_feature_content_source,source=trivy_3,target=/tmp/build-features-src/trivy_3 cp -ar /tmp/build-features-src/trivy_3 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/trivy_3  && cd /tmp/dev-container-features/trivy_3  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/trivy_3": while running runtime: exit status 1

[2023-09-25T12:34:21.913Z] Stop (43876 ms): Run: podman buildx build --load --build-context dev_containers_feature_content_source=/var/folders/qf/35fdm64d3hz4k4hd0g11118c0000gn/T/devcontainercli/container-features/0.51.1-1695645206637 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:jammy --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-skycompute-7f44036d371c05cd11fee86088e03b0a833349e335a04856d31b932f24e300ba-features -f /var/folders/qf/35fdm64d3hz4k4hd0g11118c0000gn/T/devcontainercli/container-features/0.51.1-1695645206637/Dockerfile.extended /Users/pramod/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2023-09-25T12:34:21.914Z] Error: Command failed: podman buildx build --load --build-context dev_containers_feature_content_source=/var/folders/qf/35fdm64d3hz4k4hd0g11118c0000gn/T/devcontainercli/container-features/0.51.1-1695645206637 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:jammy --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-skycompute-7f44036d371c05cd11fee86088e03b0a833349e335a04856d31b932f24e300ba-features -f /var/folders/qf/35fdm64d3hz4k4hd0g11118c0000gn/T/devcontainercli/container-features/0.51.1-1695645206637/Dockerfile.extended /Users/pramod/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2023-09-25T12:34:21.914Z]     at gAA (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1066)
[2023-09-25T12:34:21.914Z]     at FK (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1002)
[2023-09-25T12:34:21.914Z]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2023-09-25T12:34:21.914Z]     at async FAA (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:3660)
[2023-09-25T12:34:21.914Z]     at async GC (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:4775)
[2023-09-25T12:34:21.914Z]     at async VeA (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:12240)
[2023-09-25T12:34:21.914Z]     at async WeA (/Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:11981)
[2023-09-25T12:34:21.919Z] Stop (55625 ms): Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/pramod/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --docker-path podman --container-session-data-folder /tmp/devcontainers-9b0664ea-e900-4c3c-aae8-0fb3a1a169b41695645205385 --workspace-folder /Users/pramod/github/playground/skycompute --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/pramod/github/playground/skycompute --id-label devcontainer.config_file=/Users/pramod/github/playground/skycompute/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/pramod/github/playground/skycompute/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-09-25T12:34:21.919Z] Exit code 1
[2023-09-25T12:34:21.920Z] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/pramod/.vscode/extensions/ms-vscode-remote.remote-containers-0.309.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/pramod/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --docker-path podman --container-session-data-folder /tmp/devcontainers-9b0664ea-e900-4c3c-aae8-0fb3a1a169b41695645205385 --workspace-folder /Users/pramod/github/playground/skycompute --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/pramod/github/playground/skycompute --id-label devcontainer.config_file=/Users/pramod/github/playground/skycompute/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/pramod/github/playground/skycompute/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-09-25T12:34:21.920Z] Exit code 1

Add apk support (alpine)

Add apk installer support for alpine

Install multiple exes from a GH Release

Hi there,

I'm struggling with copying 2 executables to a dev container from the same Github Release - can you help as I cant have multiple features with the same key in a devcontainer.json - so I need to get it working where it copies both files

		"ghcr.io/devcontainers-contrib/features/gh-release:1": {
			"repo": "Azure/apiops",
			"version": "v4.7.0",
			"binaryNames": "extractor",
			"assetRegex": "^(extractor.linux-x64.exe)"
		}

Above is working for the extractor, but I need to get the publisher too - can you help me define the syntax?

support for containerEnv

On debian-like distros, prevent interactive installs

On debian-like distros (ie, those with apt or apt-get) some packages will attempt to perform configuration or otherwise request user interaction during install. This is obviously fatal when spinning up a devcontainer.

Note that this is most often observed with tzdata, which is often a dependency of any package that performs user interaction or general UI functionality (eg, if the environment is supposed to be working with one of the standard 'nix UI frameworks).
The traditional way to work around this - especially if you only need a UTC zone - is to add DEBIAN_FRONTEND=noninteractive to the install line. Note that you don't want to set this as a permanent environment variable, as it would affect any command lines spawned once the container is brought up.

Current workaround is to install whichever packages would require interaction inside the base dockerfile, but this is annoying in some cases.

Tests of installers should run on container image and use current git code

currently tests are using binary dcontainer isntallation in devcontainer, which forces a build of a tagged commit in order to generate the dcontainer binary.
requirements are:

test on current git code
test on vary container images (debian, ubuntu, alpine etc)

devcontainer build broken when using Ubuntu due to alpine detection

I am unable to build a devcontainer based on Ubuntu due to featmake incorrectly matching my container build environment to alpine, which uses the apk package manager install of apt.

It looks like it is connected to this commit: 755e25b

See fix at #2

EDIT

Referenced wrong commit - fixed now

[Bug]: gh-release requires git as part of the install process, but does not install it itself.

more details here
devcontainers-contrib/features#569

Alpine compatibility

Alpine compatibility is required

GH-release does not decompress binaries that are compressed only with .gz without .tar

When installing binaries using gh-release that are only compressed with .gz without .tar, it installs the .gz file without decompressing it, causing the binary not to work.

/workspaces/codespaces-blank $ sudo nanolayer install gh-release jpillora/chisel chisel
'chisel_1.8.1_checksums.txt' filtered by '(\.sig$|\.text$|\.txt$|[Cc]hecksums|sha256|\.sha1$|\.md5$)' (negative)
'chisel_1.8.1_darwin_amd64.gz' filtered by '([Mm]ac[Oo][Ss]|[Mm]ac\-[Oo][Ss]|\-osx\-|_osx_|[Dd]arwin|\.dmg)' (negative)
'chisel_1.8.1_darwin_arm64.gz' filtered by '([Aa]rm64|ARM64|-ARM)' (negative)
'chisel_1.8.1_linux_386.gz' filtered by '(i386|-386|_386)' (negative)
'chisel_1.8.1_linux_arm64.gz' filtered by '([Aa]rm64|ARM64|-ARM)' (negative)
'chisel_1.8.1_linux_armv6.gz' filtered by '([Aa][Rr][Mm]v6)' (negative)
'chisel_1.8.1_linux_armv7.gz' filtered by '([Aa][Rr][Mm]v7)' (negative)
'chisel_1.8.1_linux_ppc64.gz' filtered by '(-ppc|ppc64|PPC64|_ppc)' (negative)
'chisel_1.8.1_linux_ppc64le.gz' filtered by '(-ppc|ppc64|PPC64|_ppc)' (negative)
'chisel_1.8.1_linux_s390x.gz' filtered by '(s390x|s390)' (negative)
'chisel_1.8.1_windows_386.gz' filtered by '(i386|-386|_386)' (negative)
'chisel_1.8.1_windows_amd64.gz' filtered by '(windows|Windows|WINDOWS|win32|\-win\-|\.msi$|.msixbundle$|\.exe$)' (negative)
'chisel_1.8.1_windows_arm64.gz' filtered by '([Aa]rm64|ARM64|-ARM)' (negative)
'chisel_1.8.1_windows_armv6.gz' filtered by '([Aa][Rr][Mm]v6)' (negative)
'chisel_1.8.1_windows_armv7.gz' filtered by '([Aa][Rr][Mm]v7)' (negative)
'chisel_1.8.1_linux_mips64le_hardfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mips64le_softfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mips64_hardfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mips64_softfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mipsle_hardfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mipsle_softfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mips_hardfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_mips_softfloat.gz' filtered by '([Aa]md64|-x64|x64|x86[_-]64)' 
'chisel_1.8.1_linux_amd64.gz' filtered by '(?i)(ubuntu)' 
'chisel_1.8.1_linux_amd64.gz' filtered by '.*static.*' 
'chisel_1.8.1_linux_amd64.gz' filtered by '(?i)(debian)' 
resolved asset: chisel_1.8.1_linux_amd64.gz
asset recognized as a binary

/workspaces/codespaces-blank $ chisel
bash: /usr/local/bin/chisel: cannot execute binary file: Exec format error

/workspaces/codespaces-blank $ sha256sum /usr/local/bin/chisel
0461e84f847489e8968b011128b6be6b001f487ae75b2a0c14ff6d4eafc9f2df  /usr/local/bin/chisel

The compressed file chisel_1.8.1_linux_amd64.gz has SHA256 of 0461e84f847489e8968b011128b6be6b001f487ae75b2a0c14ff6d4eafc9f2df.

The binary file chisel_1.8.1_linux_amd64 has SHA256 of c237f1a3f75b2759f66ec741448bb352e95e186a9a689f87c8641b44a13d878b

Add support for declaration of test scenarios in feature-definition

AssetResolverError: No matches found even though the binary exists.

When attemping to install porjo/youtubeuploader, it says no matches found even though the binary exists named youtubeuploader_23.02_Linux_x86_64.tar.gz

/workspaces/codespaces-blank $ sudo nanolayer install gh-release porjo/youtubeuploader youtubeuploader
Traceback (most recent call last):

  File "/usr/local/py-utils/bin/nanolayer", line 8, in <module>
    sys.exit(main())

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/nanolayer/__main__.py", line 50, in main
    app()

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/typer/main.py", line 328, in __call__

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/typer/main.py", line 311, in __call__

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/click/core.py", line 1130, in __call__

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/typer/core.py", line 778, in main

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/typer/core.py", line 216, in _main

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/click/core.py", line 1404, in invoke

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/click/core.py", line 760, in invoke

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/typer/main.py", line 683, in wrapper

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/nanolayer/cli/install.py", line 159, in install_gh_release_binary
    GHReleaseInstaller.install(

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/nanolayer/installers/gh_release/gh_release_installer.py", line 163, in install
    resolved_asset = AssetResolver.resolve(

  File "/usr/local/py-utils/venvs/nanolayer/lib/python3.10/site-packages/nanolayer/installers/gh_release/resolvers/asset_resolver.py", line 224, in resolve
    raise cls.AssetResolverError("No matches found")

nanolayer.installers.gh_release.resolvers.asset_resolver.AssetResolver.AssetResolverError: No matches found

Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

It is better to have a verbose option to debug why nanolayer can't find the binary.

Consider adding as subproject to features repo

Similar to the cookie cutter template, I suggest combining this repository into the features repository.

This would make it easier to see how a feature is generated. Since this cli repo is only used to generate features, there's very very tight coupling between it and the features repo. This strongly indicates that they should be versioned together.

Later, if at some point this CLI grows, it can and should be split out into it's own repo again.

This is something to consider. You can close this issue as "Not planned" if this isn't something we want to tackle.

Add apt-get, apt, aptitude and gh-release installers to dcontainer

these will serve as a basis for composing features

microdnf support

Apologies if this is the wrong place to raise this, but would it be possible to support microdnf and dnf installers?

Thanks.

Enhancement: ability to install a feature from devcontainer-contrib cli

add the ability to install a feature from devcontainer-contrib cli

(and deprecate "featmake" bash script)

dcontainer apt-get installation capabilities

apt-get installation capabilities including cache pruning, ppa adding and error handling would be beneficial as a beuilding steps for devcontainer features who rely on apt-get packages

Release an aarch64 linux build

Can you add aarch64 linux musl to your releases builds?

It could be useful for Alpine container on Apple Silicon machines :-)

Feature install fail

dcontainer CLI fails feature installation.

The error is:

#8 [internal] load metadata for mcr.microsoft.com/devcontainers/base:jammy
------
 > [dev_containers_target_stage  5/18] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=vela-asdf_1,target=/tmp/build-features-src/vela-asdf_1     cp -ar /tmp/build-features-src/vela-asdf_1 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/vela-asdf_1  && cd /tmp/dev-container-features/vela-asdf_1  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/vela-asdf_1:
#16 5.653   File "click.core", line 1657, in invoke
#16 5.653   File "click.core", line 1404, in invoke
#16 5.653   File "click.core", line 760, in invoke
#16 5.653   File "typer.main", line 683, in wrapper
#16 5.653   File "dcontainer.cli.feature", line 94, in install_command
#16 5.653   File "dcontainer.cli.install.install_feature", line 16, in install_feature
#16 5.653   File "dcontainer.oci.oci_feature_installer", line 60, in install
#16 5.653   File "dcontainer.oci.oci_feature_installer", line 174, in _resolve_remote_user
#16 5.653 TypeError: getpwnam() argument must be str, not bool
#16 5.683 ERROR: Feature "./features/src/vela-asdf" (Unknown) failed to install!
------
ERROR: failed to solve: executor failed running [/bin/sh -c cp -ar /tmp/build-features-src/vela-asdf_1 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/vela-asdf_1  && cd /tmp/dev-container-features/vela-asdf_1  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/vela-asdf_1]: exit code: 1

The error is caused by mismatching arguments between install_feature.py and: oci_feature_installer.py (v0.1.2):

# install_feature.py

def install_feature(
    feature: str,
    options: Optional[Dict[str, str]] = None,
    remote_user: Optional[str] = None,
    verbose: bool = False,
) -> None:
    OCIFeatureInstaller.install(OCIFeature(feature), options, remote_user, verbose)

# oci_feature_installer.py

@classmethod
def install(
    cls,
    feature_oci: OCIFeature,
    options: Optional[Dict[str, Union[str, bool]]] = None,
    envs: Optional[Dict[str, str]] = None,
    remote_user_name: Optional[str] = None,
    verbose: bool = False,
) -> None:

The issue is that remote_user is passed as envs and verbose as remote_user_name.

remove cache related flags

currently many cache related flags are poluting the interface. Since the purpose of this project is to enable as slim installation as possible, the options to disable this behavior is adding unnecessary confusion

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.