Giter Site home page Giter Site logo

vmware-tanzu / build-tooling-for-integrations Goto Github PK

View Code? Open in Web Editor NEW
7.0 5.0 4.0 283 KB

This project enables developers to start building and packaging new Tanzu integrations, including cluster packages and custom CLI commands.

License: Apache License 2.0

Go 69.08% Dockerfile 8.83% Shell 3.97% Makefile 18.12%
integration tanzu tooling

build-tooling-for-integrations's People

Contributors

avi-08 avatar codegold79 avatar dependabot[bot] avatar fmanning07 avatar github-actions[bot] avatar jmoroski avatar rajathagasthya avatar sathyanarays avatar seemiller avatar vandy-p avatar vmwghbot avatar web-flow avatar yharish991 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

build-tooling-for-integrations's Issues

Address dependency security alerts

Describe the bug
We got dependabot alerts that there are security vulnerabilities in one of the dependencies we use. We need to upgrade golang.org/x/net depdency to v0.7.0 or later to address this issue.

Update golangci-lint version

Is your feature request related to a problem? Please describe.

We should update golangci-lint to the latest version v1.52.2.

In addition, we should also remove the following linters as they are deprecated:

deadcode
structcheck
varcheck

Describe the solution you'd like

A new version of golangci-lint to be run during linting.

Describe alternatives you've considered

Keep the old version and miss out on improvements.

Additional context

package-tooling-image dockerfile should allow the tag to be configurable

Is your feature request related to a problem? Please describe.

On line 26 of package-tooling-image/Dockerfile, the tag is currently hardcoded to main

RUN go install github.com/vmware-tanzu/build-tooling-for-integrations/package-tools@main

Describe the solution you'd like

This tag should be configurable.

Add `govulncheck` support during linting

Is your feature request related to a problem? Please describe.

govulncheck is used to detect vulnerabilities in Go code (https://go.dev/blog/vuln). We should integrate this as part of the linting process.

Describe the solution you'd like

govulncheck to detect vulnerabilities on every change.

Describe alternatives you've considered

Additional context

Update and ensure contributor expectations are clear

Describe the bug

There are gaps in telling community members what is expected of them when contributing to this repository.

Additional context

This issue can be updated to include more expectations that are not clear in current documentation.

Expectations that should be written out explicitly

  • All PRs, regardless of how small, are expected to have an associated issue. Yes, even PRs for fixing typos or updating copyright years. Fixed by #76.
  • add more here

Fix release process in Build tooling repo

Describe the bug
Currently, the release process in build tooling takes an incorrect commit and creates the release. This results in the version not getting updated in the template Makefile and package tooling docker image in the release branch

Allow users to specify which plugins they want to build and publish while using Build Tooling for Integrations

Is your feature request related to a problem? Please describe.

After PR #59 is merged, Build Tooling for Integrations will build and publish all plugins in the cmd/plugin directory. There is no way to select which plugins in the directory to build and publish.

Describe the solution you'd like

@sathyanarays wrote in a comment (#59 (comment)),

... in the docker build targets, we provide an option to select the components by setting some environment variables. If this is true, we need to maintain parity with other make targets, right?

It would be nice to be able to select which plugins are built and published.

Describe alternatives you've considered

I had considered allowing for a flag to be configured that Tanzu CLI Plugin Builder has available for the build command. Here is the help information on that flag:

 --match string              match a plugin name to build, supports globbing (default "*")

However, it does not meet the parity requirement @sathyanarays mentioned as it only allows for choosing plugins that match a string. I think having such an option would only be confusing to users. It would be better not to have the ability to choose rather than have an ability that behaves so differently than building controllers using Build Tooling for Integrations.

Additional context

Tanzu Plugin Builder is maintained at https://github.com/vmware-tanzu/tanzu-cli/tree/main/cmd/plugin/builder

Any changes to the --match or plugin selection ability will need to be done in that repository.

Typo in comment

Describe the bug

In repo_bundle_generate.go: PacakageMetadata => PackageMetadata

Enable passing in goflags for plugin builds

Is your feature request related to a problem? Please describe.
When I build in an orphaned git submodule, I get an error,

building local repository at ./artifacts/plugins, v0.1.0, [all]
🦠- building plugin at path "cmd/plugin/feature"
🦠- running godep path "cmd/plugin/feature"
ðŸ¦$ /usr/local/go/bin/go mod download
🦠- $ /usr/local/go/bin/go build -o /workspace/artifacts/plugins/linux/arm64/kubernetes/feature/v0.1.0/tanzu-feature-linux_arm64 -ldflags  -X 'github.com/vmware-tanzu/tanzu-plugin-runtime/plugin/buildinfo.Version=v0.1.0' -tags  ./.
🦠- error: exit status 1
🦠- output: error obtaining VCS status: exit status 128
 	Use -buildvcs=false to disable VCS stamping.
🦠- error compiling plugin feature

To get passed this error, I need to pass in the goflag, -buildvcs=false.

Describe the solution you'd like
Enable passing in goflags for plugin builds.

Describe alternatives you've considered

  • change Docker context to include parent of submodule
  • use Git Clone
  • copy over git submodule's parent's .git file to submodule

Additional context
Add any other context or screenshots about the feature request here.

Validate `COMPONENTS` variable in Makefile

Is your feature request related to a problem? Please describe.
Developers sometimes might not set the COMPONENTS variable in the right format, so we need to validate the COMPONENTS variable and report it if it is not set in the right format

Describe the solution you'd like
The solution is to have a make target that validates the COMPONENTS variable and report if it is not set in the right format

Add SBOM generation for Go Modules

Is your feature request related to a problem? Please describe.

Build tooling should include targets/commands for generating SBOM for each Go module in the repo it's being used. SBOM should be generated in SPDX format.

https://github.com/opensbom-generator/spdx-sbom-generator is a good tool to use.

Describe the solution you'd like

When I build my modules, I'd like SBOMs generated and output to a deterministic place with a deterministic name.

Describe alternatives you've considered

Additional context

Add example seed projects to the repository

Is your feature request related to a problem? Please describe.

As a developer who is willing to use build tooling, would like to see example seed projects in the repository to make it easy for me to get started quickly.
Examples that we can be add are:

  1. Integration with one go module
  2. Integrations with multiple go modules

Prefer local development use cases for environment variable defaults

Is your feature request related to a problem? Please describe.
As an engineer trying to get started with this build tooling, I must first configure several environment variables.

Describe the solution you'd like
Preferred Experience
Copy the Makefile into a project and run make init without having to configure anything else.

High Level Design
Today the init process downloads a container and an image. If the init process also deployed a local registry server then there would be no need to require the user to configure additional values to point to a docker registry, or provide credentials.

They'd still be free to change these values if they have more specialized needs, but this would provide a configuration that worked out of the box.

A single local registry could be shared between multiple projects. This might lead to a design decision for build-tooling to maintain some stateful data on a local development machine. The stateful information relevant to this feature would be the endpoint (including port) to access the local registry. This would allow other projects to check if a local registry has already been deployed and started.

Additional context
By choosing this pattern we are optimizing the local development experience. This isn't really at the expense of the CI pipeline experience, because there is no reliable way to optimize defaults for CI pipelines across all intended users. Those workflows will require environment variables to be overridden. That is a reasonable expectation of a pipeline configuration.

CC: @navidshaikh

Use apt-get and tanzu builder install to install Tanzu CLI plugin builder instead of cloning and building from source code

Is your feature request related to a problem? Please describe.

When work on building CLI plugins started, the ability to download Tanzu CLI plugin builder using apt-get was not available. At that time, I decided to–as a temporary measure–git clone and build the Tanzu CLI plugin builder from source code.

Describe the solution you'd like

Now that apt-get is available, use that instead of building from source code.

Describe alternatives you've considered

None

Additional context

None

Add CVE scan workflow for pull requests

Is your feature request related to a problem? Please describe.
As a developer/maintainer of build-tooling-for-integrations repository, I would like to have a CVE scan as part of PR checks to know if my changes introduce any vulnerabilities.

Support a way to configure GOPROXY when building images

Is your feature request related to a problem? Please describe.
The default value of GOPROXY is https://proxy.golang.org,direct, but sometimes we might want to download the dependencies from a different proxy for various reasons. So the build tooling should support a way to configure the GOPROXY when building images

Describe the solution you'd like
Pass an arg when building the docker image and use that arg in the Dockerfile and set it as an env var

Package tooling image fails to start

Describe the bug
Running make package-vendir-sync target fails with these errors:

Digest: sha256:7e8660bb12c067a1d5da31c8debd6f183eecb0b24bac57185609840e50405dbe
Status: Downloaded newer image for ghcr.io/vmware-tanzu/package-tooling:v0.0.2
Stopping and removing any existing local docker registry...
Error response from daemon: No such container: registry
Starting local docker registry...
Unable to find image 'mirror.gcr.io/library/registry:2' locally
docker: Error response from daemon: manifest for mirror.gcr.io/library/registry:2 not found: manifest unknown: Failed to fetch "2" from request "/v2/library/registry/manifests/2".
See 'docker run --help'.

Reference workflow: https://github.com/vmware-tanzu/tanzu-framework/actions/runs/4936068189/jobs/8823154641?pr=4604

To Reproduce
Steps to reproduce the behavior:

  1. Run make package-vendir-sync target
  2. See error docker: Error response from daemon: manifest for mirror.gcr.io/library/registry:2 not found: manifest unknown: Failed to fetch "2" from request "/v2/library/registry/manifests/2"

Expected behavior
The entrypoint script of package-tooling image tries to run an image referenced from mirror.gcr.io, which is an caching repository and only retains frequently accessed images. From the docs:

Container Registry adds frequently requested images to the cache so they are available for future requests. It also periodically removes images that are no longer requested.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

build-tooling-getting-started questions

Describe the bug

Some areas of the build-tooling-getting-started.md is unclear and I have some questions.

Questions

In the section Steps to use the build tooling, it says,

Copy the contents of the Makefile

Where do I copy the Makefile to? My Documents folder? The root directory of a plugin repository? Can I follow the COMPONENTS example where it uses COMPONENTS ?= featuregates.featuregates-controller-manager.featuregates, and put a copy of the Makefile in vmware-tanzu/framework/featuregates/controller?

In step 2, it says,

...is the name of the package, this should be same as the directory name that holds the package definition of featuregates in packages directory that's in the project's root directory.

I'm not sure what it means by "name of the package". What do you mean by "package"? Is that a Carvel term? A Go term? And what is a "package definition"? Might be worth having links to the definitions/examples of packages and package definition.

In step 4, it says,

If you are using one of the seed projects, you need to customize the existing package in that project...

How do I customize the existing project?

In step 6, it says,

Following env vars should be set when running the above make target

    OCI_REGISTRY - remote OCI registry where you would like to push the images and package bundles.
    IMG_VERSION_OVERRIDE - image tag for the image to be built.
    PACKAGE_REPOSITORY - package repository of the package bundles.
    PACKAGE_VERSION - package version of the package bundle being built.
    PACKAGE_SUB_VERSION - package subversion of the package bundle being built.
    REGISTRY_USERNAME - OCI registry username.
    REGISTRY_PASSWORD - OCI registry password.
    REGISTRY_SERVER - OCI registry server url.

What are examples of what the OCI_REGISTRY, IMG_VERSION_OVERRIDE, PACKAGE_REPOSITORY, PACKAGE_VERSION, PACKAGE_SUB_VERSION look like? What is the difference between OCI_REGISTRY and REGISTRY_SERVER?

What rules do we need to follow for the above mentioned variable? Or maybe there are no rules. For version, can I use anything I want?

Generate thick repo bundle when generating repo bundle in package tooling

Is your feature request related to a problem? Please describe.
Right now when we generate repo bundle using package tooling, it just creates a bunch of package CRs and bundles them in a imgpkg bundle, and to consume that repo bundle it needs to be pushed to an OCI registry.

We also want to generate a thick repo bundle to support airgap scenarios and to provide teams a way to publish the repo bundle and all of its associated contents to their own OCI registry

Support running envtests in build tooling

Is your feature request related to a problem? Please describe.
Integration tests for controllers are written using envtest, right now this is not supported in build tooling and we need a way to support that in build tooling so that we can run integration tests as part of the test suite.

Pull request template isn't being used

Describe the bug
This repo has a pull request template. Because of its file location it isn't being used as the default template when creating a new PR.

To Reproduce
Try to open a new Pull Request

  1. Go to 'Pull Requests' from main repo page
  2. Click on 'New pull request' button
  3. Observe that the template isn't being used

Expected behavior
Clicking on 'New pull request' should use the pull request template as its default starting point.

Make version and URL of downloaded imgpkg binary configurable in Build Tooling For Integrations

Background Information

Currently, the Carvel imgpkg tool is used by Build Tooling for Integrations to build and publish both controllers and Tanzu CLI plugins.

The imgpkg URL for package-tools (for building and publishing controllers) is stored in a config file located in package-tools/cmd/config/carvel-tools-config.yaml:

url: https://github.com/vmware-tanzu/carvel-imgpkg/releases/download/{{ .VERSION }}/imgpkg-{{ .OS }}-{{ .ARCH }} 

The imgpkg URL for plugins is stored in templates/Dockerfile:

then wget -O /bin/imgpkg https://github.com/vmware-tanzu/carvel-imgpkg/releases/download/${IMGPKG_VERSION}/imgpkg-linux-${TARGETARCH} && \ 

At one point (code was not merged), the imgpkg version for plugins was retrieved via a config file that was not always reachable:

IMGPKG_VERSION ?= $(shell awk 'BEGIN{ RS="- " }{ if( $$2=="imgpkg" ){ print $$4 } }' ../../package-tools/cmd/config/carvel-tools-config.yaml)

This issue was created in response to a few feedback comments from @sathyanarays and @avi-08 for a pull request:

Feature Description:

Build Tooling for Integrations should get the Carvel imgpkg version and URL using a common pathway or a common place.

Expected Outcome:

  • The URL and version should be configurable by the end user.
  • Enable end users to download the Carvel imgpkg binary from a private, internal location, and not from the public internet.
  • The URL and versions used by package-tools and building/publishing Tanzu CLI plugins should by default be the same. According to @yharish991, divergent imgpkg versions may cause problems.

Ensure output packages are reproducible over time

Describe the bug
Given a commit ID or TAG, the output of make all should be identical over time. In the current state of Makefile, there are references to the image quay.io/hyayiv/packaging:latest and the executables inside this image are being used to build the packages, push them to OCI registry, etc.,. The underlying image pointed by quay.io/hyayiv/packaging:latest is subject to change over time. So, it's better use a specific tag instead of latest.

To Reproduce

  • Run make all
  • Slightly change the packaging behavior, build the image quay.io/hyayiv/packaging:latest and push
  • Run make all again
  • The outputs produced in the first step and third step might be different.

Expected behavior
Same version of Makefile should produce identical outputs on running make all. The output should not be dynamic and not dependent on external factors.

Screenshots
N/A

Provide issue template for documentation gaps

Is your feature request related to a problem? Please describe.
The existing issue templates aren't a great match for reporting documentation-related issues

Describe the solution you'd like
A new issue template for creating doc-related issues. I'd also like labels automatically added to issues.

Describe alternatives you've considered
Creating a blank issue as a starting point is possible, but doesn't promote consistency.

Add support for building client libraries

Is your feature request related to a problem? Please describe.
Right now the build tooling doesn't support building client libraries, we should have a way to support that.

Revisit golangci lint rules

Describe the bug
.golangci.yaml contains the lint rules that we enforce with the build tooling, we inherited these rules from tanzu-framework repo, and we need to revisit these rules

Add a release workflow

Describe the solution you'd like

Add a GitHub workflow action that automates the release process.

When a tag is created, the docker images for the Dockerfiles in the package-tooling-image and root directories should be created and pushed to ghcr.io/vmware-tanzu.

support building submodules in build tooling

Is your feature request related to a problem? Please describe.
Right now build tooling supports building only git repos without submodules, but we also need to support the use case of building submodules too.

Bump golang version to 1.19

Is your feature request related to a problem? Please describe.
The current version of build tooling uses golang 1.18, we need to bump it to 1.19

Fix "/bin/sh: line 0: [: too many arguments" error when trying to build multiple Tanzu CLI plugins and "cmd/plugin: No such file or directory" error when doing make verify

Describe the bug

When I do make cli-plugin-build, there are no errors. However, when I specify multiple plugins, e.g. make cli-plugin-build CLI_PLUGINS="plugin-demo-bar plugin-demo-foo", I get an error:

/bin/sh: line 0: [: too many arguments

Also, when I do make verify in another repo, I get,

/bin/sh: line 0: cd: cmd/plugin: No such file or directory

To Reproduce
Steps to reproduce the behavior:

  1. Go to examples/multi-module-integration
  2. Run make cli-plugin-build CLI_PLUGINS="plugin-demo-bar plugin-demo-foo"
  3. See "too many arguments" error
  4. Go to Tanzu Framwork repo root
  5. Run make verify
  6. See "no such directory" error

Expected behavior
I expect no error if I specify plugins or I don't specifiy plugins

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: macOS
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Verify build tooling on a window machine

Is your feature request related to a problem? Please describe.
So far we have used the build tooling on linux and mac machines, but we have not got a chance to test and use the build tooling on a windows machine. This issue is for tracking that and identifying any issues with using build tooling on a windows machine

Docker build fails with build-context set

Describe the bug
The docker build command in the Make file fails. It gets fixed if the line #syntax=docker/dockerfile:1.4 is moved to the top

To Reproduce

  • Create an empty directory
  • Copy the Makefile and Dockerfile from the `templates' directory to the newly created directory
  • Copy the module1, module2 and packages directory
  • Run make init
  • Run make all

This can also be reproduced by running the following docker command instead of make all:

docker build . -f Dockerfile --target lint --build-context component=module1

Expected behavior
The docker build succeeds

Screenshots

$ make all
make COMPONENT="module1" lint
make[1]: Entering directory '/home/sathyanarays/hello-world-plugin'
cp .golangci.yaml module1
docker build . -f Dockerfile --target lint --build-context component=module1
[+] Building 0.1s (2/2) FINISHED                                                                                                                                                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s
 => => transferring dockerfile: 1.54kB                                                                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s
error: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: unsupported frontend capability moby.buildkit.frontend.contexts
make[1]: *** [Makefile:49: lint] Error 1
make[1]: Leaving directory '/home/sathyanarays/hello-world-plugin'
make: *** [Makefile:24: "module1"?"module1"?"module1"] Error 2

Desktop (please complete the following information):

$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Docker version

$ docker version
Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:57 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:03 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Buildx version

$ docker buildx version
github.com/docker/buildx v0.8.2-docker 6224def4dd2c3d347eee19db595348c50d7cb491

envtest fails on mac M1

Describe the bug
make test target fails to execute envtests on mac M1

To Reproduce
Steps to reproduce the behavior:

  1. Run make test
  2. See below error:
=> ERROR [test 1/1] RUN --mount=target=.     --mount=type=cache,target=/go/pkg/mod     --mount=type=cache,target=/root/.cache/go-build     --mount=from=test-base,src=/bi  1.8s
------                                                                                                                                                                           
 > [test 1/1] RUN --mount=target=.     --mount=type=cache,target=/go/pkg/mod     --mount=type=cache,target=/root/.cache/go-build     --mount=from=test-base,src=/bin/k8s,target=/bin/k8s     cd readiness/controller && mkdir /out && KUBEBUILDER_ASSETS=/bin/k8s/1.26.1-linux-amd64 go test -v -coverprofile=/out/cover.out ./...:
#10 0.602 ?     github.com/vmware-tanzu/tanzu-framework/readiness/controller    [no test files]
#10 1.659 === RUN   TestResourceExistenceCondition
#10 1.659 Running Suite: ResourceExistenceCondition Suite - /workspace/readiness/controller/pkg/conditions
#10 1.659 ================================================================================================
#10 1.659 Random Seed: 1683105524
#10 1.659 
#10 1.659 Will run 4 of 4 specs
#10 1.659 ------------------------------
#10 1.659 [BeforeSuite] [FAILED] [0.004 seconds]
#10 1.659 [BeforeSuite] 
#10 1.659 /workspace/readiness/controller/pkg/conditions/resourceexistencecondition_test.go:40
#10 1.659 
#10 1.659   Timeline >>
#10 1.659   STEP: bootstrapping test environment @ 05/03/23 09:18:44.13
#10 1.659   2023-05-03T09:18:44Z        DEBUG   controller-runtime.test-env     starting control plane
#10 1.659   2023-05-03T09:18:44Z        ERROR   controller-runtime.test-env     unable to start the controlplane        {"tries": 0, "error": "fork/exec /bin/k8s/1.26.1-linux-amd64/etcd: no such file or directory"}

Expected behavior
/bin/k8s/1.26.1-linux-amd64/etcd: no such file or directory This line indicates that platform architecture is not correct. It should be referring to 1.26.1-linux-arm64 binary.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Add a make target to only docker image without publishing it

Is your feature request related to a problem? Please describe.
To support airgap scenarios, we need a way to only build images without publishing them

Describe the solution you'd like
Add a new make target called docker-build-all to support building docker images without publishing them

Bump golang version to 1.18

Is your feature request related to a problem? Please describe.
The current version of build tooling uses golang 1.17, we need to bump it to 1.18

Cache the Tanzu CLI binary during Docker build stage so it's not installed again

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

The current approach installs Tanzu CLI for each plugin it is building, we need to optimize this just to install only once.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Cache the Tanzu CLI binary so it's not installed again.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Copy the binaries back to the host machine after building the image

Is your feature request related to a problem? Please describe.
Right now the build tooling only generates the image for the go modules, but we also need to copy the generated binary back to the host machine so that developers can run it locally to test the functionality.

Describe the solution you'd like
Add a build make target to do the above.

Document environment variables that need to be set before running `make all`

Describe the bug
This page provides documentation on adding a package by running make all command. But, the Makefile expects a bunch of environment variables to be set for proper execution. Some of the variables are as follows:

KBLD_CONFIG_FILE_PATH
IMAGE
PACKAGE_NAME
OCI_REGISTRY
PACKAGE_VERSION
PACKAGE_SUB_VERSION
PACKAGE_REPOSITORY
REGISTRY_USERNAME
REGISTRY_PASSWORD
REGISTRY_SERVER
PACKAGE_VALUES_FILE
REPO_BUNDLE_VERSION
REPO_BUNDLE_SUB_VERSION

No documentation has been provided on the meaning of these values and how to set them. The documentation for these are necessary as they are also inputs for the make all command.

Investigate build tooling not working in Linux machines

Describe the bug
Docker desktop claims that buildx is installed by default when v0.8 or up is installed, but for some reason, buildx is not installed on Linux machines by default causing the build tooling unusable. We need to investigate why buildx is not working on linux machines and if a straight forward resolution is not found, we need to revert the changes and support building multiple projects with build tooling.

Update build-tooling templates for a multi-module project

Is your feature request related to a problem? Please describe.
Existing build tooling templates (Makefile and Dockerfile) can perform test-build-containerise for a single module. However, using it is difficult in a multi-module project like tanzu-framework where these modules can have inter-dependencies. As there is no way to reference module dependecies from the same project, users will have to raise multiple PRs in order to add changes which span across these modules.

Describe the solution you'd like
A way to be able to refer project-local dependencies while building a component, so that all relevant cross-module changes can be added in a single PR. Refer this PR for expected changes.

Describe alternatives you've considered
NA

Additional context
NA

Docker cache gets invalidated for unit-test stage

Describe the bug
templates/Makefile defines the test target as:

test: fmt vet
	$(DOCKER) build . -f Dockerfile --target test --build-arg COMPONENT=$(COMPONENT) --build-arg GOPROXY_ARG=$(GOPROXY)
	@$(DOCKER) build . -f Dockerfile --target unit-test-coverage --build-arg COMPONENT=$(COMPONENT) --output build/$(COMPONENT)/coverage

The Dockerfile makes use of shared cache for go modules across various stages. Everything works expected except in building unit-test-coverage stage; the cache gets invalidated and go modules is computed again.

To Reproduce
Steps to reproduce the behavior:

  1. Run make test
  2. Console logs will show unit-test stage re-computing the go mod download from base layer

Expected behavior
Cache should be computed once and re-used across the Docker build stages. This is due to missing GOPROXY argument in the Makefile for invoking unit-test-coverage stage of Docker. This PR contains the relevant changes to fix it.

Screenshots
NA

Desktop (please complete the following information):
NA

Smartphone (please complete the following information):
NA

Additional context
Add any other context about the problem here.

Git dubious ownership error when using package tooling image

Describe the bug
Git in versions over v2.30.3 added an owner check for the top-level directory, it complains about dubious ownership if the git repository directory is not owned by the current user. golang:1.19 base image used in the package tooling image brings a new git version, so we are seeing an issue with running git commands in the package tooling image.

The solution is to mark the git project directory as a safe directory using the command git config --global --add safe.directory

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.