semaphoreci/artifact

artifact's Issues

Flag: --expire-in 10d or -e 10d

`sem artifact` would be the only cli-way to download the artifacts, when will it ship?

Afaict, sem artifact would be the only "cli-way" to download artifacts. We'd really like that feature.

Its marked as a TODO, is there an estimation when it would ship?
🙏

Artifact yank seems to be broken on v0.4.5

Something is wrong with yanking on artifact v0.4.5.

0.4.4

semaphore@semaphore-vm:~/toolbox$ artifact pull job test-results/junit.json
2021/05/04 09:45:18 successful pull for current job
semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json
2021/05/04 09:45:26 successful yank for current job
semaphore@semaphore-vm:~/toolbox$ 

0.4.5

semaphore@semaphore-vm:~/toolbox$ artifact pull job test-results/junit.json
2021/05/04 09:34:57 successful pull for current job
semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json
2021/05/04 09:35:11 http status fail
2021/05/04 09:35:11 Artifact not found. Please check if the artifact you are trying to yank exists.

Running yank with -v flag gives more information about that:

semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json -v
2021/05/04 10:09:33 initiating artifact...
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	initiating artifact...	{"token": "d493f2ff-8350-46b7-a6ae-57918b9a82ee"}
2021/05/04 10:09:33 artifact initiated
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	artifact initiated	{"org URL": "https://semaphore.semaphoreci.com", "gatewayAPIBase": "/api/v1/artifacts", "gatewayAPI": "https://semaphore.semaphoreci.com/api/v1/artifacts"}
2021/05/04 10:09:33 YankPath
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	YankPath	{"input file": "test-results/junit.json", "output file": "test-results/junit.json"}
2021/05/04 10:09:33 http status fail
2021-05-04T10:09:33.910Z	WARN	log/log.go:67	http status fail	{"while doing": "Upload", "url": "https://storage.googleapis.com/vfb77u567adcgpqcdeframasmygvcp/artifacts/jobs/5a450a1f-8dd0-4382-be7a-f519edb63726/test-results/junit.json?Expires=1620124173&GoogleAccessId=artifacthub%40semaphore2-prod-artifacts.iam.gserviceaccount.com&Signature=KGOAA2sqyV13DsxkgAttqQiAksEuQqnWgNCm%2FYkWUvrrezpDlSNYUNP8WWELGai%2BPHQFCgkJPCg33LMjE20%2FMWD%2FXHSw26hCWbRlIl0VXnFYWVB8aVEyqa2MJIjFNoHZZfL%2FPhsxbhbNJzmQ0J%2Bf25ix61rY4RAlHmT4jNDFTn80MwMzRpIJkde0DQaG7ephCpvw6ypOz60H5ezrk5tkz8VDvXBJhZCb0OQ58x%2FDH8ekuOkkRQgs%2FkBfE1i2ZrjS7OGGhoLqHwz2PqiblN%2BKPg%2Bz%2FHZvtrX0uNT9OvmKkUCRQnhxyOfaej8jPqjEHnQY5F%2FfDFXTPqeKDK4qIsXtPA%3D%3D", "status code": 403, "status": "Forbidden", "content": "<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>PUT\n\n\n1620124173\n/vfb77u567adcgpqcdeframasmygvcp/artifacts/jobs/5a450a1f-8dd0-4382-be7a-f519edb63726/test-results/junit.json</StringToSign></Error>"}
github.com/semaphoreci/artifact/pkg/util/log.Logger.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:67
github.com/semaphoreci/artifact/pkg/util/log.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:120
github.com/semaphoreci/artifact/pkg/util/http.formatIfErr
	/home/semaphore/artifact/pkg/util/http/httputil.go:31
github.com/semaphoreci/artifact/pkg/util/http.do
	/home/semaphore/artifact/pkg/util/http/httputil.go:47
github.com/semaphoreci/artifact/pkg/util/http.DeleteURL
	/home/semaphore/artifact/pkg/util/http/httputil.go:77
github.com/semaphoreci/artifact/pkg/gcs.doYankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:468
github.com/semaphoreci/artifact/pkg/gcs.YankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:458
github.com/semaphoreci/artifact/cmd.runYankForCategory
	/home/semaphore/artifact/cmd/yank.go:30
github.com/semaphoreci/artifact/cmd.glob..func8
	/home/semaphore/artifact/cmd/yank.go:46
github.com/spf13/cobra.(*Command).execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/semaphoreci/artifact/cmd.Execute
	/home/semaphore/artifact/cmd/root.go:32
main.main
	/home/semaphore/artifact/main.go:6
runtime.main
	/usr/local/golang/1.12/go/src/runtime/proc.go:200
2021/05/04 10:09:33 Artifact not found. Please check if the artifact you are trying to yank exists.
2021-05-04T10:09:33.910Z	WARN	log/log.go:67	Artifact not found. Please check if the artifact you are trying to yank exists.
github.com/semaphoreci/artifact/pkg/util/log.Logger.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:67
github.com/semaphoreci/artifact/pkg/util/log.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:120
github.com/semaphoreci/artifact/pkg/gcs.YankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:460
github.com/semaphoreci/artifact/cmd.runYankForCategory
	/home/semaphore/artifact/cmd/yank.go:30
github.com/semaphoreci/artifact/cmd.glob..func8
	/home/semaphore/artifact/cmd/yank.go:46
github.com/spf13/cobra.(*Command).execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/semaphoreci/artifact/cmd.Execute
	/home/semaphore/artifact/cmd/root.go:32
main.main
	/home/semaphore/artifact/main.go:6
runtime.main
	/usr/local/golang/1.12/go/src/runtime/proc.go:200

Project, workflow, job level expire values to config file

The default for all three is "never", give the possibility to set these globals.

Regression in latest artifact push workflow

A regression was introduced today which is breaking all of our builds. Pushing nested folders results in a missing middle folder!

Here is an example with the following dir structure

- workdir
  - test2
    - test_file
    -  nested
      - nested_file

semaphore@semaphore-vm:~/workdir$ mkdir test2
semaphore@semaphore-vm:~/workdir$ touch test2/test_file
semaphore@semaphore-vm:~/workdir$ mkdir test2/nested
semaphore@semaphore-vm:~/workdir$ touch test2/nested/nested_file
semaphore@semaphore-vm:~/workdir$ artifact push workflow --force test2
2022/05/19 23:28:09 successful push for current workflow

This results in a workflow artifact directory of:

- test2
  - test_file
  - nested_file

We have downstream blocks depending on a specific file structure, so the fact the artifact push workflow is now flattening them is quite problematic, breaking builds for about 20 of our projects. Can somebody take a 👀 at the changes released in semaphore internally today for regressions?

Flag: `--force` of `-f`

Using `/dev/stdin` as an input no longer works

It seems like in a recent release it's no longer possible to pipe data into artifact push /dev/stdin - the artifact is created as a zero-size file. My suspicion is that this is because of this change c86f55d which tries to stat the file (which has no size) and always sends an empty body as a result.

Implement `artifact push job SINGLE_FILE_PATH`

Push, pull, yank for directory

Typo error in `path_resolver` package

artifact/pkg/files/path_resolver.go

Line 52 in 2f90de9

return nil, fmt.Errorf("project ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it")

Current:
"project ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it"

Excepted:
"job ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it"

Restore of dir with a single file restores file to dir name instead

To reproduce:

mkdir savepath
echo "some file content" > savepath/file.txt
artifact push workflow savepath

# change node
rm -r savepath
artifact pull workflow savepath
ls -al savepath
>> -rw-rw-r-- 1 semaphore semaphore 18 Aug 24 11:59 savepath
cat savepath
>> some file content

This is unexpected, and erases the actual filename.
If there are 2 files in the pushed folder the script behaves as expected.

A workaround is to tar the folder first and save the tar, but this is less than ideal for a typically small number of files.

Is flag --config being used?

@vendelin8 👋 I'm currently writing documentation page for artifact CLI and I'm not sure if this flag is intended or deprecated.

When running artifact pull job (without arguments) I noticed the following lines in the output log

Global Flags:
      --config string   config file (default is $HOME/.artifact.yaml)

Store expire dates for pushed files to the bucket

Expire data is stored in the same Bucket, where the files are pushed.
Home for expire data is /var/expires-in/. When a file is pushed with an expire date other than never, an object is created with

• filename: <expire_timestamp_in_seconds>-<random_postfix>
• content: the path in the Bucket to delete after the expire date

Implement `artifact yank job SINGLE_FILE_PATH`

Finish project setup on Semaphore. (Use makefile approach from semaphore/cli)

Flag `--destination` or `-d sets` destination directory or file path

Implement `artifact pull job SINGLE_FILE_PATH`

Flag: --job <job-id> or -j <job-id>