Giter Site home page Giter Site logo

oras-project / oras Goto Github PK

View Code? Open in Web Editor NEW
1.3K 24.0 153.0 6.13 MB

OCI registry client - managing content like artifacts, images, packages

Home Page: https://oras.land

License: Apache License 2.0

Go 97.99% Makefile 0.92% Dockerfile 0.15% Shell 0.94%
oci docker registry storage

oras's Introduction

oras's People

Contributors

1shubham7 avatar deitch avatar dependabot[bot] avatar distorhead avatar enraiha0307 avatar estesp avatar feynmanzhou avatar jasminetmsft avatar jdolitsky avatar jpreese avatar junczhu avatar katexochen avatar khanjasir90 avatar kranurag7 avatar lachie83 avatar linuxsuren avatar lizmsft avatar mxey avatar nathana1 avatar pytimer avatar qweeah avatar richicoder1 avatar sajayantony avatar shizhmsft avatar stevelasker avatar terryhowe avatar wangxiaoxuan273 avatar wju-msft avatar wwwsylvia avatar yuehaoliang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oras's Issues

Friendly Error Output

Pulling with no tags shows

oras pull localhost:5000/hello
Error: object required

Default tag should not be used and should output error message like

Error: image reference format is invalid. Please specify <name:tag|name@digest>

Does not handle artifacts without fully qualified names gracefully

To repro the issue use an image that is not fully qualified as seen below.

 1 ❯ oras push test:test .
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/deislabs/oras/vendor/github.com/containerd/containerd/reference.Spec.Hostname(0xc000022bb0, 0x9, 0x0, 0x0, 0x147b544, 0xc0001db788)
...

Push without an tag

Users may want to push a manifest without a tag. That is, with digest only.

oras push localhost:5000/hello hi.txt

The above command should output the digest.

Related #77

Convenient way to change config media type

Projects, which use oras to push artifacts, may not care about the content of the config but they do care about the media type.

We should support an option so that they can change config media type in a convenient way.

Simplifying the ORAS client experience

Simplifying the ORAS Client Experience

When pushing artifacts to an OCI compliant registry, a manifest.config.mediaType is required to identify the artifact, and layer.config.mediaTypes are required for each layer.

As we look to improve on the current experiences, the current ORAS client requires either too little, defaulting in the wrong defaults, or too much, creating a less than productive experience, easy to make mistakes.

To provide a progressive disclosure experience, I'm proposing two changes:

  1. Provide for an orasConfig file, enabling the user to provide the repetitive values to the oras client.
  2. ORAS provides for default unkown mediaTypes for the artifact and layers. #115

Current Basic Example

Walking through a simple example, the user specifies the artifact path and the files.

oras login demo42.azurecr.io -u $USR -p $PWD 
oras push demo42.azurecr.io/samples/docs-in-markdown:v1 \
    ./readme.md \
    ./detail.md

This generates a manifest that defaults to application/vnd.oci.image.config.v1+json for the artifact and vnd.oci.image.layer.v1.tar for the layers.

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:532eaabd9574880dbf76b9b8cc00832c20a6ec113d682299550d7a6e0f345e25",
      "size": 4,
      "annotations": {
        "org.opencontainers.image.title": "orasTest.txt"
      }
    }
  ]
}

This is incorrect, as application/vnd.oci.image.config.v1+json and vnd.oci.image.layer.v1.tar are reserved for container images that would be instanced with the docker or containerD client. For the initial experience to be correct, the user must specify their type, which they may not be ready to think about.

This is the context behind the two-stage proposal.

Providing mediaTypes

To properly set the mediaTypes with the current version of the ORAS client, the user would have to provide a much more complex cli call:

oras push demo42.azurecr.io/samples/docs-in-markdown:1 \
  --manifest-config ./config.json:application/vnd.contoso.docs-in-markdown.config.v1+json \
  ./readme.md:application/vnd.contoso.docs-in-markdown.layer.v1+md \
  ./detail.tar:application/vnd.contoso.docs-in-markdown.layer.v1+tar

This isn't something the user would want to type each time. Considering these values don't change very often, requiring them to be entered each time leads to frustration and errors.

Optimizing for Usage

There are a few small enhancements we could make, such as a config file for defining the artifact mediaType.

artifactType: application/vnd.contoso.docsinmarkdown.config.v1+json

However, what do we do about the layer mediaTypes?

I considered a mapping of extensions to mediaTypes

artifactType: application/vnd.contoso.docs-in-markdown.config.v1+json
layerTypes:
    - md: application/vnd.contoso.docs-in-markdown.layer.v1+md
    - tar: application/vnd.contoso.docs-in-markdown.content.v1.tar

However, this doesn't scale well as an artifact could likely use the same .tar or even .md extension for several layer types.

Even in the simplest case, the user would still need to pass in the collection of files, each time.

oras push demo42.azurecr.io/samples/docs-in-markdown:v1 \
    ./readme.md \
    ./detail.md

orasConfig

Artifact authors define these types once for an artifact. They continue to iterate, possibly adding new files. To simplify ORAS usage, either as a client or go library, an orasConfig is proposed.

The elements to be stored in the mapping file include:

example: demo42.azurecr.io

  • artifactName: The content addressable path to the artifact.

    example: demo42.azurecr.io/samples/docs-in-markdown:v1

  • artifactType: the single mediaType used to define the artifact.

    example: application/vnd.contoso.docs-in-markdown.config.v1+json

  • layerTypes: a collection of layer types used for the artifact.

    example:

    • application/vnd.oci.image.layer.v1.tar
    • application/vnd.oci.image.layer.v1.tar+gzip
  • files: a collection of files to upload, mapped to a specific type, with wildcard support to minimize having to specify every file.

    example:

    • ./readme.md
    • ./description.md
    • ./docs/*.md

Smart Defaults

Using a config file, the user could minimize the content they would type with the ORAS client:

oras push -c orasConfig.yaml

or, we could make orasConfig.yaml a default, if found:

oras push

If the orasConfig.yaml isn't found, the ORAS client generates an error.

orasConfig Format

The usual yaml vs. json format debate is the obvious, but lets consider this file is mostly user editable. There's likely not a lot of content. For this reason, even with the normal space vs. tab yaml issues, I'm proposing the following yaml format.

version: "0.1-preview"
name: demo42.azurecr.io/samples/docs-in-markdown:v1
artifactType: application/vnd.contoso.docs-in-markdown.config.v1+json
layers:
  - readme:
    - type: application/vnd.contoso.docs-in-markdown.readme.layer.v1+md 
    - files: 
      - "./readme.md"
  - content:
    - type: application/vnd.contoso.docs-in-markdown.content.layer.v1+md 
    - files:
      - "./index.md"
      - "./docs/*.md"

Why the Nested Layer Collections?

The types of artifacts likely group into 3 different scenarios:

  1. There are 1 or more layers. The order may, or may not be important, but all the types are the same.
  2. There are multiple layer types, however the ordering isn't important.
  3. There are multiple layer types, and the ordering is important.

Some artifact types, like the docker/OCI image format, depend on ordinal layers to build the filesystem layout. In these cases, it's critical the layers can be specified in a specific order, with their respective types.

In the above example, the first type readme.layer could be repeated after content.layer. This example is a bit contrived, but it future-proofs the design to support the three scenarios, without too much complexity.

orasConfig Tokens

To support dynamic environments, environment variables would be supported.

version: "0.1-preview"
name: {{$registry}}/samples/docs-in-markdown:{{$tag}}
artifactType: application/vnd.contoso.docs-in-markdown.config.v1+json
layers:
  - readme:
    - type: application/vnd.contoso.docs-in-markdown.readme.layer.v1+md 
    - files: 
      - "./readme.md"
  - content:
    - type: application/vnd.contoso.docs-in-markdown.content.layer.v1+md 
    - files:
      - "./index.md"
      - "./docs/*.md"

Once the orasConfig.yaml file is defined, the user simply add/edits files in the collection. Or, iterates on the types. They can then simply type oras push from the directory of the orasConfig.yaml file.

Correlation to Artifact Mapping Schema

The above proposal is not meant to be a replacement of the Artifact Mapping Definition

There are two use cases for development and publishing

  1. Making the artifact well known to registry operators. This is the purpose of the Artifact Mapping. Artifact Authors will define their types, with content related to the name of their type and reference information on all instances of the type.
  2. Pushing specific artifacts to registries. For each artifact pushed, there's a collection of files, mapped to artifact and layer types. This is the purpose of this configuration.

What's in a Name?

The thing I'm most debating, and looking for suggestions, is how to name the 3 types of "config" files.

Assuming we agree to the following scenarios:

  1. Defining the content of a specific artifact - the purpose of this proposal
  2. Defining the artifact type, for consumption by registry operators. See Artifact Mapping Definition for more info.
  3. Defining the schema of manifest.config.json, used by artifact tooling. See Config Schema Validation for more info.

Working reference of names:

fileName Scenario
orasConfig.yaml 1 - the collection of files and types, used by the ORAS client/library when pushing to an OCI compliant registry. The focus of this proposal.
artifactMapping.json 2 - defining the type, used by registry operators.
artifactMappingSchema.json 2 - defining the schema used by artifactMapping.json
config.json 3 the optional config.json file used in the oci-manifest to define configuration values of the specific artifact pushed to a registry.
artifactConfigSchema.json 3 provides schema validation for a specific artifact type, when the artifact optionally uses a config file.

Conversation

  • What do people think about simplifying the ORAS client experience?
  • Does orasConfig make sense when ORAS is used as a library?
  • What's the config filename (propose orasConfig.yaml) ?
  • The schema of the config file?

Make push/pull information out of verbose

New experience will be

$ oras pull example.azurecr.io/wordpress:5.7.0
Downloaded c7de1a51f942 wordpress-5.7.0.tgz
Pulled example.azurecr.io/wordpress:5.7.0
Digest: sha256:ce6a4473161afbc005507a2e2cd516afb2a51c9b3678210dc22bb9390ac1f7d3

Do not allow path traversal by default when pushing package

Do not allow path traversal by default when pushing package. It's a gesture to indicate that the tool considers this as a potential security issue, and it helps to avoid generating package with security hole unexpected. User may choose to allow it but it should not be the default

Updating from commit 550ed34caf4a to 40d20966e9ea introduces ^M newline characters in code

Example:

diff --git a/vendor/github.com/deislabs/oras/pkg/oras/push_opts.go b/vendor/github.com/deislabs/oras/pkg/oras/push_opts.go
index b4d75daf5..fd60d666a 100644
--- a/vendor/github.com/deislabs/oras/pkg/oras/push_opts.go
+++ b/vendor/github.com/deislabs/oras/pkg/oras/push_opts.go
@@ -4,6 +4,7 @@ import (
        "path/filepath"
        "strings"
 
+       "github.com/containerd/containerd/images"^M
        orascontent "github.com/deislabs/oras/pkg/content"
 
        ocispec "github.com/opencontainers/image-spec/specs-go/v1"
@@ -15,6 +16,7 @@ type pushOpts struct {
        configAnnotations   map[string]string
        manifestAnnotations map[string]string
        validateName        func(desc ocispec.Descriptor) error
+       baseHandlers        []images.Handler^M
 }
 
 func pushOptsDefaults() *pushOpts {
@@ -98,3 +100,12 @@ func ValidateNameAsPath(desc ocispec.Descriptor) error {
 
        return nil
 }
+^M
+// WithPushBaseHandler provides base handlers, which will be called before^M
+// any push specific handlers.^M
+func WithPushBaseHandler(handlers ...images.Handler) PushOpt {^M
+       return func(o *pushOpts) error {^M
+               o.baseHandlers = append(o.baseHandlers, handlers...)^M
+               return nil^M
+       }^M
+}^M

This is just a subset of the diff, but these ^M characters are scattered throughout

ECR support

Hello,

I tried using ORAS with ECR support and I get Error: unexpected response: 404 Not Found
I know the project is still in its infancy but I wanted to hack on it in AWS and I was wondering if anyone got it to work. Anything I can do to help test this? :)

Update dependency to resolve test failure

Build and test are failing due bad dependency.

Sending build context to Docker daemon  4.899MB
Step 1/4 : FROM golang:1.11
 ---> 943a5a7ecac8
Step 2/4 : WORKDIR /go/src/github.com/deislab/oras
 ---> Running in 7b6a065e4570
Removing intermediate container 7b6a065e4570
 ---> e1e3ba161997
Step 3/4 : COPY . .
 ---> 01a91c7466c7
Step 4/4 : RUN make test
 ---> Running in 8b87e155213d
go get -u github.com/golang/dep/cmd/dep
/go/bin/dep ensure -v --vendor-only
(1/59) Wrote github.com/gorilla/[email protected]
(2/59) Wrote github.com/morikuni/aec@master
(3/59) Wrote github.com/prometheus/client_model@master
(4/59) Wrote github.com/opencontainers/[email protected]
(5/59) Wrote github.com/prometheus/[email protected]
(6/59) Wrote github.com/pmezard/[email protected]
(7/59) Wrote github.com/opencontainers/[email protected]
(8/59) Wrote github.com/prometheus/procfs@master
(9/59) Wrote github.com/phayes/[email protected]
(10/59) Wrote github.com/inconshreveable/[email protected]
(11/59) Wrote github.com/kardianos/osext@master
(12/59) Wrote github.com/konsorten/[email protected]
(13/59) Wrote github.com/matttproud/[email protected]
(14/59) Wrote github.com/pkg/[email protected]
(15/59) Wrote github.com/miekg/dns@0d29b283ac0f967dd3a02739bf26a22702210d7a
(16/59) Wrote github.com/prometheus/[email protected]
(17/59) Wrote rsc.io/letsencrypt (from https://github.com/dmcgowan/letsencrypt.git)@master
(18/59) Wrote github.com/sirupsen/[email protected]
(19/59) Wrote github.com/opencontainers/[email protected]
(20/59) Wrote github.com/spf13/[email protected]
(21/59) Wrote github.com/spf13/[email protected]
(22/59) Wrote github.com/stretchr/[email protected]
(23/59) Wrote github.com/yvasiyarov/go-metrics@master
(24/59) Wrote github.com/yvasiyarov/[email protected]
(25/59) Wrote github.com/yvasiyarov/newrelic_platform_go@b21fdbd4370f3717f3bbd2bf41c223bc273068e6
(26/59) Wrote golang.org/x/net@master
(27/59) Wrote golang.org/x/sync@master
(28/59) Wrote golang.org/x/crypto@master
(29/59) Wrote golang.org/x/time@master
(30/59) Wrote golang.org/x/sys@master
(31/59) Wrote golang.org/x/[email protected]
(32/59) Wrote github.com/xenolf/lego@a9d8cec0e6563575e5868a005359ac97911b5985
(33/59) Wrote gopkg.in/square/[email protected]
(34/59) Wrote gopkg.in/[email protected]
(35/59) Wrote github.com/Azure/go-ansiterm@master
(36/59) Wrote google.golang.org/[email protected]
(37/59) Wrote github.com/Shopify/logrus-bugsnag@master
(38/59) Wrote github.com/docker/[email protected]
(39/59) Wrote github.com/Microsoft/[email protected]
(40/59) Wrote github.com/bshuster-repo/[email protected]
(41/59) Wrote github.com/beorn7/perks@master
(42/59) Wrote github.com/bugsnag/[email protected]
(43/59) Wrote google.golang.org/genproto@master
(44/59) Failed to write github.com/bugsnag/bugsnag-go@109a9d63f2b57c4623f4912b256bac4a7a63517d
(45/59) Failed to write github.com/containerd/[email protected]
(46/59) Failed to write github.com/containerd/continuity@master
(47/59) Failed to write github.com/davecgh/[email protected]
(48/59) Failed to write github.com/docker/[email protected]
(49/59) Failed to write github.com/docker/libtrust@master
(50/59) Failed to write github.com/docker/docker@master
(51/59) Failed to write github.com/docker/[email protected]
(52/59) Failed to write github.com/docker/go-metrics@master
(53/59) Failed to write github.com/docker/[email protected]
(54/59) Failed to write github.com/gogo/[email protected]
(55/59) Failed to write github.com/docker/[email protected]
(56/59) Failed to write github.com/garyburd/[email protected]
(57/59) Failed to write github.com/gofrs/[email protected]
(58/59) Failed to write github.com/golang/[email protected]
(59/59) Failed to write github.com/gorilla/[email protected]
grouped write of manifest, lock and vendor: error while writing out vendor tree: failed to write dep tree: failed to export github.com/bugsnag/bugsnag-go: fatal: failed to unpack tree object 109a9d63f2b57c4623f4912b256bac4a7a63517d
: exit status 128
make: *** [vendor] Error 1
Makefile:40: recipe for target 'vendor' failed
The command '/bin/sh -c make test' returned a non-zero code: 2

Update help for push command

The push command does not show anything regarding syntax to push different media types, for example

oras push localhost:5000/hello:latest hi.txt:application/vnd.me.hi bye.txt:application/vnd.me.bye

instead:

Push files to remote registry

Usage:
  oras push name[:tag|@digest] file [file...] [flags]

Logo

As we all know, every amazing project needs an amazing logo

Add Go examples section to docs

The level of detail in the docs is great, but it would also be nice to have end-to-end examples of using the Go package for persons looking for a quick copy-paste (similar to whats at the bottom of README currently)

Cross-platform support on path

On windows, running

oras push localhost:5000/oras:win dir\hello.txt

results in

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:0d74c782a0fd750336d9703eb9995985255bb5dd383c28d1828a6379a5418e4e",
      "size": 14,
      "annotations": {
        "org.opencontainers.image.title": "dir\\hello.txt"
      }
    }
  ]
}

Pulling that image under Linux/Unix will result in

-rw-rw-rw- 1 shizh shizh 14 Apr 11 18:12 'dir\hello.txt'

All paths should be converted to slash-seperated.

Security discussions on FileStore

FileStore allows

  • Write to any relative path (e.g. foo/bar, ../../foo/bar) or absolute path /etc/foo/bar
  • Overwrite by default

The above scenarios are potentional security issues for pulling from public registries.

Suggestion: allow options for the above scenarios.

Usage as a Go library with Go modules

I'm relatively new to using Go modules, so some of this might be a little incorrect. A word of warning.

I wanted to use oras as a library, and successfully tried out the example provided in examples. I then wanted to recreate the functionality in oras which reuses the docker config to authenticate (in my case against an Azure registry. That led me to the code in the auth package.

In theory it should be enough to:

auth "github.com/deislabs/oras/pkg/auth/docker"

And then liberally copy the code from https://github.com/deislabs/oras/blob/master/cmd/oras/resolver.go#L22-L30.

In theory...

In practice this imports docker, which doesn't use tags on the repo to the go mod liking, which means you get an old version of Docker which imports logrus as with a capital letter "S" which causes the world to explode.

I think that means Oras is using Docker v17.03.2-ce, which was shipped 2 years ago https://github.com/moby/moby/releases.

go: github.com/Sirupsen/[email protected]: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"                                                                                           
go: error loading module requirements

Some context for this issue sirupsen/logrus#451.

I've got this working by:

  1. importing the latest version of Docker
go get github.com/docker/docker master
go mod tidy
  1. Updating where the AuthConfig type comes from in https://github.com/deislabs/oras/blob/master/pkg/auth/docker/resolver.go#L8
+ "github.com/docker/cli/cli/config/types"
- "github.com/docker/docker/api/types"

I think the fix here is to:

  • move over to modules from dep (this might be optional)
  • Find a suitable, modern, Docker tag to depend on
  • Update the types as noted above

At the moment anyone using Modules (recommended with 1.12) will run into this issue I think. It's possible this doesn't hit you if you're on macOS due to the case-incensitive filesystem, I've not tested that. The above was done on Linux.

Pushing multiple container images in the same repo with Oras

Background: I'm looking at pushing CNAB Bundles to OCI registries using Oras - I am currently only pushing the bundle descriptor (a bundle.json file) using a custom media type, and the next step is to also push all container images referenced in the bundle.

Now, I could push the exported image archive, but the entire image would result in a single layer (not ideal when working with large images) - and I'm interested in how Oras deals with pushing (multiple) container images in the same repository.

Thanks!

New ORAS Specification

I suggest that:

  • the admins of deislabs org create a new repo at deislabs/oras-spec
  • a new ORAS Specification is created, based upon the content of this PR
  • any tools using the ORAS CLI/Go module should adhere to this spec by default
  • build and launch a simple informational webpage (e.g. oras-spec.org)

The ORAS Specification should be a superset of the OCI Distribution Specification.

The primary goal is so that tools wishing to leverage OCI for arbitrary artifact distribution have an opinionated framework for doing so.

The secondary goal is to continue to garner support and prove out the ORAS method of doing things, and potentially/eventually/hopefully making this a part of OCI proper.

A non-goal of the spec is to fragment the OCI ecosystem. It is not meant to divide, but to further enable and embolden orgs that wish to achieve multi-artifact support today. If you are reading this as a registry provider - your thoughts here are appreciated (even if it is that of indifference).

cc: @SteveLasker @sajayantony @glyn @jzelinskie @bacongobbler @lachie83

Add new store that complies with OCI Image Layout Spec

Add new OCIStore at pkg/content/oci.go that strictly follows the OCI Image Layout Spec.

Example OCI Image storage layout on filesystem:

./index.json
./oci-layout
./blobs/sha256/3588d02542238316759cbf24502f4344ffcc8a60c803870022f335d1390c13b4
./blobs/sha256/4b0bc1c4050b03c95ef2a8e36e25feac42fd31283e8c30b3ee5df6b043155d3c
./blobs/sha256/7968321274dc6b6171697c33df7815310468e694ac5be0ec03ff053bb135e768

oci-layout looks like

{
    "imageLayoutVersion": "1.0.0"
}

example index.json looks like

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "size": 7143,
      "digest": "sha256:0228f90e926ba6b96e4f39cf294b2586d38fbb5a1e385c05cd1ee40ea54fe7fd",
      "annotations": {
        "org.opencontainers.image.ref.name": "stable-release"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 7143,
      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "annotations": {
        "org.opencontainers.image.ref.name": "v1.0"
      }
    },
    {
      "mediaType": "application/xml",
      "size": 7143,
      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
      "annotations": {
        "org.freedesktop.specifications.metainfo.version": "1.0",
        "org.freedesktop.specifications.metainfo.type": "AppStream"
      }
    }
  ],
  "annotations": {
    "com.example.index.revision": "r124356"
  }
}

The archive importer and exporter from containerd has a lot of good stuff:

https://github.com/containerd/containerd/blob/129942ca4d5764be1af04e03a30c841ddb452e11/images/archive/importer.go#L82

https://github.com/containerd/containerd/blob/b7f093eaa2785af9ca41ea27d0fd2a15a34603a2/images/archive/exporter.go#L323

cc @bacongobbler @shizhMSFT @sajayantony

Pulling a single file from a repository

Supposing I pushed multiple files to a repository using oras - is it possible / reasonable to be able to pull only a single file (layer, in the current format), referenced by digest?

In the example below, to only pull the layer corresponding to the first file, sha256:3fe24f297ab8bddcdef54e0fc016114e55e064d6a9e43834fd10963ba99d3cae?

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:3fe24f297ab8bddcdef54e0fc016114e55e064d6a9e43834fd10963ba99d3cae",
      "size": 138,
      "annotations": {
        "org.opencontainers.image.title": "Makefile"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:040fc227a60a0de08a8083cee60bec935af2bebf81d665a3a8c0401988948a17",
      "size": 1082,
      "annotations": {
        "org.opencontainers.image.title": "LICENSE"
      }
    }
  ]
}

ORAS mediaTypes should default to unknown

When pushing artifacts to an OCI compliant registry, a manifest.config.mediaType is required to identify the artifact, and layer.config.mediaTypes are required for each layer.

As we look to improve on the current experiences, the current ORAS client requires either too little, defaulting in the wrong defaults, or too much, creating a less than productive experience, easy to make mistakes.

To provide a progressive disclosure experience, I'm proposing two changes:

  1. Provide for an orasConfig file, enabling the user to provide the repetitive values to the oras client. See #114
  2. ORAS provides for default unknown mediaTypes for the artifact and layers.

Current Basic Example

Walking through a simple example, the user specifies the artifact path and the files.

oras login demo42.azurecr.io -u $USR -p $PWD 
oras push demo42.azurecr.io/samples/docs-in-markdown:v1 \
    ./orasTest.txt

This generates a manifest that defaults to application/vnd.oci.image.config.v1+json for the artifact and vnd.oci.image.layer.v1.tar for the layers.

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:532eaabd9574880dbf76b9b8cc00832c20a6ec113d682299550d7a6e0f345e25",
      "size": 4,
      "annotations": {
        "org.opencontainers.image.title": "orasTest.txt"
      }
    }
  ]
}

This is incorrect, as application/vnd.oci.image.config.v1+json and vnd.oci.image.layer.v1.tar are reserved for container images that would be instanced with the docker or containerD client. For the initial experience to be correct, the user must specify their type, which they may not be ready to think about.

When no artifact or layer mediaTypes are provided, this proposal says ORAS should default:
manifest.config.mediaType = application/vnd.unknown.config.v1+json
layer.mediaType = application/vnd.unknown.layer.v1.tar

See this OCI Artifacts PR 7 for more info.

Support insecure registry

Oras version:

$ oras version
Version:        0.6.0
Go version:     go1.12.4
Git commit:     b285197778e05cd348abb8ff50faf0ef7e3554a2
Git tree state: clean

What happened

Now i use oras to login the harbor registry, it failed. The error log:

$ oras login docker.hikcloud:30001
Username: admin
Password:
INFO[0005] Error logging in to v2 endpoint, trying next endpoint: Get https://docker.registry/v2/: x509: certificate has expired or is not yet valid
Error: Get https://docker.registry/v2/: x509: certificate has expired or is not yet valid

I use the docker client and config the insecure-registries, it login successed.

I see the oras code about login, i find the oras can't setting the insecure-registries parameter, so i hope the oras can add this flags when login the registry.

Is it my idea correct? If this idea ok, i can implement this feature.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.