Comments (18)
ECR strictly implements image-spec v1.1.0-rc2
with the subject
field. ECR does not support the artifactType
field, which is introduced since image-spec v1.1.0-rc3
and causing the "Invalid JSON syntax" problem. oras v1.1.0
implements image-spec v1.1.0-rc4
and oras attach
dropped the support for previous RC versions of image-spec
.
Note that oras v1.0.x
works with ECR as it implements image-spec v1.1.0-rc2
.
In summary, this issue is a feature request to bring back the --image-spec
option and introduce options like v1.1
, v1.1.0-rc2
.
from oras.
I would treat rejecting an unknown property as generating an error.
Totally agree. I was more pointing at SHOULD NOT vs MUST NOT.
I also completely agree that the spirit of this passage is something implementors should follow, but I don't think you can say that a registry is non-compliant if it refuses unknown fields on a known media type.
I also feel that there are a number of non-OCI compliant registries out there, some intentionally so for good reasons
I think this is the main observation for maintainers here to consider. Not supporting this category of registries limits the use of oras attach
, likely going forward beyond 1.1 GA.
from oras.
oras attach
also stopped working for the Gitlab container registry. I assumed it was a gitlab regression but it is not been resolved in a few updates of Gitlab so it might be oras or an incompatibility between the two. This might be related to what you are seeing with ECR.
Error: PUT "https://reg.git.example.com/v2/ktarplee/test/telemetry/base/manifests/sha256:2ed182a012cd258381a134d4475d66460388fbeeb91333a3627d47a92ae6fc38": response status code 400: manifest blob unknown: blob unknown to registry
from oras.
I just hit this issue for a push, and downgrading worked for me! Specifically:
wget https://github.com/oras-project/oras/releases/download/v1.0.1/oras_1.0.1_linux_amd64.tar.gz
# places ./oras in pwd
tar -xzvf oras_1.0.1_linux_amd64.tar.gz
./oras push <container> <targz>
....
Pushed [registry] ......
Digest: sha256:a93a09035767b94185b20cdec3f7bdc5331425ff9a4ddf76e5c93febf3bea390
So it's fairly easy to grab an old release, extract to just the PWD, use as needed, and you are good :) But also we should hopefully have support for a fix in the newer versions (I have 1.0.1 installed on my system, which produced the error above).
from oras.
We chose to remove the flag --image-spec
from oras attach
in ORAS v1.1.0 because there was a breaking change in the OCI Image Spec v1.1.0-rc4 that removed the OCI Artifact Manifest. This has been discussed in #1043 and documented in the ORAS Spec.
For registries that implement image-spec v1.1.0-rc2
, an workaround is to use ORAS v1.0.1 that allows to specify --image-spec v1.1-image
with oras attach
.
This sounds more like a registry compatibility issue with ORAS. To address this issue, I am fine with bringing --image-spec
back to oras attach
to increase the ORAS client compatibility.
from oras.
The slack conversation is here for reference.
from oras.
I have 1.0.1 installed on my system, which produced the error above
@vsoch Do you mean 'which produced the logs above'?
from oras.
@vsoch Do you mean 'which produced the logs above'?
No, I was referencing the error in the original issue.
from oras.
oras attach
also stopped working for the Gitlab container registry. I assumed it was a gitlab regression but it is not been resolved in a few updates of Gitlab so it might be oras or an incompatibility between the two. This might be related to what you are seeing with ECR.Error: PUT "https://reg.git.example.com/v2/ktarplee/test/telemetry/base/manifests/sha256:2ed182a012cd258381a134d4475d66460388fbeeb91333a3627d47a92ae6fc38": response status code 400: manifest blob unknown: blob unknown to registry
@ktarplee From the error logs, it doesn't reveal the root cause so it is hard to determine that this is an incompatibility problem similar to this issue.
Could you try to paste your complete reproduced steps and debug log (with --debug
)?
from oras.
Here is the full error I was getting (sorry didn't realize it wasn't here, maybe I chose the wrong issue)!:
Error: PUT "xxxxxxxxxxxxxxx": response status code 405: unsupported: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax'
I think the underlying issue is the registry ECR is expecting the old manifest without the artifact.
from oras.
oras attach
also stopped working for the Gitlab container registry. I assumed it was a gitlab regression but it is not been resolved in a few updates of Gitlab so it might be oras or an incompatibility between the two. This might be related to what you are seeing with ECR.Error: PUT "https://reg.git.example.com/v2/ktarplee/test/telemetry/base/manifests/sha256:2ed182a012cd258381a134d4475d66460388fbeeb91333a3627d47a92ae6fc38": response status code 400: manifest blob unknown: blob unknown to registry
@ktarplee From the error logs, it doesn't reveal the root cause so it is hard to determine that this is an incompatibility problem similar to this issue.
Could you try to paste your complete reproduced steps and debug log (with
--debug
)?
@FeynmanZhou I looked into this a little more and I do think this is a unrelated issue. For instance, I just checked that if I use oras 1.0.1 the problem persists. So this is really a separate issue however at this point I do not know if the issue is with oras or with the gitlab registry implemention. When I get a few more hints at the problem I will open a new issue (unless you want me to open one now).
from oras.
So this is really a separate issue however at this point I do not know if the issue is with oras or with the gitlab registry implemention. When I get a few more hints at the problem I will open a new issue (unless you want me to open one now).
Doesn't GitLab have an allow list for the config mediaType? That would break the ability to push any artifact that isn't pre-approved by them. If they still have that policy, I'd just mark the registry as unsupported and focus on other issues.
Update: unless something has changed in the last few months, they still have an allow list for media types:
https://gitlab.com/gitlab-org/container-registry/-/merge_requests/1273
from oras.
After some offline discussions, we've aligned that chasing different RC versions of the 1.1 spec that happen to work with 1.0 registries is not the right approach. Eventually, the spec will be GA and we'll all move on with 1.0 or 1.1.
The remaining value of this issue is to consider if oras attach
should work with 1.0 registries that disallow unknown fields in a manifest. I cannot speak for all registries, but I assume that some will remain 1.0, and some will continue to refuse unknown fields.
Note, I believe this is compliant 1.0 behavior. Image spec extensibility considerations state that registries "SHOULD NOT generate an error if they encounter an unknown property in a known media type". I don't think there's specification passage that indicates a registry MUST accept unknown fields (happy to be corrected).
So the decision to remove the --image-spec
option boils down to a decision to remove support for these registries. If that is a position that the project has, then I think we can close this issue as will-not-fix.
@shizhMSFT @FeynmanZhou and other maintainers, feel free to make the call here and comment further, or close. Thanks!
from oras.
Note, I believe this is compliant 1.0 behavior. Image spec extensibility considerations state that registries "SHOULD NOT generate an error if they encounter an unknown property in a known media type". I don't think there's specification passage that indicates a registry MUST accept unknown fields (happy to be corrected).
I would treat rejecting an unknown property as generating an error. To me the spirit of the statement from OCI is to allow forward compatibility, new fields can be added without fear of every new field being a breaking change that requires an upgrade of builders, registries, and runtimes. So I don't believe this is OCI compliant.
I also feel that there are a number of non-OCI compliant registries out there, some intentionally so for good reasons. So it's useful for tooling to follow the robustness principle: "be conservative in what you send, be liberal in what you accept".
from oras.
It looks like the latest version of Gitlab fixed their regression with oras attach
. I issue we where seeing before is not longer an issue. It works again so I will assume it was a server side thing.
from oras.
It seems that there have been some recent changes on the AWS ECR side which almost make it compatible with oras attach
.
When the artifact manifest is uploaded I see the following response:
Error: PUT "https://XXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/v2...": response status code 405: unsupported: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax'
The manifest being uploaded looks something like:
{
"annotations": {
"org.opencontainers.image.created": "2024-03-08T19:12:54Z"
},
"artifactType": "application/vnd.dev.sigstore.bundle+json",
"config": {
"data": "e30=",
"digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
"mediaType": "application/vnd.oci.empty.v1+json",
"size": 2
},
"layers": [
{
"annotations": {
"org.opencontainers.image.title": "att.json"
},
"digest": "sha256:d1c3842b99b57095dccfb71e590003e2a66c701a8eff20ffd7cba1ae07e3fd3d",
"mediaType": "application/vnd.oci.image.layer.v1.tar",
"size": 5752
}
],
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"schemaVersion": 2,
"subject": {
"digest": "sha256:221189b511bd1b01a233c9af11467cddb722347a2bef2162385b34034f78d212",
"mediaType": "application/vnd.oci.image.index.v1+json",
"size": 855
}
}
However, I noticed that if I simply remove the config.data
field from the request body, ECR will accept this manifest!
It seems like this style of embedded content is optional and up to the discretion of the client.
Is it worthwhile to add a flag to disable embedded content in order to achieve compatibility with AWS ECR?
from oras.
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.
from oras.
BTW, oras attach
now works again on Gitlab (after the latest updates).
from oras.
Related Issues (20)
- `oras manifest delete` fails with `Docker-Content-Digest` header missing HOT 3
- response status code 401: unauthorized: authentication required HOT 5
- make image spec flag defaults to `v1.1` HOT 1
- display handlers should be able to output logs
- Improve formatted output of oras manifest fetch HOT 1
- Need to improve error message when basic auth flags are used with identity token flags HOT 4
- list all pushed tags in the json metadata output of `oras push` HOT 4
- unify the naming of all metadata fields lower cased
- Windows unit tests cannot be compiled
- `oras discover` shows wrong error message HOT 1
- oras build fails with go toolchain not available HOT 5
- Image pushed to ACR is not letting us create containers saying "doesnt support specified OS Windows" HOT 1
- `--format` and `--pretty` should not be used at the same time for `oras manifest fetch` HOT 1
- improve error message of when a set of flags is required to use together HOT 1
- `--pretty` should not prettify file output HOT 1
- Add new commands to support uploading and downloading images HOT 2
- Make `oras push` / `oras attach` deterministic HOT 2
- Have an option to verify the locally pulled files HOT 15
- Error trying to push a manifest to azure registry with retention policy HOT 9
- Template formatting issues for manifest content HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from oras.