Comments (6)
@quba42 No fix from me - however, I will observe that there are sync-paths that "ignore conflicts" - two threads try to create-or-update the same Thing at the same time, both try to create(), one wins, one gets the UQ-violation and switches to update instead. Postgres logs the error/collision - but it's harmless.
Unless the task fails, or the resulting repo is missing content, this is not an error.
from pulp_deb.
From: knzivid (knzivid)
Date: 2021-05-28T16:46:12Z
Interestingly
When I run pulp/pulp
official image with these package versions, I do not see this error from postgres.
pulp-ansible 0.7.3
pulp-certguard 1.3.0
pulp-container 2.6.0
pulp-deb 2.12.0
pulp-file 1.7.0
pulp-maven 0.2.0
pulp-python 3.2.0
pulp-rpm 3.12.0
pulpcore 3.12.2
When I run the same in https://pulpproject.org/related-tooling/#docker-pulp-pulp-docker-images with these versions, I see this error in the log.
pulp-deb 2.12.0
pulp-file 1.7.0
pulpcore 3.12.2
Then I looked at the logs in /var/lib/pgsql/data/log
in the pulp/pulp
docker image and I found similar logs!
2021-05-28 16:16:52.436 UTC [696] ERROR: duplicate key value violates unique constraint "core_artifact_sha256_key"
2021-05-28 16:16:52.436 UTC [696] DETAIL: Key (sha256)=(addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc) already exists.
2021-05-28 16:16:52.436 UTC [696] STATEMENT: INSERT INTO "core_artifact" ("pulp_id", "pulp_created", "pulp_last_updated", "file", "size", "md5", "sha1", "sha224", "sha256", "sha384", "sha512") VALUES ('d437ca8a-0d13-4c7a-ba23-911316364809'::uuid, '2021-05-28T16:16:52.435103+00:00'::timestamptz, '2021-05-28T16:16:52.435116+00:00'::timestamptz, 'artifact/ad/dfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', 4645, NULL, NULL, '3c9ad31f31074b93fb2ca51d8618f851ff9edd67ae509da53eb9a6e2', 'addfd22c48ca7f6e67197a63d6ba78dc706678e4b198c901de1bfd99060629cc', '682032a010aacda85558a346cff70840ff3f0882942b6f22ae5651faca9171f4751574341c285fbc9c4201e693f92c0c', 'b504ff76b0ab65b4e894949a9ce65ced4dc820e4cca961c5926744c453fca0d24ec82bc1e2134c765d2916a9f0d173380d5d7d67cb64da4c5bc22f55b2940dd9'), ('18b3ee57-eb6b-404c-8e81-a7763c165c29'::uuid, '2021-05-28T16:16:52.435669+00:00'::timestamptz, '2021-05-28T16:16:52.435678+00:00'::timestamptz, 'artifact/88/89e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 5527, NULL, NULL, 'd58236d07ed20cf31c96eba8148b733e891967f71b6c418759c3b7b2', '8889e025a6e18c2a0e31edd2f430ee2a7be8a88c48f156706ec672cc87cb832d', 'f026b266558924947a1bc796296f9ee6b15033df275d05a576a09505bb66a3870bc61ce166c405887864ae940f08a9a4', 'b8eb88c5b37e13e53fa549831758dbb7564197323f7fd3d2a19b3ba8a7a9f10718939cb8d94db47987b7e461b4f29012ee2479a7c2bc3d3a73bd1fb8746e338c'), ('6fb3b30f-7d58-43db-8a48-2482e160c7b3'::uuid, '2021-05-28T16:16:52.435871+00:00'::timestamptz, '2021-05-28T16:16:52.435880+00:00'::timestamptz, 'artifact/b7/4740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', 833, NULL, NULL, '3d42a1ff9d8d9caaa5a9526be54d786af61a3d16d00f45f0a740639a', 'b74740e506eec14a271e74ee7096b0ce6194a4810f568727da9557127f8db4b9', '2d3b9145ff8fef94ea470404c3b2327a76347b40e1b23ee301d16146bbf139d6bef83e8a04b49af5d65d4f6a6c73be72', '9008fd1c21bc112726f9687536e2dd8c2a5c5d3133338eff8605b4268bf0dc5fb7a974ea4509c5040e9cf8e2676ff7da29a5da100187f9d0fd8f565626594036') RETURNING "core_artifact"."pulp_id"
Is this harmless?
from pulp_deb.
From: @quba42 (quba42)
Date: 2021-06-02T12:00:39Z
Not sure if this is harmless. I will try to reproduce this when I get to it.
If it is possible to install packages from the synced repository in question, it may well be harmless...
from pulp_deb.
I cannot synchronize this repository at all, since some of the packages cannot be downloaded. This happens through browser as well as through Pulp, so it does not appear to be a Pulp issue. As a result, I am closing this ticket "cannot reproduce". I am happy to re-open if there is further user input on how to reproduce this.
The error as encountered by my Pulp instance:
pulpcore-worker[13000]: pulp [1ec10a9ffbe14b569d7edb730bca4902]: pulpcore.tasking.pulpcore_worker:INFO: Task cacb278c-041e-468c-9816-3c6c52cca121 failed (503, message='first byte timeout', url=URL('https://apt.llvm.org/bionic/pool/main/l/llvm-toolchain-11/libclang-cpp11_11.1.0~++20211011094159+1fdec59bffc1-1~exp1~20211011214614.8_amd64.deb'))
pulpcore-worker[13000]: pulp [1ec10a9ffbe14b569d7edb730bca4902]: pulpcore.tasking.pulpcore_worker:INFO: File "/home/vagrant/devel/pulpcore/pulpcore/tasking/pulpcore_worker.py", line 362, in _perform_task
pulpcore-worker[13000]: result = func(*args, **kwargs)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulp_deb/pulp_deb/app/tasks/synchronizing.py", line 141, in synchronize
pulpcore-worker[13000]: DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 161, in create
pulpcore-worker[13000]: loop.run_until_complete(pipeline)
pulpcore-worker[13000]: File "/usr/lib64/python3.10/asyncio/base_events.py", line 641, in run_until_complete
pulpcore-worker[13000]: return future.result()
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
pulpcore-worker[13000]: await asyncio.gather(*futures)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
pulpcore-worker[13000]: await self.run()
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/artifact_stages.py", line 183, in run
pulpcore-worker[13000]: pb.done += task.result() # download_count
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/artifact_stages.py", line 209, in _handle_content_unit
pulpcore-worker[13000]: await asyncio.gather(*downloaders_for_content)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/models.py", line 89, in download
pulpcore-worker[13000]: download_result = await downloader.run(extra_data=self.extra_data)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/download/http.py", line 253, in run
pulpcore-worker[13000]: return await download_wrapper()
pulpcore-worker[13000]: File "/usr/local/lib/pulp/lib64/python3.10/site-packages/backoff/_async.py", line 133, in retry
pulpcore-worker[13000]: ret = await target(*args, **kwargs)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/download/http.py", line 249, in download_wrapper
pulpcore-worker[13000]: return await self._run(extra_data=extra_data)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/download/http.py", line 274, in _run
pulpcore-worker[13000]: self.raise_for_status(response)
pulpcore-worker[13000]: File "/home/vagrant/devel/pulpcore/pulpcore/download/http.py", line 185, in raise_for_status
pulpcore-worker[13000]: response.raise_for_status()
pulpcore-worker[13000]: File "/usr/local/lib/pulp/lib64/python3.10/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status
pulpcore-worker[13000]: raise ClientResponseError(
from pulp_deb.
@ggainey might have a PR for this pulp/pulpcore#2103
from pulp_deb.
Thanks @ipanova. If it is being fixed within pulpcore, then I guess it is nevertheless fine to keep this pulp_deb issue closed (since it is not a pulp_deb but a pulpcore issue).
from pulp_deb.
Related Issues (20)
- Filters for exploring Content relationships HOT 1
- duplicate key value violates unique constraint "core_publishedmetadata_publication_id_relative__60dd832b_uniq" HOT 12
- Requesting pulp_deb bindings release 2.19.z with updated faraday dependencies HOT 6
- Allow users to set 'origin' and 'label' in release files HOT 1
- Improve the signing service documentation
- Autopublish for Debian plugin like the RPM one does HOT 4
- Distribution Packages and Packages.gz contain incorrect data (Pulp signing-service GPG key) HOT 15
- Missing packages from Packages file HOT 1
- Proxy sync fails when newer binding is used to sync content on older pulp_deb api HOT 2
- Faster async metadata signing HOT 1
- Refactor: `package` and `relative_path` fields on `Content`
- pulp orphan cleanup failure (protected foreign keys) HOT 17
- Cannot sync repo with blank custom field HOT 1
- Ability to use Package filenames when publishing HOT 3
- PackageReleaseComponents are not being properly handled in the case of duplicates HOT 2
- Ability to specify Origin when publishing repository HOT 1
- Add publication options for packages with Phased-Update-Percentage HOT 4
- Please add `names__in` for deb HOT 2
- As a REST api user I would like to know which distributions are included in a publication
- 'NoneType' object has no attribute 'artifact_set_sha256' when syncing a recently updated deb remote HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pulp_deb.