Account import fails with error: 'attributedTo'

[dnoesgaard]

Account import fails with error: 'attributedTo'

[hughrun]

oof, that's a bug.

In your export there's a Tombstone comment for the book "Piranesi":

{
    "id": "https://books.theunseen.city/user/daniel/comment/50874",
    "type": "Tombstone",
    "@context": "https://www.w3.org/ns/activitystreams",
    "progress": null,
    "progress_mode": "PG"
}

Because it's a Tombstone, it can't be attributed to any user, so it doesn't have an attributedTo key or value, hence the loop fails with a KeyError.

There really needs to be some more robust exception checking in the user account import process generally but in this case we just need to skip this section and import the comment as-is.

[hughrun]

I've made some adjustments to bookwyrm/models/bookwyrm_import_job.py to handle tombstones and non-existent values better but I'm getting a different error now when I attempt importing your file in a dev system. It's fairly clear what it's saying, but I'm not clear on what would be causing it. I believe this is tripping when we add statuses. @mouse-reeve any ideas? Could it be a race condition?

celery_worker-1   | [2024-07-15 07:11:46,546: ERROR/ForkPoolWorker-3] Task bookwyrm.activitypub.base_activity.set_related_field[cda3f85e-9619-45b3-a2ec-919fac4fde04] raised unexpected: IntegrityError('duplicate key value violates unique constraint "bookwyrm_user_username_key"\nDETAIL:  Key (username)=([email protected]) already exists.\n')
celery_worker-1   | Traceback (most recent call last):
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
celery_worker-1   |     return self.cursor.execute(sql, params)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "bookwyrm_user_username_key"
celery_worker-1   | DETAIL:  Key (username)=([email protected]) already exists.
celery_worker-1   |
celery_worker-1   |
celery_worker-1   | The above exception was the direct cause of the following exception:
celery_worker-1   |
celery_worker-1   | Traceback (most recent call last):
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
celery_worker-1   |     R = retval = fun(*args, **kwargs)
celery_worker-1   |                  ^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
celery_worker-1   |     return self.run(*args, **kwargs)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/contextlib.py", line 81, in inner
celery_worker-1   |     return func(*args, **kwds)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/activitypub/base_activity.py", line 288, in set_related_field
celery_worker-1   |     item = activity.to_model(model=model)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/activitypub/base_activity.py", line 160, in to_model
celery_worker-1   |     changed = field.set_field_from_activity(
celery_worker-1   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/models/fields.py", line 88, in set_field_from_activity
celery_worker-1   |     formatted = self.field_from_activity(
celery_worker-1   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/models/fields.py", line 174, in field_from_activity
celery_worker-1   |     return activitypub.resolve_remote_id(
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/activitypub/base_activity.py", line 398, in resolve_remote_id
celery_worker-1   |     return item.to_model(model=model, instance=result, save=save)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/activitypub/base_activity.py", line 174, in to_model
celery_worker-1   |     changed = field.set_field_from_activity(
celery_worker-1   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/app/bookwyrm/models/fields.py", line 466, in set_field_from_activity
celery_worker-1   |     getattr(instance, self.name).save(*formatted, save=save)
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/files.py", line 99, in save
celery_worker-1   |     self.instance.save()
celery_worker-1   |   File "/app/bookwyrm/models/user.py", line 365, in save
celery_worker-1   |     super().save(*args, update_fields=update_fields, **kwargs)
celery_worker-1   |   File "/app/bookwyrm/models/activitypub_mixin.py", line 215, in save
celery_worker-1   |     super().save(*args, **kwargs)
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/base_user.py", line 76, in save
celery_worker-1   |     super().save(*args, **kwargs)
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save
celery_worker-1   |     self.save_base(
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/model_utils/tracker.py", line 378, in inner
celery_worker-1   |     return original(instance, *args, **kwargs)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 877, in save_base
celery_worker-1   |     updated = self._save_table(
celery_worker-1   |               ^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1020, in _save_table
celery_worker-1   |     results = self._do_insert(
celery_worker-1   |               ^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1061, in _do_insert
celery_worker-1   |     return manager._insert(
celery_worker-1   |            ^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
celery_worker-1   |     return getattr(self.get_queryset(), name)(*args, **kwargs)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1805, in _insert
celery_worker-1   |     return query.get_compiler(using=using).execute_sql(returning_fields)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
celery_worker-1   |     cursor.execute(sql, params)
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
celery_worker-1   |     return super().execute(sql, params)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
celery_worker-1   |     return self._execute_with_wrappers(
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
celery_worker-1   |     return executor(sql, params, many, context)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
celery_worker-1   |     with self.db.wrap_database_errors:
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
celery_worker-1   |     raise dj_exc_value.with_traceback(traceback) from exc_value
celery_worker-1   |   File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
celery_worker-1   |     return self.cursor.execute(sql, params)
celery_worker-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery_worker-1   | django.db.utils.IntegrityError: duplicate key value violates unique constraint "bookwyrm_user_username_key"
celery_worker-1   | DETAIL:  Key (username)=([email protected]) already exists.