papermerge/papermerge-core

In this repository is the source code of Papermerge DMS backend core, REST API server, and frontend UI

papermerge-core's Issues

What is considered a Release Candidate when using docker-compose?

Following the documentation for setting up Papermerge via docker-compose I am confused as to what version of the software is considered to be a stable release candidate.

The link to the GitHub Packages list shows all the versions are labeled as alpha or "a". Coupled with the warning "Currently docker tag latest points to latest 2.1.0-alphaXYZ version which is not yet production ready." leaves me confused as to how I should interpret the stable releases when none of them have release candidate or "rc" associated with them.

The rest of the documentation is easy to follow but I would like some suggestions as to what should be consider as stable for the fresh install and moving forward.

PDF file Upload

Hi,

PDF file with a size of 100 MB are not uploaded for me. Is this a known problem?

with kind regards
Chris

Add frontend localization

In other words - UI should be available in different languages (i18n).
One possible candidate for UI localization is https://react.i18next.com/

Add support for jpeg and png to 3.x

Gentle reminder for me to add support for jpeg and png images back as it is highly requested feature.

Remark:
Jpeg and png image are now supported only for papermerge <= 2.0.x, though documentation does not mention that detail, which leads to confusion.

Cannot Login Into New Installation

Hi I have difficulties to login to new installation with the below error:

Do you have any idea to solve it? Thank you very much.

Below are the logs:

• _papermerge-db-1_logs.txt
• _papermerge-redis-1_logs.txt
• _papermerge-es-1_logs.txt
• _papermerge-frontend-1_logs.txt
• _papermerge-ws_server-1_logs.txt
• _papermerge-backend-1_logs.txt
• _papermerge-worker-1_logs.txt
• _papermerge-traefik-1_logs.txt

Add multiple OCR languages

Hi, it's possible to have an official worktrough to add multiple ocr languages in papermerge?
Many thank's,
Fabio.
labels: docker, documentaion (sorry i can't add labels by myself)

should we manually give all access to the django superuser because i(superuser) am getting all the roles like give permission to the user and managing access of the document or the folder?

IMPORTER_DIR misspelled

papermerge-core/papermerge/core/management/commands/worker.py

Line 80 in a6ad68f

reason_msg += "1. IMPORDER_DIR not configured\n"

Thanks for papermerge, but why its so hard to install??

Hi,

First of all, thanks for this amazing product!! And the Documentation is also great.
I'm running Papermerge for a few month with the Image from "LinuxServer.io" release group. It was really easy to install and runs perfect. But the last working release has a few annoying bugs and the Image is not updated. (Its deprecated)

So I came here for the official image.
But I have a very hard time to set this up.
I tried via Docker Compose and with Portainer and repo.

The main Issue is, I always get the "404 not found" Error when accessing the WebUI (same as this Issue).
Its think because of that "HOSTNAME" or "USE_HOSTNAME" variable.
When I use the Domain Name I get this 404 Error. When Using the IP-Adress of the Host, I get the Login UI, but I also get this Syntax Error....

I have given up..

I don't know if I'm just stupid, but I also run over 20 other Docker Containers (not all from LinuxServer) with great success.
I'm not that deep into "creating Containers" but why is the Image from LinuxServer so simple and easy to use and also possible with one Image, and this official image gives me a headache.

I will stay a little longer on the LinuxServer image, and hope for a fix or anything. I'm not sure.

Best regards

Simon

unable to perform ocr

GoBD compliance

The goal of this issue is to provide a manual and working copy of papermerge to be GoBD compliant.

The basic installation of papermerge will already meet alot of the requirements of the german GoBD

The Bitkom e.V. published a guide e.g. checklist to check if a software and process can be GoBD compliant.

In my humble opinion there is currently on major blocker and this is 2.2.1 c) requirement 17 where it reads:

Vollständigkeit von Daten und Dokumenten über die gesamte Aufbewahrungsfrist

• Keine Löschmöglichkeit vor dem Ende der Aufbewahrungsfrist.
• Keine automatisierte Löschung von Daten und Dokumenten nach Ende der Aufbewahrungsfrist (z. B. stets alle Daten oder Dokumente löschen, die älter als X Jahre sind).
• Für die Löschung ist zwingend eine organisatorische Freigabe einzuholen, um dem Umstand gerecht zu werden, dass entsprechend § 147 Abs. 3 S. 3 AO die Aufbewahrungsfrist nicht abläuft, soweit und solange die Unterlagen steuerlich von Bedeutung sind und deren Festsetzungsfrist noch nicht abgelaufen ist (Ablaufhemmung).
• Hinweis: Außersteuerliche Regelungen können eine längere Aufbewahrungsfrist erfordern.

This delete lock (Keine Löschmöglichkeit vor dem Ende der Aufbewahrungsfrist) includes priviliged access like root / admin. One way to implement such mechanism is MinIO Retention, it turnes your S3 bucket into a WORM (Write Once Read Many) storage backend. And the Cohasset Associates, Inc. did already an assesment on MinIO to deploy such an S3 Storage to be SEC 17a-4(f), FINRA 4511(c) and CFTC 1.31(c)-(d) compliance. We can consider GoBD and SEC 17a-4(f), FINRA 4511(c) and CFTC 1.31(c)-(d) comparable as both deal with storing of tax data on digital devices.

From version 2.1 ongoing papermerge is moving towards a kubernetes ready architecture and RWX file storage to store data.
The RWX storage is also used to share the data between the app and the worker nodes. But this development moved papermerge a bit further away from the S3 backend.

A further goal of this issue is to adapt papermerge in order to use an S3 WORM storage backend (storing only relevant data, this includes any intermediate steps in the processing of the original data to the processed data, but nothing more - as this data is stored for at least 10 years on that WORM drive).

To archive this goal, we need to adapt not only the core, but also other parts of the papermerge project. We should link all adaptions in order to be GoBD compliant to this issue, so that we can track the development.

ARM support

First of all, thank you very much! Awesome work and the project is amazing.
I was wondering if there will be an ARM support?

I was trying to build it for ARM and got issues with the xapianpy package.
Working on rebuilding that.
https://github.com/roifgroup/xapian-bindings
Package output:
https://github.com/roifgroup/xapian-bindings/actions/runs/7358648229

Was wondering if there is a point in going further with this of not to bother with ARM support?

Update to Tesseract 5?

Hello. Has anyone tried this with tesseract 5? I'm getting terrible OCR accuracy.

BTW,,, thank you for your work on this! You have done a great job and I hope it goes places for you.

Exception when opening the Inbox

In my fresh papermerge 3 installation I added a document to the inbox and it looked fine. Later (I suspect after OCR had finished, but unsure about that) when I opened the inbox again I got this in the browser:

And docker logs says:

79.220.xxx.xxx - - [22/Nov/2023:06:54:29 +0000] "GET /api/nodes/2a20e31f-a79b-430e-968a-51bb689cec40?page_number=1&page_size=15&order_by=-title HTTP/1.1" 500 21 "http://49.13.xxx.xxx:12000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0"
Exception in ASGI application
Traceback (most recent call last):
  File "/core_app/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/core_app/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 289, in __call__
    await super().__call__(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/core_app/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/core_app/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/core_app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/core_app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 291, in app
    content = await serialize_response(
  File "/core_app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 154, in serialize_response
    raise ResponseValidationError(
fastapi.exceptions.ResponseValidationError

Docker container start fail: "exec /run.bash: exec format error"

I followed instructions from https://docs.papermerge.io/Installation/docker.html to setup Papermerge on

• RPi4
• Ubuntu Server 20.04.4 LTS (GNU/Linux 5.4.0-1065-raspi aarch64)
• Docker 20.10.17
• Docker Compose v2.6.0

using the docker-compose.yml from the documentation.

When starting with docker compose -f docker-compose.yml --env-file .env up the frontend, worker, ws and backend containers fail with a

exec /run.bash: exec format error

A docker logs papermerge-frontend-1 delivers

exec /usr/sbin/nginx: exec format error,

all other failed containers

exec /run.bash: exec format error.

The other containers (like traefik, pg, ...) are starting up fine. When trying to load Papermerge in the browser I get a 404 page not found, presumably deliverd by traefik which can't find the papermerge containers.

Anybody knows this issue? Any hints? Thanks.

I want to use inbox feature by using imap client what documentation should i follow please guide

Python 3.9 Support

Are there reasons to stay at python 3.8? would be great if you could raise the max version to 3.9. when I saw correctly, configula and OCRmyPDF_papermerge are also affected. The main reason for this would be the use of piwheels for simplified ARM builds based on Python 3.9 wheels for Bullseye.

New feature: automatically assign a document serial number after upload

Hello,

It would be great having a serial number/code, automatically added, when uploading a new document.
It could be generated from user defined schemas: with the year, department, type of document..
This new serial number could be confirmed or edited immediately after the upload choosing from a drop down list of available schemas.

For instance, quotation:

2023/q12/c123 (year/quotation number/customer number)

or

2023-23635 (year-recursive registration document number)

The schema should be user defined because every business has their own different organization in terms of docs management. Then, this document "identifier"could be used for searching as search key, partial or typed in fully.

In business environment, this would be a required feature because it is common in every-day docs administration.

Folders and tags are very good for to archive documents and keep them in order because organized in common sense locations but I still feel the serial number a requirement.

Without this feature I feel Papermerge difficult to use in business environment even thought I really like it because it is clean, easy to use and good looking.

I am available to share ideas about this feature request and further explain use cases for business.

Thank you so much and great job!

F.

[BUG] Tags Editor: Selection of saved tags

Scenario:

1. Create tag in Tags submenu (I will assume name test_tag)
2. Select folder/document
3. 2. Right click -> Tags
4. Insert test_tag text and input separator (for the tag to be added to the tag line)
5. Remove this tag by pressing X button on that tag
6. Insert test_tag again

Expected result:
6. test_tag is added as a tag to Tags Editor line

Current result:
6. just after usage of tag separator, test_tag text dissipaters from Tags Editor

Additional information:
Issue occurs only to pre-existing tags. Meaning that with adding and removing same new tag won't reproduce the problem.
Version: Papermerge 2.0

A 50-page pdf file is very slow to open

A 50-page pdf file is very slow to open, and it takes 10 minutes to open it all. What is the reason?

Security issue - IDOR

Hi! I've noticed that the document-versions/uuid:pk/download/ API is vulnerable to IDOR vulnerability allowing any user to download any file.

I've tried to download a file uploaded by the admin account (and only visible to the admin account) using a low-privileged user, and the file was successfully downloaded

Admin view:

User view:

Downloading admin file using direct url from the user account:

Can you verify the issue, please?
Thanks!

mysql broken, docker broken?

Hello, swapping to papermerge from linuxserver/papermerge docker and I have a few problems.

First of all, docker image is 2.1.5 and i see latest release is 2.1.9
Second problem is after defining environment variables for a mysql server, after I run the image i look in the logs and have a python error, something about missing mysqlclient module missing

"Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 15, in
import MySQLdb as Database
ModuleNotFoundError: No module named 'MySQLdb'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/manage.py", line 22, in
main()
File "/app/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/venv/lib/python3.10/site-packages/django/core/management/init.py", line 446, in execute_from_command_line
utility.execute()
File "/venv/lib/python3.10/site-packages/django/core/management/init.py", line 420, in execute
django.setup()
File "/venv/lib/python3.10/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/venv/lib/python3.10/site-packages/django/apps/registry.py", line 116, in populate
app_config.import_models()
File "/venv/lib/python3.10/site-packages/django/apps/config.py", line 304, in import_models
self.models_module = import_module(models_module_name)
File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/venv/lib/python3.10/site-packages/rest_framework/authtoken/models.py", line 9, in
class Token(models.Model):
File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 141, in new
new_class.add_to_class("_meta", Options(meta, app_label))
File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 369, in add_to_class
value.contribute_to_class(cls, name)
File "/venv/lib/python3.10/site-packages/django/db/models/options.py", line 235, in contribute_to_class
self.db_table, connection.ops.max_name_length()
File "/venv/lib/python3.10/site-packages/django/utils/connection.py", line 15, in getattr
return getattr(self._connections[self._alias], item)
File "/venv/lib/python3.10/site-packages/django/utils/connection.py", line 62, in getitem
conn = self.create_connection(alias)
File "/venv/lib/python3.10/site-packages/django/db/utils.py", line 208, in create_connection
backend = load_backend(db["ENGINE"])
File "/venv/lib/python3.10/site-packages/django/db/utils.py", line 113, in load_backend
return import_module("%s.base" % backend_name)
File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 17, in
raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?
Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 15, in
import MySQLdb as Database
ModuleNotFoundError: No module named 'MySQLdb'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/manage.py", line 22, in
main()
File "/app/manage.py", line 18, in main
execute_from_command_line(sys.argv)"

endlessly repeats same thing

BTW, same error i also get with ghcr 2.1.9

[Suggestion] Logout confirmation

Dear Developers,

It would be nice to logout without the need of confirming it.
If you like that function perhaps it would be good decision to add option in Preferences to allow single click logout.

Thank you for your time,
Best Regards

No OCR

I just deployed my first papermerge instance in a docker container as proposed

docker run -p 8030:8000 -e PAPERMERGE__MAIN__SECRET_KEY=abc -e DJANGO_SUPERUSER_PASSWORD=123 papermerge/papermerge:latest

I can log in and add some PDF, but no matter what I do, no OCR is happening. Not automatically, neither when I start the OCR manually. The OCR status indicator always stays grey.

What do I miss?

Fix compose file

I had to edit the compose file to fix an issue of Traefik not allowing me to access the webpage from another machine.
I would get a 404 Page Not Found error uintill i did the fix below.

1. Replace all instances of papermerge.local with ${HOSTNAME} in docker-compose.yml
2. add HOSTNAME=<hostname of PC> to .env

PM 2.1.x not able to mount DB & Redis outsie docker path

Hi,

I use docker with the latest PM stack (in general the git stack).

One challenge; I like to separate the data part form the appl.-part.
Therefore I (successfully) mounted the media & Index files and dir to own, permanent docker volumes (bind mounts to NFS-shares). All fine.

But trying to do the same with the DB and/or Redis volume I get the following error within deployment of the Backend services:

Created Error response from daemon:
failed to copy file info for /var/lib/docker/volumes/postgres_data/_data:
failed to chown /var/lib/docker/volumes/postgres_data/_data: lchown
/var/lib/docker/volumes/postgres_data/_data: operation not permitted.

for redis and postgres I can only use the docker-generated volumes in:
/var/lib/docker/volumes/ without further bindings to external FS.

owner within docker path is Mr. coredump:

`...root@PaperMerge:~# ls -la /var/lib/docker/volumes/pm21*
/var/lib/docker/volumes/pm21_media_root:
total 18
drwx-----x 3 root root 4 Jan 7 12:05 .
drwx-----x 7 root root 8 Jan 7 12:05 ..
drwxrwxrwx 5 nobody nogroup 12 Jan 7 15:02 _data
-rw------- 1 root root 96 Jan 7 12:05 opts.json

/var/lib/docker/volumes/pm21_postgres_data:
total 18
drwx-----x 3 root root 3 Jan 7 12:05 .
drwx-----x 7 root root 8 Jan 7 12:05 ..
drwx------ 19 systemd-coredump systemd-coredump 26 Jan 7 12:06 _data

/var/lib/docker/volumes/pm21_redis_data:
total 10
drwx-----x 3 root root 3 Jan 7 12:05 .
drwx-----x 7 root root 8 Jan 7 12:05 ..
drwxr-xr-x 2 systemd-coredump systemd-coredump 3 Jan 7 16:03 _data

/var/lib/docker/volumes/pm21_xapian_index:
total 18
drwx-----x 3 root root 4 Jan 7 12:05 .
drwx-----x 7 root root 8 Jan 7 12:05 ..
drwxrwxrwx 5 nobody nogroup 12 Jan 7 15:02 _data
-rw------- 1 root root 96 Jan 7 12:05 opts.json`

any ideas to get get db & redis deployed on a bind mount?
Can I map Dr. coredump to GID 1000 or similar?

Papermerge behind Traefik v2 transparent proxy

Hello,

I run all my containers behind a transparent proxy to provide https and dns (eg container.mysubdomain.mydomain.tld)

I modified the docker-compose.yml file for the elastic search version.

The app starts up and works as I think it should, but here and there errors start to occur, like creating a API key didn't show the key the first time, deleting a key gets the front end stuck and leaves me having to delete cookies to re log back in.

I have attached a copy of my docker-compose file for reference.

Any assistance greatly appreciated.

papermerge_docker-compose.yml.txt

Allow password authentication for redis

Hi,

thank you for this great project. This effort and work is underestimated in any time. 👍
maybe I did overlook this setting in the documentation and the code base. But is there a way to use at leas password authentication for the redis server? I think it is crucial to implement it, for security reasons.

Regards
Daniel

Unknown command: 'worker'.

After installing according to the Contributor's Manual document, the following problems occur

1. run the ./manage.py worker step,
   The following error message appears。
   Unknown command: 'worker'.

2. The backend and frontend starts normally, but the front-end page cannot be accessed. The chrome console prompts an error as shown below。

Backend startup screenshot

Frontend startup screenshot

Please help to see, thanks.

Which was the last version of papaermerge-core which supported folder sharing among different users?

What is the easiest way of creating a folder and giving access to a group of users by modifying present code

Not an Issue but a general question

Dear @ciur and community

On my journey to better understand containers I am trying to deploy the papermerge-core image on a serverless service, namly Cloud Run on GCP. Also this would be the most cost efficient solution for me personally.

My understanding is that it is technically possible to run papermerge on Cloud Run when using their second generation execution envirionment. I am able to run the papermerge-core image on Cloud Run without issues.

However, to make this work for good we need persistent storage and a database. I am now stuck at the storage part when trying to integrate gcsfuse. I did the following.

0. Clone the papermerge repo
1. Adjust the Docker file so that gcsfuse is installed and a folder to mount the storage to is created.

FROM papermerge/base:1.1.1 as build

### STEP 1 - pull all python dependencies in virtual env
ENV IN_DOCKER=1
ENV POETRY_VIRTUALENVS_CREATE=false
ENV UWSGI_PROFILE=gevent
ENV VIRTUAL_ENV=/venv

RUN apt-get install -y --no-install-recommends \
    build-essential \
    python3-dev \
    tesseract-ocr \
    tesseract-ocr-deu \
    imagemagick \
    gcc

RUN pip install --upgrade poetry
RUN python -m venv /venv

ENV PATH="/venv/bin:$PATH"

COPY poetry.lock pyproject.toml /
RUN poetry install --no-root --no-dev -vvv

## STEP 2 - use slim base image
FROM python:3.10-slim

ENV PATH="/venv/bin:$PATH"
ENV PYTHONBUFFERED=1
ENV VIRTUAL_ENV=/venv

RUN apt-get update;
RUN apt-get install -y --no-install-recommends \
    build-essential \
    python3-dev \
    postgresql-client \
    tesseract-ocr \
    tesseract-ocr-deu \
    imagemagick \
    poppler-utils \
    git \
    libmagic1 \
    ghostscript \
    file \
    gcc \ 
    wget

# install gcsfuse
RUN wget "https://github.com/GoogleCloudPlatform/gcsfuse/releases/download/v0.42.4/gcsfuse_0.42.4_amd64.deb" && \
    chmod +x gcsfuse_0.42.4_amd64.deb && \
    apt install ./gcsfuse_0.42.4_amd64.deb -y

# create dir for gcsfuse mount
RUN mkdir -p /gcs

COPY docker/prod/uwsgi.ini /etc/uwsgi/papermerge.ini
COPY docker/prod/scripts /
RUN chmod +x /run.bash

COPY --from=build /venv /venv

WORKDIR app

# sources
COPY papermerge/ ./papermerge/
COPY docker/prod/config/ ./config/
COPY docker/prod/manage.py ./

EXPOSE 8000

ENTRYPOINT ["/run.bash"]
CMD ["server"]

2. Adjust the run.bash startup script so that gcsfuse is executed at init so that storage is mounted

#!/bin/bash

export PATH="/venv/bin:${PATH}"

CMD="$1"
PYTHON="/venv/bin/python"
MANAGE="${PYTHON} manage.py"
BUCKET="pm-6cf6b091c755272a74369f8a2a90b796"
MNT_DIR="/gcs"


if [ -z "${DJANGO_SETTINGS_MODULE}" ]; then
  # default value for DJANGO_SETTINGS_MODULE environment variable
  export DJANGO_SETTINGS_MODULE=config.settings
fi

if [ -z "${DJANGO_SUPERUSER_USERNAME}" ]; then
  # default value for DJANGO_SUPERUSER_USERNAME environment variable
  export DJANGO_SUPERUSER_USERNAME=admin
fi

if [ -z "${DJANGO_SUPERUSER_EMAIL}" ]; then
  # default value for DJANGO_SUPERUSER_EMAIL environment variable
  export [email protected]
fi

if [ -z $CMD ]; then
  echo "No command specified"
  exit 1
fi

exec_server() {
  exec uwsgi --ini /etc/uwsgi/papermerge.ini
}

exec_ws_server() {
  exec daphne -b 0.0.0.0 --port 8000 config.asgi:application
}

exec_collectstatic() {
  $MANAGE collectstatic --noinput
}

exec_migrate() {
  # run migrations
  $MANAGE migrate --no-input
}

exec_update_index() {
  # Create/Update search index
  $MANAGE update_index &
}

exec_createsuperuser() {
  # user envrironment variables:
  #   (1) DJANGO_SUPERUSER_USERNAME
  #   (2) DJANGO_SUPERUSER_EMAIL
  #   (3) DJANGO_SUPERUSER_PASSWORD
  # to create superuser if (1) and (2) are set
  if [ -n "${DJANGO_SUPERUSER_USERNAME}" ] && [ -n "${DJANGO_SUPERUSER_EMAIL}" ]; then
    echo "Creating superuser username=${DJANGO_SUPERUSER_USERNAME}"
    $MANAGE createsuperuser --noinput \
      --username ${DJANGO_SUPERUSER_USERNAME} \
      --email ${DJANGO_SUPERUSER_EMAIL} || true
  fi
}

exec_worker() {
  exec celery --app config worker \
   -n "worker-node-${HOSTNAME}@papermerge" ${PAPERMERGE__WORKER__ARGS}
}

exec_gcsfuse() {
  echo "Mounting GCS Fuse."
  gcsfuse --debug_gcs --debug_fuse $BUCKET $MNT_DIR
  echo "Mounting completed."
}

exec_init() {
  exec_collectstatic
  exec_migrate
  exec_createsuperuser
  exec_update_index
  exec_gcsfuse
}

case $CMD in
  init)
    exec_init
    ;;
  migrate)
    exec_migrate
    ;;
  collectstatic)
    exec_collectstatic
    ;;
  createsuperuser)
    exec_createsuperuser
    ;;
  server)
    # starts REST API webserver
    exec_init
    exec_server
    ;;
  ws_server)
    # start websockets server
    exec_init
    exec_ws_server
    ;;
  worker)
    exec_worker
    ;;
  *)
    $MANAGE $@
    ;;
esac

The mount of the storage seems to work fine, but when building this image and deploying it, I am not able to access the app. When comparing the logs I always receive a SIGTERM event before the uWSGI worker comes up on port 7000 as in the working logs using the standard image.
Snipped from working example using the ready made image.

Snipped from the log output when using my image with gcsfuse. Please find a log export here

I would highly appreciate any feedback which points me in the right direction so that I can resolve this.

Is it okay to use 2.1.3 version of ember js frontend with papermerge core 2.1.7

Mounting /app in a docker volume causes issue

Using 2.1.9

trying to mount /app outside of the docker container I get this error:

2023-09-19 16:47:56,659 INFO exited: nginx (exit status 1; not expected)
Traceback (most recent call last):
  File "/venv/bin/daphne", line 8, in <module>
2023-09-19 16:47:56,825 INFO gave up: nginx entered FATAL state, too many start retries too quickly
    sys.exit(CommandLineInterface.entrypoint())
  File "/venv/lib/python3.10/site-packages/daphne/cli.py", line 170, in entrypoint
    cls().run(sys.argv[1:])
  File "/venv/lib/python3.10/site-packages/daphne/cli.py", line 232, in run
    application = import_by_path(args.application)
  File "/venv/lib/python3.10/site-packages/daphne/utils.py", line 12, in import_by_path
    target = importlib.import_module(module_path)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'config'

this is odd, I feel it should be working like it does for other containers.
Only the supervisor.log files are created in the container mounted /app folder.

This with a linux mounted fs with root:root 777. It shouldn't be a perm issue. At least supervisor is able to write in it.

My ultimate goal is to run the container as a specific papermerge user a limit the mounted /app folder to this user. But this is not the issue here, it just wouldn't use the folder to create the db or anything it needs to. It fails first on launchong ws_server

PM 2.0 infinity loop multiplication of files pushed via import-Folder

hi,

using PM 2.0 on docker (portainer) / image ghcr.io/linuxserver/papermerge:latest
When I upload a file via UI all fine.
When I copy same file (diff. filename) to the import folder (Importer_DIR) PM starts duplicating the file.

What logs can be provided?

Thanks
Ole

Tags are not shared

Dear Developers,

Thank you for this repository personally I cannot wait for you to add double panel.

Problem:
When the folder is accessed by the user (with access rights to that folders), Tags set on the shared documents/folders, are not shared with the target user.

Example:
admin Taged folder_1 with "Receipt" Tag
admin has shared folder_1 to user_1 (Allow | Full Control)
user_1 can see the tag on the folder, but do not have access to this tag in the Tag menu (can't pin it/edit it). (No Tags from admin are visible at user context)
user_1 cannot add new Tag to the shared folder. (Tag appears for a moment but dissipaters after refresh).

Please let me know how I can support you to be able to solve that issue quickly.

Thank you in advance.

Cannot sign into new installation

I have installed the Portainer setup method, with docker.

The issue I am currently having though is this:

I cannot get any further. It locks me out for some reason. Here are the logs:
_papermerge_redis_1_logs.txt
_papermerge_db_1_logs (1).txt
_papermerge_es_1_logs (1).txt
_papermerge_frontend_1_logs.txt
_papermerge_ws_server_1_logs.txt
_papermerge_backend_1_logs.txt
_papermerge_worker_1_logs.txt
_papermerge_backend_init_1_logs (1).txt
_portainer_logs.txt
_papermerge_traefik_1_logs.txt

Please help. I don't know what to do at this point.

I have created two users and kept them in same group. Each user has created his own folder at home .Will they be able to view each others folder?

Backup command returns "django.db.utils.OperationalError: no such table: core_user " and fails

This is a fantastic project. Thank you so much,

I installed with Portainer in minutes, and started using right away.

I am running the following versions:
Backend version: 2.1.8
Frontend version: 2.1.2

I tried to perform a backup following the instructions at https://docs.papermerge.io/backup_restore.html but as soon as I run:

docker run papermerge/papermerge backup /place/backup/file/here/

it gives me the following output and fails:

/venv/lib/python3.10/site-packages/requests/__init__.py:109: RequestsDependencyWarning: urllib3 (1.26.13) or chardet(None)/charset_normalizer (3.0.1) doesn't match a supported version!
  warnings.warn(Traceback (most recent call last):
  File "/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 477, in execute
    return Database.Cursor.execute(self, query, params) 
sqlite3.OperationalError: no such table: core_user

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/manage.py", line 22, in <module> main()
  File "/app/manage.py", line 18, in main execute_from_command_line(sys.argv)
  File "/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line    utility.execute()
  File "/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/venv/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv self.execute(*args, **cmd_options)
  File "/venv/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute output = self.handle(*args, *options)
  File "/app/papermerge/core/management/commands/backup.py", line 41, in handle  backup_data(file_path=file_path)
  File "/app/papermerge/core/backup_restore/backup.py", line 227, in backup_data dict_data = dump_data_as_dict()
  File "/app/papermerge/core/backup_restore/backup.py", line 211, in dump_data_as_dict  result_dict['users'] = UserSerializer(User.objects, many=True).data
  File "/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 768, in data ret = super().data
  File "/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 253, in data self._data = elf.to_representation(self.instance)
  File "/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 686, in to_representation  return [
  File "/venv/lib/python3.10/site-packages/django/db/models/query.py", line 320, in __iter__    self._fetch_all()
  File "/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1507, in _fetch_all   self._result_cache = list(self._iterable_class(self))
  File "/venv/lib/python3.10/site-packages/django/db/models/query.py", line 57, in __iter__   results = compiler.execute_sql(
  File "/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1361, in execute_sql    cursor.execute(sql, params)
  File "/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute   return self._execute_with_wrappers(
  File "/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers   return executor(sql, params, many, context)
  File "/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute  with self.db.wrap_database_errors:
  File "/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__   raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute   return self.cursor.execute(sql, params)
  File "/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 477, in execute   return Database.Cursor.execute(self, query, params) django.db.utils.OperationalError: no such table: core_user

Thanks again for doing this.

I wanted to edit and create new database tables so how can i access the models such as core_users and core_groups?

Imap guidelines

I wanted to implement Inbox feature basically sending files through imap but didnt found any resources on papermerge.Could you provide me any docs or resources?

Upgrade from 2.0 -> 2.1 Issues

I've been trying to get my papermerge 2.0.1 installation updated to the new 2.1 release, and having lots of issues. The root of the issue seems to be that the core_user.id column in my old database is an integer, when some migrations are expecting a uuid field. This means that 9 migrations fail to apply when first launching, and from there everything just seems to cascade and I can't sign in due to more errors with the core_user table when attempting a login.

Is this an upgrade that's expected to work? Right now things seem pretty borked, but I'm happy to pull a DB backup and help gather more data if this is something that's expected to work

Unable to set group to the custom created users

I created groups and then i tried to assign them to my custom created users.After assigning the user with respected group and then saving all the changes,When i try i open that edit user page again I don't see the group getting ticked mark in the edit page.
Is this a backend bug ?

Hello Sir,

Authentication credentials were not provided

Backend and Frontend start normally, but an error occurs when accessing the front-end page chrome console, http://120.76.102.100:8000/api/users/me/ 401 (Unauthorized) error 。

Visit http://120.76.102.100:8000/api/users/me/ and return information "errors":[{"detail":"Authentication credentials were not provided.","status":"401","source": {"pointer":"/data"},"code":"not_authenticated"}]}

Docker Compose deployment and operation also have this problem, please help, thank you!

Unable to log in

Followed setup instructions given at https://docs.papermerge.io/Setup/docker-compose.html
Works out fine till login, says: "SyntaxError: Unexpected token '<', " <!doctype "... is not valid JSON"

What can I do? Google ist not really helpful since I am a noob :(

Can not login anymore

I juste restarted my docker container. Now I can not login in anymore.

Beneath the login screen I get an error message.

SyntaxError: Unexpected token '<', "<html> <h"... is not valid JSON

(BTW: I needed do delete the cookies in my browser before that, otherwise the login screen just stayed an empty grey background without the login pop-up)

Importer does not import files in directories.

Hey @ciur, thank you for your awesome project.
I found that if I am using Papermerge's importer for local files, I only can import files and not folder within files.

As a better example, I do have a folder structure like this

Bank/Sparkasse/Kontoauszug/Kontoauszug.pdf

It would be awesome if Papermerge is able to import this structure into Papermerge's internal structure.
What do you think about this feature? I am not a fully fledged python dev, but maybe with some assistance, I can try to add this to the local importer function.

Thanks!

i just want to know whether starting this django application automatically runs worker or should we manually run runserver and then run worker

Drag 'n' Drop not working

I'm trying out papermerge for the first time and set up a test installation with docker compose on a hetzner machine. Drag 'n' Drop doesn't work for me. The browser just opens any PDF document instead. Tested with Firefox and Chrome on macOS.

The console shows this error in Chrome:

Is there anything I can do to resolve this?