ciur / papermerge Goto Github PK
View Code? Open in Web Editor NEWOpen Source Document Management System for Digital Archives (Scanned Documents)
Home Page: https://papermerge.com
License: Apache License 2.0
Open Source Document Management System for Digital Archives (Scanned Documents)
Home Page: https://papermerge.com
License: Apache License 2.0
Hi π,
thanks for implementing the backup and restore feature @frenos π.
I would like to propose additional user-centered stories here on how I would like to further shape this feature. Let me know what you think.
Given an offsite backup strategy via rclone
to backblaze
And incurred costs for each uploaded byte to the service
As a User, I want to perform incremental offsite backups from my papermerge
instance
so that I do not have to upload the complete and huge backup *.tar
file to the service for every new papermerge
backup which would increase my costs for the offsite backup service
Given the media root folder as a host mount in my docker installation rather than a volume (reference)
And the host mount is already included in my own rsync
backup strategy
As a User, I want to export/backup only the backup.json
file from papermerge
so that I only need to add the backup.json
to my existing rsync
backup strategy
I;m running the docker version (1.2.2 or 1.2.3). After i login on my Edge browser, i see the start screen. When clicking on Inbox, upload a document or do anything else, it will first return to the login screen.
I also tried chrome and to "older" edge, there was no problem,
Chome: Versie 83.0.4103.116
Edge: Versie 85.0.552.1
Simpel log from docker (can i debug more?)
Edge:
[11/Jul/2020 17:40:25] "GET /admin/core/basetreenode/1/list/ HTTP/1.1" 302 0
[11/Jul/2020 17:40:25] "GET /admin/login/?next=/admin/core/basetreenode/1/list/ HTTP/1.1" 200 3265
Chrome:
[11/Jul/2020 17:40:42] "GET /admin/core/basetreenode/1/list/ HTTP/1.1" 200 25983
[11/Jul/2020 17:40:42] "GET /document/2/preview/4/page/1 HTTP/1.1" 200 5080
[11/Jul/2020 17:40:43] "GET /clipboard/ HTTP/1.1" 200 1
Also tried the master version (1.4.0) there was the same problem.
Internal Server Error: /upload/
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\contrib\auth\decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\views\generic\base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\django\views\generic\base.py", line 97, in dispatch
return handler(request, *args, **kwargs)
File "C:\Users\Admin\papermerge-proj\papermerge\core\views\documents.py", line 282, in post
page_count = get_pagecount(f.temporary_file_path())
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\mglib\pdfinfo.py", line 83, in get_pagecount
compl = subprocess.run(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 489, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
[21/Jul/2020 17:46:30] "POST /upload/ HTTP/1.1" 500 128740
I am installing the docker version and there are Python dependency issues that are easily resolved but curious as to why they are missing:
django-rest-framework
django-rest-knox
But, once those are resolved, there are other dependencies that are unresolvable having to do with pdftk:
papermerge_service | ImportError: cannot import name 'pdftk' from 'pmworker' (/usr/local/lib/python3.7/dist-packages/pmworker/__init__.py)
papermerge_service | Traceback (most recent call last):
papermerge_service | File "manage.py", line 24, in <module>
papermerge_service | execute_from_command_line(sys.argv)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
papermerge_service | utility.execute()
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/core/management/__init__.py", line 377, in execute
papermerge_service | django.setup()
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/__init__.py", line 24, in setup
papermerge_service | apps.populate(settings.INSTALLED_APPS)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/apps/registry.py", line 114, in populate
papermerge_service | app_config.import_models()
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/apps/config.py", line 211, in import_models
papermerge_service | self.models_module = import_module(models_module_name)
papermerge_service | File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
papermerge_service | return _bootstrap._gcd_import(name[level:], package, level)
papermerge_service | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
papermerge_service | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
papermerge_service | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
papermerge_service | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
papermerge_service | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
papermerge_service | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
papermerge_service | File "/opt/papermerge/papermerge/core/models.py", line 27, in <module>
papermerge_service | from pmworker import pdftk
papermerge_service | ImportError: cannot import name 'pdftk' from 'pmworker' (/usr/local/lib/python3.7/dist-packages/pmworker/__init__.py)
In digging through the pmworker files, I do not see a pdftk function or class that could be imported.
Would love to get this to work and give this a proper test drive.
Tried this on Ubuntu 20.04
Docker version 19.03.9, build 9d988398e7
docker-compose version 1.25.5, build 8a1c60f6
Setup using these instructions: https://papermerge.readthedocs.io/en/latest/installation/docker.html
Chose to pull from docker hub rather than build locally
The first error from docker-compose logs app
is the following:
Internal Server Error: /admin/login/ papermerge_service | Traceback (most recent call last): papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/db/backends/utils.py", line 86, in _execute papermerge_service | return self.cursor.execute(sql, params) papermerge_service | psycopg2.errors.UndefinedTable: relation "django_site" does not exist papermerge_service | LINE 1: ..."django_site"."domain", "django_site"."name" FROM "django_si... papermerge_service |
http://localhost:8000 shows the following:
`Request Method: | GET |
---|---|
http://192.168.1.2:8000/admin/login/?next=/admin/core/basetreenode/ | |
3.0.2 | |
ProgrammingError | |
relation "django_site" does not exist LINE 1: ..."django_site"."domain", "django_site"."name" FROM "django_si... ^ | |
/usr/local/lib/python3.7/dist-packages/django/db/backends/utils.py in _execute, line 86 | |
/usr/bin/python3 | |
3.7.5 | |
['/opt/papermerge', '/opt/papermerge', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] | |
Wed, 20 May 2020 13:41:13 +0000 | |
` |
^
I think many of those interested in this project already have some kind of digital document archive. In many cases simply folder structures with PDFs in it.
It would be great be able to mount (e.g. via WebDAV or something more modern) the folder structure of papermerge for desktop search and import of documents.
How does Papermerge compare with other projects such as MayanEDMS and Paperless? This would be a good addition to the readme.
When I start the worker from manage.py worker and detach the process I'm not able to stop the worker.
There is no stop command and no possibility to write a pid file for the worker process.
I'm facing this issue when I try to create the start/stop mechanism for Synolog package #31 .
I have made a quick an dirty patch to pass the pidfile attributte, but I guess there are other more smarter solution.
Baybe to implement a worker start / workerr stop feature or to implement the pidfile as an optional command line argument.
What's the prefered solution?
It looks like the setting OCR_BINARY isn't taken into account.
The path for tesseract is hardcoded in mglib, as all other binaries.
I would really apriciate to have the ability to define binaries or paths with configuration file.
In the file docker/1.4/config/app.production.py the configuration variables for the database are set statically. That means, that credentials, port and the hostname cannot be chosen, but are predefined.
Can you change that to allow for custom configurations, most preferably set by environment variables?
Hello,
I tried your software and it's the one I want to handle my documents (Less complex than Mayan and more user-friendly than Paperless) but I'm unable to make it do what I want in 1.2.3.
I want to add the OCR in French and modify some part of the configuration (Mainly the DB).
I saw that all of that was available in 1.3.0 but my whole infrastructure runs on Docker.
So back to the original question: Would it be possible to get a Papermerge 1.3.0 Docker image ?
Best regards,
P-Y
I loved this masterpiece. i am new in python and django can please upload videos on How run project(windows Os).
[Update - 31.07.2020]
Verision 1.4 is in good shape. I take my time for last mile testing. I play with various settings to make sure it works in production as well. In couple of days I will release a docker image with tagged version 1.4.0.rc1 (release candidate 1)
Thank you to everybody for opening tickets, providing useful feedback and contributing to code.
I recorded a 10 minutes youtube screencast with latest updates (including demo of metadata feature) and roadmap for July and August 2020. For those of you who prefer text format - read on
Docker image downloaded and started 5 min ago, therefore latest/greatest
while uploading a 180kB, 20-page PDF document (that opens perfectly in okular)
papermerge_service | upload for f=2020-07-17_AGB_A02092019.pdf user=admin
papermerge_service | Internal Server Error: /upload/
papermerge_service | Traceback (most recent call last):
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/exception.py", line 34, in inner
papermerge_service | response = get_response(request)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 115, in _get_response
papermerge_service | response = self.process_exception_by_middleware(e, request)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 113, in _get_response
papermerge_service | response = wrapped_callback(request, *callback_args, **callback_kwargs)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
papermerge_service | return view_func(request, *args, **kwargs)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 71, in view
papermerge_service | return self.dispatch(request, *args, **kwargs)
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 97, in dispatch
papermerge_service | return handler(request, *args, **kwargs)
papermerge_service | File "/opt/papermerge/papermerge/core/views/documents.py", line 373, in post
papermerge_service | page_count = get_pagecount(f.temporary_file_path())
papermerge_service | File "/usr/local/lib/python3.7/dist-packages/pmworker/pdfinfo.py", line 63, in get_pagecount
papermerge_service | lines = compl.stdout.decode('utf-8').split('\n')
papermerge_service | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 101: invalid start byte
Hello,
I think creating a new docker-compose.yml for deployments that contains pre-built docker images published to DockerHub would make standing up a new deployment of papermerge much simpler. With the images in DockerHub they can now be referenced in a new docker-compose.yml file (similar to how docker-compose is currently referencing postgres) which would remove the build step.
This could also be used to version releases of papermerge as well.
What are your thoughts?
Howdy again,
Seem to be having a strange issue. I take the container down nightly as I do a backup of all of my containers and when i bring it back up in the morning, a couple of the files that had been deleted the previous day are back. The first time, i just thought it was me, but it has been 4 days now and those 2 files continue to appear daily. They get deleted daily and then are back in the morning when i bring it back up. Its weird. I am not a programmer or anything so let me know what you need to diagnose this.
OS: unRAID 6.8.3 (running docker-compose from the commandline)
PostGRES db container: default from your compose file
worker container version: eugenci/papermerge-worker:latest
service container version: eugenci/papermerge:1.2.x (and yes it is "x")
Only changes i have made to your compose file are:
changed the line 'depends_on: - db' to 'links: - db:db' (wouldn't talk the other way for some reason)
service version 1.2.x
added exposing ports for postgres (-p 5432:5432 which i had to do to get them to talk to one another for some reason), other than that it is what you wrote.
Also, no errors or anything in the logs. Just files reappearing.
Thanks for looking.
Paperless 1.3.0 was released some time ago. However, the provided docker images still seem to use 1.2.2 / 1.2.3.
Can you please update the docker image to 1.3.0?
Currently looking for a document management system and I am trying out Paperless and Mayan edms and I want to deploy this as well to try it out. Paperless was the easiest to install because it's in the community app store for unraid however I had to deploy mayan edms in a ubuntu vm where I used docker compose.
Unraid doesn't support docker compose natively and I tried installing it and installing papermerge but I couldn't get the web server to work. Has any gotten papermerge to work on unraid? If so how. I could just install it into a vm and have it work but I want papermerge to store all the files within my shares and not just in a docker container. Paperless does this great
https://gyazo.com/2494fb95e52a395d636b9afb3476e55f
however I can't seem to figure this out with papermerge
And eventually import back :)
Hello,
I would like to follow these instructions
(https://papermerge.readthedocs.io/en/latest/settings.html#Email)
Import my mails, unfortunately I don't know where in docker compose the config for this is, or I can't find it, can someone help me?
First of all great work :)
I just installed papermerge and noticed, that my files are displayed as folders. The icon seems to be wrong.
I installed it today via Docker with this documentation: https://papermerge.readthedocs.io/en/latest/setup/docker.html
Here you can see an example:
Do you need more information?
Crash while trying to add a user
brand new installation using docker compose files.
docker compose used:
version: '3.7'
services:
app:
image: eugenci/papermerge:1.2.2
command: python3 /opt/papermerge/manage.py runserver 0.0.0.0:8000
ports:
- "8888:8000"
depends_on:
- db
volumes:
- media_root:/opt/media
- broker_queue:/opt/broker/queue
environment:
- DJANGO_SETTINGS_MODULE=config.settings.stage
deploy:
placement:
constraints:
- node.labels.arch == amd64
- node.role == worker
db:
image: postgres:11.7
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=dbuser
- POSTGRES_PASSWORD=dbpass
- POSTGRES_DB=dbname
deploy:
placement:
constraints:
- node.labels.arch == amd64
- node.role == worker
worker:
image: eugenci/papermerge-worker:1.2.x
command: bash -c "cd /opt/papermerge-worker && celery worker -A pmworker.celery -Q papermerge -l info"
volumes:
- media_root:/opt/media
- broker_queue:/opt/broker/queue
environment:
- CELERY_CONFIG_MODULE=config
- C_FORCE_ROOT=True
deploy:
placement:
constraints:
- node.labels.arch == amd64
- node.role == worker
volumes:
postgres_data:
broker_queue:
media_root:
Hi @ciur
first off, you accomplished a piece of software I was looking for!! Awesome! THX!
Unfortunately, I run into issues...
My server runs a bitwarden instance with docker, if I now try to run sudo docker-compose up -d
to start papermerge I get the following error:
ERROR: Network "docker_default" needs to be recreated - internal has changed
If I stop bitwarden and than start papermerge only, everything works seamless... but I need to have both pieces run in parallel. Port issues are not given...
Any hint on how this could be solved?!
Best regards and Thank you for any help!
goeste
Me again.
I was curious if you could add a default setting to sort everything alphabetically automatically. I know we have the option to sort that way, but the default sorting is in whatever order you added folders or files.
Hey,
Thanks for this amazing software! I've taken a jab at creating an ansible task to install this, to integrate it into my personal fork of this project. Let me know if this is something you're interested in adding to the project.
Cheers!
---
- name: Create Papermerge Directories
file:
path: "{{ item }}"
state: directory
with_items:
- "{{ papermerge_data_directory }}/postgres"
- "{{ papermerge_data_directory }}/broker"
- name: Create Postgres for Papermerge
docker_container:
name: papermerge-postgres
image: postgres:11.7
pull: yes
state: started
restart: yes
volumes:
- "{{ papermerge_data_directory }}/postgres:/var/lib/postgresql/data:rw"
env:
POSTGRES_USER: "papermerge"
POSTGRES_PASSWORD: "supersecure"
restart_policy: unless-stopped
memory: 1g
- name: Create Papermerge service container
docker_container:
name: papermerge-service
image: eugenci/papermerge
pull: yes
state: started
restart: yes
command: python3 /opt/papermerge/manage.py runserver 0.0.0.0:8000
links:
- papermerge-postgres:db
ports:
- "{{ papermerge_port }}:8000"
env:
POSTGRES_DB: "papermerge"
POSTGRES_USER: "papermerge"
POSTGRES_PASSWORD: "supersecure"
DJANGO_SETTINGS_MODULE: "config.settings.stage"
volumes:
- "{{ papermerge_media_directory }}:/opt/media:ro"
- "{{ papermerge_data_directory }}/broker:/opt/broker/queue:rw"
restart_policy: unless-stopped
memory: 1g
- name: Create Papermerge worker container
docker_container:
name: papermerge-worker
image: eugenci/papermerge-worker
pull: yes
state: started
restart: yes
command: bash -c "cd /opt/papermerge-worker && celery worker -A pmworker.celery -Q papermerge -l info"
env:
CELERY_CONFIG_MODULE: "config"
C_FORCE_ROOT: "true"
volumes:
- "{{ papermerge_media_directory }}:/opt/media:ro"
- "{{ papermerge_data_directory }}/broker:/opt/broker/queue:rw"
restart_policy: unless-stopped
memory: 1g
Hy @ciur
just cloned the new release and started up docker. unfortunately the .jpg bug is still there
regards tom
I recorded new demo video - here is youtube link.
In video is presented a workflow which I use very often for my own documents (batch scan -> email -> papermerge -> organize pages).It might be useful for you as well.
There is a online demo as well: https://demo.papermerge.com
Credentials are open for everyone:
Hey, I just found papermerge and it seems very nice, the only thing which stops me from using is the folder system. I really love to use tags in many applications, so I would love to use tags to categorize my documents. Do maybe could implement such a feature in the future? Because with folders you're very limited to sort files.
I was kindly asked to create a tutorial/documentation on how to run papermerge on windows.
Well, I never done that... And I don't see it as necessary. However, just as proof of concept will try that :).
Hi,
As far as i understand the documentation and the section about the metadata plugins in the file papermerge.conf.py.example, there should be python plugins which extract the metadata from the ocr'd content. (btw, the documentation doesn't mention the metadata plugins).
I found the metadata plugin lidl-recepts-de on pypi.org (oh and btw the 'i' is missing compared to the linked github repo papermerge/lidl-receipts-de and the comment in papermerge.conf.py.example) and the code seems pretty easy, so i would like to implement my own metadata plugins for all my monthly/quarterly/yearly reoccurring documents and of course publish them for everyone to use.
So before i start implementing these plugins i have the following questions:
Shouldn't the plugin names have a specific prefix like 'papermerge-metadata-' or something like that (which would give 'papermerge-metadata-lidl-recepts-de')? That way it would be easier to find the plugin on pypi.org.
Should there really be a single plugin for every single type of document (one for lidl, one for rewe,...)? If someone would only implement a metadata plugin for each bigger grocery/electronics/household/furniture/etc store in the german speaking theritory, there would be hundreds of single plugins. Then there will be many plugins which are maintained by other people and these plugins will get outdated when you change something on the interface between papermerge and these metadata plugins. Would't it be much better to just have a single plugin where everything is contained and everyone contributes through push requests? Or maybe use a completely other concept, i.e. a frontend where the user defines a rule to match the document type (if the document is from lidl or rewe) and then adds rules to extract the necessary metadata for this document type and maybe a github repo with existing templates.
Is there a way to map a result of the metadata plugin to a configured metadata with another name?
Example: The lidl-recepts-de plugin returns metadata for 'shop', 'price' and 'date', but i would like to have them displayed in the frontend in german as 'Firma', 'Betrag' und 'Datum'. (this is based on the code, i didn't actually tested it with a lidl receipt).
Some background information:
My objective is to just scan a document (or get it via mail) and then have the DMS automatically add metadata and push it into the right folder without any further manual intervention from me (at least for all monthly reoccurring bills, cause i hate doing the same manual thing each month if i can automate it). If i scan a bill i.e from the grocery store Lidl, a custom script would get the picture from my scanner (cause my scanner doesnt directly support remote storage) and upload it via api to a custom inbox folder in papermerge, then there would be the metadata 'shop' on this folder and another custom script would then move the document (via api and a not yet existing move command) to the directory bills/lidl which has the metadata date, price and location of the store defined. This way papermerge would be pretty much exactly the DMS i'm looking for.
I'm running Version 1.3.0. Oh and I'm from Austria, so you could even write in German in case you PM me.
Since the app container runs a HTTP server on port 8000, it should expose that port:
EXPOSE 8000
That makes integration into a reverse proxy easier ;)
What about the idea to create a package for Synology NAS devices, maybe based and distributed on SynoCommunity spksrc?
We could create a simple installation wizard to make the configuration simple and comfortable.
Was well I would prefer to create a native package (not based on docker), because this can be used on smaller devices without docker as well.
I would really appreciate to use Synology single sign on, is there an authentication middleware available for Django?
I've followed the steps in the docs for a manual install under ubuntu 18.04.2. All requirements.txt libs were satisfied correctly. When I run manage.py migrate, I get an error
from mglib import step ImportError: cannot import name 'step'
I also get similar errors if I manage.py createsuperuser.
Doing an inspect on mglib in the python repl suggests that the library has no functions, which seems odd (but I am no python expert). Is pip pulling in the wrong mglib somehow?
I have been trying to deploy papermerge on Podman containers and came to the point, where it would be necessary to customize a few settings and override config settings in the configuration files within the containers. I know this is possible via a volume mounts, but feels a bit cumbersome to me (maybe I have overlooked something, and this is possible in some more elegant way - in this case please simply close and forget this issue π ).
Since I have used Django Environ in a few projects, I asked myself, if you would be interested in discussing an adoption of this configuration method in papermerge.
This way, you could use sensible defaults in a simple settings.py and in case of running on a docker/podman container have all necessary settings overridden using environment variables and in case of running under systemd with an environment file. I know, this would be quite a breaking change.
If you are interested, I am really interested in helping out and creating a pull request.
Nonetheless, it's just an idea.
found this issue while testing jpeg.
I will provide a fix as soon as possible.
Hi,
I'm a long time Teedy user but wanted to give papermerge a try since the youtube clips look amazing.
However, I'm having trouble setting it up via docker compose.
This is my compose file:
version: '2.0'
services:
app:
image: eugenci/papermerge:1.2.3
container_name: papermerge_service
command: python3 /opt/papermerge/manage.py runserver 0.0.0.0:8000
ports:
- 8222:8000
depends_on:
- db
volumes:
- /home/a300/apps/papermerge/media_root:/opt/media
- /home/a300/apps/papermerge/broker_queue:/opt/broker/queue
environment:
- DJANGO_SETTINGS_MODULE=config.settings.stage
db:
image: postgres:11.7
container_name: postgres_db
volumes:
- /home/a300/apps/papermerge/postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=dbuser
- POSTGRES_PASSWORD=dbpass
- POSTGRES_DB=dbname
worker:
image: eugenci/papermerge-worker:1.2.x
container_name: papermerge_worker
command: bash -c "cd /opt/papermerge-worker && celery worker -A pmworker.celery -Q papermerge -l info"
volumes:
- /home/a300/apps/papermerge/media_root:/opt/media
- /home/a300/apps/papermerge/broker_queue:/opt/broker/queue
environment:
- CELERY_CONFIG_MODULE=config
- C_FORCE_ROOT=True
worker and db container start just fine and nothing indicates that there is a problem. But the main app gives me the following error:
File "/usr/local/lib/python3.7/dist-packages/django/utils/asyncio.py", line 26, in inner,
return func(*args, **kwargs),
File "/usr/local/lib/python3.7/dist-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection,
connection = Database.connect(**conn_params),
File "/usr/local/lib/python3.7/dist-packages/psycopg2/__init__.py", line 126, in connect,
conn = _connect(dsn, connection_factory=connection_factory, **kwasync),
django.db.utils.OperationalError: could not translate host name "db" to address: No address associated with hostname,
All containers run on bridge network.
Here is the full log file of the app container:
https://pastebin.com/BF5P3JaR
What am I missing here?
Thanks for your help
I can't find any documentation on how the files are stored on disk. Are they in the folders to match the structure should on the webUI?
I got feedback about lack of support of other than German + English languages.
I will make this high priority issue and in the next release (1.3) I will add support for all languages (the once supported by tesseract). I mean, users will be able to OCR documents in other (than German and English) languages as well.
In order to make the install easy for u new user (you have only have one chance for the first impression), I suggest during install to create within the install directory the folders for: /data /media/ /media/documents /media/import. For example and based on this I would update the .conf file.
# Paths & Folders
#######################
# You can specify where you want the SQLite database to be stored instead of
# the default location of **/data/** within the install directory.
# DBDIR = "/path/to/papermerge/db"
# Override the default MEDIA_ROOT. This is where all files are stored.
# The default location is **/media/documents/** within the install directory.
# MEDIA_DIR = "/path/to/media/dir"
# Override the default STATIC_ROOT. All static files created with
# "collectstatic" manager-command, **their default location is /.... ? Is where?**
# STATIC_DIR = "/path/to/static/dir"
# Override the default MEDIA_URL here. Unless you're hosting Papermerge off a subdomain
# like /papermerge/, you probably don't need to change this. Ich verstehe leider nicht was damit gemeint ist, sieht aus wie ein Ordner?
MEDIA_URL = "/media/"
# Override the STATIC_URL here. Unless you're hosting Papermerge off a
# subdomain like /papermerge/, you probably don't need to change this.
STATIC_URL = "/static/"
# Document Importer
#########################
# Configuration for ./manage.py importer command
# This where ./manage importer will import your documents from.
# The default location is **/media/import/** Wie wΓ€re es auch hier einen Standard-Pfad/Order anzubieten?
# IMPORTER_DIR = "/path/to/import/dir"
MEDIA_URL = "/media/" Does this need a folder too?
What do you think?
WARNINGS:
?: Papermerge can't find tesseract. Without it, OCR of the documents is impossible.
HINT: Either it's not in your ${PATH} or it's not installed.
please tell me solution for this warnings
Issue reported by @lucaz82
Hey I cloned your repo last friday and today.
Building the containers works fine I suppose but upon starting the papermerge_service container exits with Exit 1 and says standard_init_linux.go:211: exec user process caused "no such file or directory"
I'm using docker desktop on windows
Hi π,
this is a reply to your paperless issue ticket showcasing papermerge. I would like to give you some feedback on the project from the perspective of a paperless user. Further, I would like to describe my current paperless setup and how I would love to use papermerge in the future (tested version v1.2.3)
Feel free to ask questions if something is unclear.
20200711Z - correspondent - title
and choose WebDav target in the app to upload it to my NASPost-consumption
script of paperless, I send a notification via Telegram to my phoneimport to documents
the document's metadata is merged with my existing structure. Then, I wouldn't have to delete the wrong tags and correspondents.20200711Z - correspondent - title
and choose WebDav target in the app to upload it to my NASFollowing a few things that do not work out for me and therefore block me from migrating to papermerge as a paperless user:
./import:/opt/papermerge/import/
with the worker, I have no permissions to copy files in the created host mount import
KeyError at /paste-pages/
Request Method: | GET |
---|---|
http://0.0.0.0:8000/admin/core/user/add/ | |
3.0.2 | |
NoReverseMatch | |
Reverse for 'auth_user_changelist' not found. 'auth_user_changelist' is not a valid view function or pattern name. | |
/usr/local/lib/python3.7/dist-packages/django/urls/resolvers.py in _reverse_with_prefix, line 677 | |
/usr/bin/python3 | |
3.7.5 | |
['/opt/papermerge', '/opt/papermerge', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] | |
Fri, 24 Jul 2020 07:36:55 +0000 |
By config/settings/development.py is ignored
This sentence may need a quick look-over: I can't parse out its meaning. I suspect it should be
By config, settings/development.py is ignored
.. and I'd love to just make that trivial change, but I'm not sure that's the intent.
Hello, im an Spanish user. I want to test the app since I understand a bit of python, but noticed there is no Spanish support. I gave a quick look the source and saw some language related stuff in the database.
Do you plan to add language support?
Can you outline the areas where language is hard coded?
thanks
The search function returns nothing. Even when searching for a word in a file name. Any help?
Also, when starting the worker, logs shows it as fine. But when I upload a file to be OCR'd, i get this.... For every file. Eventually i can highlight words in documents but i cannot search for anything using those words. Just empty search results.
`Arguments: ()
[2020-05-02 14:25:44,325: INFO/ForkPoolWorker-1] worker_log success task_id=3e55acf4-883e-4ec0-bd3f-1d70cc3c4e2b user_id=1 doc_id=91 page_num=1 page_type=pdf total_exec_time=9.04
[2020-05-02 14:25:44,331: INFO/ForkPoolWorker-1] Task pmworker.tasks.ocr_page[3e55acf4-883e-4ec0-bd3f-1d70cc3c4e2b] succeeded in 9.049290722003207s: True
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] --- Logging error ---
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] Traceback (most recent call last):
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] File "/usr/lib/python3.7/logging/handlers.py", line 934, in emit
self.socket.send(msg)
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] OSError: [Errno 9] Bad file descriptor
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] During handling of the above exception, another exception occurred:
[2020-05-02 14:25:44,333: WARNING/ForkPoolWorker-1] Traceback (most recent call last):
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] File "/usr/lib/python3.7/logging/handlers.py", line 855, in _connect_unixsocket
self.socket.connect(address)
[2020-05-02 14:25:44,334: INFO/MainProcess] Received task: pmworker.tasks.ocr_page[13d1bd01-8fef-4510-9dc9-bab1738e1f5a]
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] FileNotFoundError: [Errno 2] No such file or directory
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] During handling of the above exception, another exception occurred:
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] Traceback (most recent call last):
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] File "/usr/lib/python3.7/logging/handlers.py", line 937, in emit
self._connect_unixsocket(self.address)
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] File "/usr/lib/python3.7/logging/handlers.py", line 866, in _connect_unixsocket
self.socket.connect(address)
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] FileNotFoundError: [Errno 2] No such file or directory
[2020-05-02 14:25:44,334: WARNING/ForkPoolWorker-1] Call stack:
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/bin/celery", line 8, in
sys.exit(main())
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/main.py", line 16, in main
_main()
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/celery.py", line 322, in main
cmd.execute_from_commandline(argv)
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/celery.py", line 495, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/base.py", line 305, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/celery.py", line 487, in handle_argv
return self.execute(command, argv)
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/celery.py", line 419, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/worker.py", line 223, in run_from_argv
return self(*args, **options)
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/base.py", line 253, in call
ret = self.run(*args, **kwargs)
[2020-05-02 14:25:44,335: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bin/worker.py", line 259, in run
worker.start()
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/worker/worker.py", line 205, in start
self.blueprint.start(self)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/bootsteps.py", line 369, in start
return self.obj.start()
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/concurrency/base.py", line 132, in start
self.on_start()
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/concurrency/prefork.py", line 117, in on_start
**self.options)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/pool.py", line 1046, in init
self._create_worker_process(i)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/pool.py", line 1158, in _create_worker_process
w.start()
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/process.py", line 124, in start
self._popen = self._Popen(self)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/context.py", line 333, in _Popen
return Popen(process_obj)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/popen_fork.py", line 24, in init
self._launch(process_obj)
[2020-05-02 14:25:44,336: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/popen_fork.py", line 79, in _launch
code = process_obj._bootstrap()
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/process.py", line 327, in _bootstrap
self.run()
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/pool.py", line 292, in call
sys.exit(self.workloop(pid=pid))
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/billiard/pool.py", line 362, in workloop
result = (True, prepare_result(fun(*args, **kwargs)))
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/app/trace.py", line 558, in _fast_trace_task
uuid, args, kwargs, request,
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/app/trace.py", line 385, in trace_task
R = retval = fun(*args, **kwargs)
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/usr/local/lib/python3.7/dist-packages/celery/app/trace.py", line 650, in protected_call
return self.run(*args, **kwargs)
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] File "/opt/papermerge-worker/pmworker/tasks.py", line 95, in ocr_page
f"worker_log task_id={self.request.id}"
[2020-05-02 14:25:44,337: WARNING/ForkPoolWorker-1] Message: 'worker_log task_id=ac060393-e3fa-4a6f-9ea4-16809a0b56f5 user_id=1 doc_id=93 page_num=1'
Arguments: ()
[2020-05-02 14:25:44,333: INFO/ForkPoolWorker-1] worker_log task_id=ac060393-e3fa-4a6f-9ea4-16809a0b56f5 user_id=1 doc_id=93 page_num=1`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.