librephotos/librephotos

A self-hosted open source photo management service. This is the repository of the backend.

LibrePhotos

_{Mockup designed by rawpixel.com / Freepik}

• The demo is available here. User is demo, password is demo1234.
• You can watch development videos on Niaz Faridani-Rad's channel
• You can join our Discord.

Installation

Step-by-step installation instructions are available in our documentation

How to help out

• ⭐ Star this repository if you like this project!
• 🚀 Developing: Get started in less than 30 minutes by following this guide.
• 🗒️ Documentation: Improving the documentation is as simple as submitting a pull request here
• 🧪 Testing: If you want to help find bugs, use the dev tag and update it regularly. If you find a bug, open an issue.
• 🧑‍🤝‍🧑 Outreach: Talk about this project with other people and help them to get started too!
• 🌐 Translations: Make LibrePhotos accessible to more people with weblate.
• 💸 Donate to the developers of LibrePhotos

Features

• Support for all types of photos including raw photos
• Support for videos
• Timeline view
• Scans pictures on the file system
• Multiuser support
• Generate albums based on events like "Thursday in Berlin"
• Face recognition / Face classification
• Reverse geocoding
• Object / Scene detection
• Semantic image search
• Search by metadata

What does it use?

• Image Conversion: ImageMagick
• Video Conversion: FFmpeg
• Exif Support: ExifTool
• Face detection: face_recognition
• Face classification/clusterization: scikit-learn and hdbscan
• Image captioning: im2txt,
• Scene classification places365
• Reverse geocoding: Mapbox: You need to have an API key. First 50,000 geocode lookups are free every month.

librephotos's Issues

Link photos from NextCloud

Currently, the Nextcloud Client downloads all the pictures into its own folder.
r/selfhosted would like an option where the original photos are just linked and not downloaded. We would still create the thumbnails and create the tags, but when we open a picture a request goes to the Nextcloud server instead.

We use this client to download the images currently:
https://github.com/owncloud/pyocclient

As I don't think we want to share all pictures via a public link, we want to save the information about the path within the photo. We need a new attribute "nextcloud_address", where we can save the server link and this would signify that the media is on a nextcloud server.

We would only download the image temporarily to create the thumbnails and read the metadata. This should be handled here and here.

We have to change all functions where an original is accessed. This would be currently on "saving metadata to disk / saving favorites" here and "download images" here. We would also need to redirect the request when accessing videos, because we read the original for that here.

macOs preview thumbnails leads to crashes

If in the folder of the data is a preview thumbnail of macOS, than the librephotos scan will crash.

Move window scrollbar only works for the first row

This attempts to move the window scrollbar to the photo when clicking the arrow buttons in the photo viewer or keyboard keys and that is a good idea, however as implemented it only works correctly for the first row of photos in any day:
https://github.com/LibrePhotos/librephotos-frontend/blob/dev/src/layouts/%5C#L378-L391

Reverse geocode with OpenStreetMap

There is a geocoder based on OpenStreetMap, where you don't need an api key, but the json response is different. It would be great if that would be the default. The json response is different to the current mapbox json reponse.

Databasequeries slow

I have around 10000 photos in my database. All the queries take multiple seconds to execute. It is probably a regression introduced by #7.

Nextcloud scan broken

I installed newest images from docker, on fresh server and nextcloud scan is not working.

Logs:

2020-12-12 17:01:05,131 : directory_watcher.py : isValidMedia : 20 : ERROR : Following image throwed an exception: /Folder/Second_Folder/
Traceback (most recent call last):
  File "/code/api/directory_watcher.py", line 17, in isValidMedia
    filetype = magic.from_file(p, mime=True)
  File "/miniconda/lib/python3.8/site-packages/magic.py", line 153, in from_file
    return m.from_file(filename)
  File "/miniconda/lib/python3.8/site-packages/magic.py", line 95, in from_file
    with open(filename):
FileNotFoundError: [Errno 2] No such file or directory: '/Folder/Second_Folder/'

2020-12-12 17:01:33,176 : directory_watcher.py : isValidMedia : 20 : ERROR : Following image throwed an exception: /Folder/Second_Folder/18-09-21 07-51-50 0959.jpg
Traceback (most recent call last):
  File "/code/api/directory_watcher.py", line 17, in isValidMedia
    filetype = magic.from_file(p, mime=True)
  File "/miniconda/lib/python3.8/site-packages/magic.py", line 153, in from_file
    return m.from_file(filename)
  File "/miniconda/lib/python3.8/site-packages/magic.py", line 95, in from_file
    with open(filename):
FileNotFoundError: [Errno 2] No such file or directory: '/Folder/Second_Folder/18-09-21 07-51-50 0959.jpg'

Local scan is working

Option to use high accuracy face recognition model

The face recognition has an option to use a high accuracy model which need 16GB of RAM see https://face-recognition.readthedocs.io/en/latest/face_recognition.html?highlight=face_locations#face_recognition.api.face_locations

We should make it a configurable option in the admin settings page.

Option to set scan directory missing in admin area

Followed the recommended Docker-compose method.
I have modified only the path to volumes under the backend inside the docker-compose.yml file (as shown)

  backend:
    image: reallibrephotos/librephotos:dev
    # For development uncomment the "build" line below and comment the "image" line above
    # build: .
    container_name: librephotos-backend
    restart: always
    volumes:
      - $HOME/Pictures/librephotos/librephotos_images:/data # CHANGE ME
      - $HOME/Pictures/librephotos/librephotos_media:/code/protected_media # CHANGE ME
      - $HOME/Pictures/librephotos/librephotos_logs:/code/logs # CHANGE ME
      - $HOME/Pictures/librephotos/librephotos_cache:/root/.cache # CHANGE ME

Logged in as admin but the link to set the scan directory is missing.

Not sure if this helps but found this issue in the backend logs on running docker logs librephotos-backend

Traceback (most recent call last):
File "/miniconda/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 55, in handle
self.handle_request(listener_name, req, client, addr)
File "/miniconda/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 143, in handle_request
super().handle_request(listener_name, req, sock, addr)
File "/miniconda/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/miniconda/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 133, in call
response = self.get_response(request)
File "/miniconda/lib/python3.8/site-packages/django/core/handlers/base.py", line 75, in get_response
response = self._middleware_chain(request)
File "/miniconda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 36, in inner
response = response_for_exception(request, exc)
File "/miniconda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
File "/miniconda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 125, in handle_uncaught_exception
return debug.technical_500_response(request, *exc_info)
File "/miniconda/lib/python3.8/site-packages/django/views/debug.py", line 91, in technical_500_response
text = reporter.get_traceback_text()
File "/miniconda/lib/python3.8/site-packages/django/views/debug.py", line 341, in get_traceback_text
return t.render(c)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 171, in render
return self._render(context)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 986, in render
output = self.filter_expression.resolve(context)
File "/miniconda/lib/python3.8/site-packages/django/template/base.py", line 693, in resolve
obj = template_localtime(obj, context.use_tz)
File "/miniconda/lib/python3.8/site-packages/django/utils/timezone.py", line 186, in template_localtime
return localtime(value) if should_convert else value
File "/miniconda/lib/python3.8/site-packages/django/utils/timezone.py", line 204, in localtime
timezone = get_current_timezone()
File "/miniconda/lib/python3.8/site-packages/django/utils/timezone.py", line 97, in get_current_timezone
return getattr(_active, "value", get_default_timezone())
File "/miniconda/lib/python3.8/site-packages/django/utils/timezone.py", line 83, in get_default_timezone
return pytz.timezone(settings.TIME_ZONE)
File "/miniconda/lib/python3.8/site-packages/pytz/init.py", line 181, in timezone
raise UnknownTimeZoneError(zone)
pytz.exceptions.UnknownTimeZoneError: ''
`

Enable object detection and classification

The project uses densecap, but the original author disabled it. There is no explanation why it was disabled. We should be trying to get it to run and evaluate if it is usable or if we need a new machine learning model.

librephotos/api/models.py

Lines 206 to 221 in 289f413

	# densecap disabled for now
	if False:
	try:
	with open(image_path, "rb") as image_file:
	encoded_string = base64.b64encode(image_file.read())
	encoded_string = str(encoded_string)[2:-1]
	resp_captions = requests.post(
	'http://localhost:5000/', data=encoded_string)
	captions['densecap'] = resp_captions.json()['data'][:10]
	self.search_captions = ' , '.join(
	resp_captions.json()['data'][:10])
	self.save()
	except:
	util.logger.warning(
	'could not generate densecap captions for image %s' %
	image_path)

We need to also need to change this function to add it to the AlbumThings:

librephotos/api/models.py

Lines 496 to 515 in 289f413

	def _add_to_album_thing(self):
	if type(self.captions_json
	) is dict and 'places365' in self.captions_json.keys():
	for attribute in self.captions_json['places365']['attributes']:
	album_thing = get_album_thing(
	title=attribute, owner=self.owner)
	if album_thing.photos.filter(
	image_hash=self.image_hash).count() == 0:
	album_thing.photos.add(self)
	album_thing.thing_type = 'places365_attribute'
	album_thing.save()
	for category in self.captions_json['places365']['categories']:
	album_thing = get_album_thing(title=category, owner=self.owner)
	if album_thing.photos.filter(
	image_hash=self.image_hash).count() == 0:
	album_thing = get_album_thing(
	title=category, owner=self.owner)
	album_thing.photos.add(self)
	album_thing.thing_type = 'places365_category'
	album_thing.save()

Photos not loading (403) after browser restart

I got an 403 error, when I close and open again chrome. To fix this I need to logout and login again, or it just randomly start woking after a while.

Docker-free installation process

I got the request for a docker-free installation process.
Jellyfin for Example has docker images but also has binaries for windows, macOS and Linux.

I think we have to use pyinstaller for that. We have to bundle portable versions of the dependencies like PostgreSQL, Redis, Nginx and Node.js with it.

To build all the binaries we should probably have a CI/CD process in place.

We would also need to create an optional settings file where you can configure the parameters we currently set via docker-compose

Rescan photos / manually add face

So I accidently deleted some faces, there is no option to rescan photos or manually add face. This would be nice feature.

Manually deleting photos via Image Similarity

Also for the future, I would like to see an "Image Similarity" page under "Dashboards"
On that page show all the groupings of two or more very similar photos and offer a delete button on each photo.
That would enable manual cleaning of similar photos (photos of the same subject taken within minutes of each other, edited photos, etc.)

Add Multithreading / Batchprocessing

I'm using local folder now. Each photo is scanned for about 8-10 seconds, while cpu usage is at about 20% with peaks to 40% (on 4c/8t xeon).
Also, ram usage seems to be very low.
I'm on default docker configuration. Maybe it is docker fault?

Album -> places showing nothing

There is some places, but no photos / albums here.

Explain directory structure

So looking through the docker-compose file there are 4 folders the backend needs:

• data
• protected_media
• logs
• cache

Those are no where described.

1. It is not clear to me if my photos go into data or protected_media.
   I sure want my media to be secure and protected?
2. Is the logs folder really needed? Shouldn't that end up in the docker stdout already?
3. Is the cache really needed in case of a container restart?

Change dependency to DeepFace

DeepFace has an ensemble model which should lead to better results than our current model.

Auto scan for photos

Any chance to implement some kind of watcher that will watch for changes and start scanning for photos? Or just a script that can be executed in cron. Would be really nice feature.

Import photos from nextcloud into data folder

Hey, I saw that there is some development in here, so I have some question / idea.

I noticed that when I imported photos from nextcloud, space on ssd drive where I keep my docker images and configs significantly increased, and when I checked what it was I saw that all data from nextcloud was copied into ownphotos image/container (not sure, I'm not an expert with docker). Is it possible to copy it to data folder instead? I'd prefer to keep photos on hdd with a lot of space.

App does not work when backend goes offline

After logged in and then some time later the backend went offline, the web app remained logged in.

For example:
I hid a photo while the backend was offline. The frontend shown the photo as hidden. I started the backend up again and refreshed the page, the change did not take effect on the database because the database was offline.

Proposal: The frontend will logout the user immediately in the event the frontend knows the backend connection is broken.

docker-compose.yml

db container is not persistent may cause unintentional headache for users. Need to change to persistent.

Explain development process

We should create a wiki page for the development process.

These questions should be answered:
How to setup the dev environment? How to use docker and docker-compose? Which IDE to use? What kind of quality shoud a pull request have? How to open a pull request?

Improve InfinteScrolling

Currently, 50k-100k images are loaded as hashes at once. This is not very efficient and leads to problems if the number of pictures is high enough.

Instead, we should load fewer images like 250 image hashes at once and load the next 250 image hashes if we are at the bottom of the page.

no response from backend

Hi,

I've got a problem, installing librephotos on my NAS. It's a Synology with an Atom.
I installed it using your compose-file. All containers are running. I just need to add the proxy to my brdige network and i was able to connect to the frontend over port 3000.

CONTAINER ID        IMAGE                                      COMMAND                  CREATED             STATUS                PORTS                  NAMES
10f9de80d73b        reallibrephotos/librephotos-proxy:dev      "/docker-entrypoint.…"   12 minutes ago      Up 12 minutes         0.0.0.0:3000->80/tcp   librephotos-proxy
5ed6c2be6935        reallibrephotos/librephotos-frontend:dev   "/bin/sh -c ./run.sh…"   12 minutes ago      Up 12 minutes         3000/tcp               librephotos-frontend
0c9743f502b2        reallibrephotos/librephotos:dev            "/bin/sh -c ./entryp…"   13 minutes ago      Up 13 minutes         80/tcp                 librephotos-backend
6fdf7bc647a6        postgres                                   "docker-entrypoint.s…"   15 minutes ago      Up 15 minutes         5432/tcp               librephotos-db
9a049db72a7b        redis                                      "docker-entrypoint.s…"   15 minutes ago      Up 15 minutes         6379/tcp               librephotos-redis

When I try to login, i've got the following error message. The browser never receives a response for the call to *:3000/api/auth/token/obtain/.

I checked the log of the backend container and got the following:

 * Restarting nginx nginx
   ...done.
./entrypoint.sh: line 29:    52 Illegal instruction     (core dumped) /miniconda/bin/python manage.py shell <<EOF
from api.models import User

if User.objects.filter(username="$ADMIN_USERNAME").exists():
    admin_user = User.objects.get(username="$ADMIN_USERNAME")
    admin_user.set_password("$ADMIN_PASSWORD")
    admin_user.save()
else:
    User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF

Running backend server...
[2020-12-10 08:23:25 +0000] [56] [INFO] Starting gunicorn 20.0.4
[2020-12-10 08:23:25 +0000] [56] [INFO] Listening at: http://0.0.0.0:8001 (56)
[2020-12-10 08:23:25 +0000] [56] [INFO] Using worker: gevent
[2020-12-10 08:23:25 +0000] [60] [INFO] Booting worker with pid: 60
[2020-12-10 08:23:25 +0000] [61] [INFO] Booting worker with pid: 61
[2020-12-10 08:23:27 +0000] [62] [INFO] Booting worker with pid: 62
[2020-12-10 08:23:28 +0000] [63] [INFO] Booting worker with pid: 63
[2020-12-10 08:23:30 +0000] [64] [INFO] Booting worker with pid: 64
[2020-12-10 08:23:30 +0000] [65] [INFO] Booting worker with pid: 65
[2020-12-10 08:23:32 +0000] [66] [INFO] Booting worker with pid: 66
[2020-12-10 08:23:32 +0000] [67] [INFO] Booting worker with pid: 67
[2020-12-10 08:23:34 +0000] [68] [INFO] Booting worker with pid: 68
[2020-12-10 08:23:34 +0000] [69] [INFO] Booting worker with pid: 69
[2020-12-10 08:23:37 +0000] [70] [INFO] Booting worker with pid: 70
[2020-12-10 08:23:37 +0000] [71] [INFO] Booting worker with pid: 71
[2020-12-10 08:23:39 +0000] [72] [INFO] Booting worker with pid: 72
[2020-12-10 08:23:39 +0000] [73] [INFO] Booting worker with pid: 73

Any idea whats going wrong?

BTW: Thanks for reviving the project!!! 😃

Login Page Error UI Broken

At start up, it takes awhile before Gunicorn starts up.

The login page went to a blank page when I tried to log in when the backend was offline.

Some code exists in the frontend to tell the user what happened however, it errored out and the page went blank.

Hide hidden photos from all views

Another thing I noticed is that when hiding a photo, Ownphotos keeps the photo item in the "Photos -> With Timestamp" page with the photo blanked out. I would prefer hidden photos to only appear in the "Photos -> Hidden" page.

Photos not centering when info panel is opened

Horizontal photos are not centering when info panel is opened:

It is working ok with vertical photos:

Also, after opening info panel and switching to next photo, there is a fragment of next photo visible at the left and sometimes right, this does not occur when I don't touch info panel.

Refreshing doen't work correctly because of caching

Comment by sim4city in discord:

---

Also, the frontend service worker is attempting to cache links to the backend because both servers are publicly accessible on the same port (:80) via nginx proxy, including links to /media/, /api/, /, etc.

Unfortunately, that is problematic for me because the web browser service worker does not show photos linked from the backend /media/... and etc. backend urls defined in ownphotos.urls.

Ctrl+R refresh gives me the frontend service worker when trying to view a photo link: .../media/photos/6bce9f3c09df581a6c82346a2847af401.jpg

That shows a blank white page from the service worker.

I worked around this by Ctrl+Shift+R to override the frontend service worker which forced the request to be made over the network to the backend django server.

That shows the jpeg file I expected.

---

Updating profile info is not working

It seems like the profile data is not updating and no request is sent from the client side.

Change dependency to ExifTool

We should refactor all calls of ExifRead and refactor it to direct calls of ExifTool via pyexiftool. This will allow us to read all the file formats including video formats.

Only high confidence scenes should be saved

Places365 is used for scene classification. At the moment all scenes that the classifier found are added. It only should add high confidence scenes instead.

librephotos/api/models.py

Lines 224 to 239 in 6c5796b

	try:
	res_places365 = inference_places365(image_path)
	captions['places365'] = res_places365
	self.captions_json = captions
	if self.search_captions:
	self.search_captions = self.search_captions + ' , ' + \
	' , '.join(
	res_places365['attributes'] + res_places365['categories'] + [res_places365['environment']])
	else:
	self.search_captions = ' , '.join(
	res_places365['attributes'] + res_places365['categories'] +
	[res_places365['environment']])
	self.save()
	util.logger.info(
	'generated places365 captions for image %s.' % (image_path))

Download resnet models when building a docker image

At startup the backend of librephotos downloads a pytorch model. This model is 250MB big and is download at start every time. This of course is inefficient, because it is a dependency that doesn't change. It should be integrated in the docker steps and the windersnet.py should be rewritten to not download the models but load them from the model integrated from the docker steps.

librephotos/wideresnet.py

Lines 9 to 15 in 6c5796b

	model_urls = {
	'resnet18': 'https://download.pytorch.org/models/resnet18-5c106cde.pth',
	'resnet34': 'https://download.pytorch.org/models/resnet34-333f7ec4.pth',
	'resnet50': 'https://download.pytorch.org/models/resnet50-19c8e357.pth',
	'resnet101': 'https://download.pytorch.org/models/resnet101-5d3b4d8f.pth',
	'resnet152': 'https://download.pytorch.org/models/resnet152-b121ed2d.pth',
	}

Benchmark and change Computer vision pretrained model

Hi,
densecap VGG16 seems to be an old and outdated pretrained models. New models seem to be more efficient. Maybe we could test some new pretrained models and change it if it
worth it ? (like PNASNet-5-Large, SENet, VGG-19, efficientnet-lite4, Inception_V4 or Inception_ResNet_V2)

Memo help for downloading pretrained models https://github.com/Cadene/pretrained-models.pytorch/tree/master/pretrainedmodels/models

LongRunningJobs do not check if the rq worker is still working

If you destroy and rebuild the container, the scan will appear as frozen. In reality the rq worker just isn't working that job anymore. At startup the LongRunningJob should look up it's rq worker and if it isn't working, it should mark the LongRunningJob as stopped.

People album wrong photo and count

hi,

i have 459 photos scanned and 4 people with correct faces set.
the people-album shows me the following. all 4 images are false (like the car) and the photo count is wrong.

Support for Video

A missing feature is support for video files.

We would need to change the data model to add video.

librephotos/api/models.py

Line 107 in 289f413

class Photo(models.Model):

We would also need to change the scanning mechanism.

librephotos/api/directory_watcher.py

Line 110 in 289f413

def scan_photos(user, job_id):

Not only that, but we would need to implement #10.

Likewise, we have to implement the thumbnail creation function. We should probably use FFmpeg with a python wrapper for that. Do we want a single thumbnail or a sprite preview?
We need to implement the calls for face classification, scene detection and object detection. We also need to decide if we only analyze the thumbnail/sprite preview or the whole video file (needs GPU support)

Then we have to change and create the API calls to display videos together with pictures and allow to open a picture. Do we need to implement transcoding?

The last step would be to refactor the light box, so that it can display videos.

Train faces fails when no faces are labeled

hi,
when executing train faces it fails after a second. checking the docker-log, i`m getting the following lines when try to train faces.

/miniconda/lib/python3.8/site-packages/django/db/models/fields/__init__.py:1365: RuntimeWarning: DateTimeField LongRunningJob.finished_at received a naive datetime (2020-12-11 21:25:04.104340) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"

workaround: set USE_TZ = False in /code/ownphotos/settings.py

but i got the next error in ownphotos.log...

2020-12-11 21:57:42,083 : face_classify.py : train_faces : 112 : INFO : Before fitting
2020-12-11 21:57:42,084 : face_classify.py : train_faces : 148 : ERROR : Expected 2D array, got 1D array instead:
array=[].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

Unable to set custom scan directory

Hi,
First, a huge thanks you for saving this repo :)

I don't find a way to change the "local scan directory" of users :
log as admin, when I click on the edit icon it's just showing this message :

Thanks

Add the size of LibrePhotos and compatibility in ReadMe

Hi

Trying to install in my Pi... The install is still running (dev: Pulling from reallibrephotos/librephotos) and already more than 3-4gig of data has been downloaded (!!). Is this normal???
I am afraid that I run out of size...and did not expect that librephotos would take such amount of space.

Scan percentage not rounding correctly.

Should be either a xx% or xx.xx%

Manage photos

Hi, could it be possible to add some actions delete, move folder on photo.
it could be great if we could manage photo directly form librephotos.

And if i delete a photo on local data it will not be deleted from thumbnail, database.

tnaks

Choose a directory from below not working in /admin

I can't get any response from /api/dirtree/ when i am opening /admin... The request is always canceled!

Screenshot

Also the api/user/ request just sometimes succeeds.

My docker-compose.yml:

version: '2.1'
services:
proxy:
image: reallibrephotos/librephotos-proxy:dev
tty: true
container_name: librephotos-proxy
restart: always
links:
- "backend:backend"
- "frontend:frontend"
ports:
- "80:80"
librephotos-db:
image: postgres
container_name: librephotos-db
restart: always
environment:
# This db password is internal, you can change it if you want, but also change it in librephotos-backend container
- POSTGRES_USER=docker
- POSTGRES_PASSWORD=AaAa1234
- POSTGRES_DB=ownphotos
volumes:
- $HOME/librephotos_data:/var/lib/postgresql/data
command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
healthcheck:
test: ["CMD-SHELL", "pg_isready -d ownphotos -U docker"]
interval: 5s
timeout: 5s
retries: 5
frontend:
image: reallibrephotos/librephotos-frontend:dev
# For development uncomment the "build" line below and comment the "image" line above
# build: ../librephotos-frontend
container_name: librephotos-frontend
restart: always
tty: true
links:
- "backend:backend"
backend:
image: reallibrephotos/librephotos:dev
# For development uncomment the "build" line below and comment the "image" line above
# build: .
container_name: librephotos-backend
restart: always
volumes:
- F://Google_Drive:/data # CHANGE ME
- E://librephotos/storage/protected_media:/code/protected_media # CHANGE ME
- E://librephotos/storage/logs:/code/logs # CHANGE ME
- E://librephotos/storage/cache:/root/.cache # CHANGE ME
environment:
- SECRET_KEY=
# This is backend host from within the service, you dont need to change this
- BACKEND_HOST=backend
- ADMIN_EMAIL=
- ADMIN_USERNAME=meyerdominik
# Change your admin password!
- ADMIN_PASSWORD=
- DEBUG=false
- DB_BACKEND=postgresql
- DB_NAME=ownphotos
- DB_USER=docker
# This db password is internal, you can change it if you want, but also change it in librephotos-db container
- DB_PASS=AaAa1234
- DB_HOST=librephotos-db
- DB_PORT=5432
- REDIS_HOST=librephotos-redis
- REDIS_PORT=6379
# https://account.mapbox.com/auth/signup/
MAPBOX_API_KEY= # CHANGE ME - You need this for reverse photo geocoding
# https://docs.djangoproject.com/en/3.1/ref/settings/#std:setting-TIME_ZONE
- TIME_ZONE=Europe/Berlin
# The number of Gunicorn worker processes; SET to (2cpu cores); "NOT (2cpu threads)"
- WEB_CONCURRENCY=4 # CHANGEME
# Gunicorn worker timeout seconds (ensure that one bad request doesn't stall other requests forever)
- WORKER_TIMEOUT=1800
links:
- "librephotos-db:librephotos-db"
- "librephotos-redis:librephotos-redis"
# Wait for Postgres
depends_on:
librephotos-db:
condition: service_healthy
librephotos-redis:
image: redis
container_name: librephotos-redis
restart: always
volumes:
librephotos-data:
media:

Frontend expects valid similar images when opening photo view

This is not always the case because the image similarity index only gets build on startup and when it finished scanning. If you start with an empty database and start to scan 2000 images, than you can't while it still scan open pictures to look at them.

Publicly sharing pictures isn't working

If you publicy share a photo it is not giving you a full link only something like "/media/thumbnails_big/a9c11224d9c4885c3eb04192ce4ba2d31.jpg"

If you try to open the url with the added baseurl you get an error in the frontend part:

Just sharing the link as a picture works.

Improving on Timestamp Metadata

Issue from sim4city:

Hi,

After using Ownphotos for a week now, I really like the features of this project.

However, I noticed a bug in my library of photos that have no exif timestamp metadata and how Ownphotos tries to guess the date and time from each file name. Unfortunately, the guess is wrong in cases where the date and time are not in the file name. For example, the code would take a file with the name "20_server.jpg" + no exif metadata and generate a date of current year current month day 20 at 00:00:00.
Relevant code: https://github.com/zainlol/ownphotos/blob/d0f70a8427a60f9e77ae6cade804b4a5dc7056a8/api/models.py#L404-L417

Making notes as I go:
I would prefer the code not try to guess the exif date time from the file name when the file does not contain exif metadata.
Coming from a Google Photos Takeout export, the photos inside had all exif data stripped out and put into json files.
https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper
That worked for me to put the exif metadata back into the files.

In this section of the code: https://github.com/zainlol/ownphotos/blob/d0f70a8427a60f9e77ae6cade804b4a5dc7056a8/api/directory_watcher.py#L53-L114
Currently, when a photo already exists in the Ownphotos database, it is ignored with no further processing.
I would prefer in this case when the photo already exists, compare the photo.exif_timestamp that was saved, to what it is at the time of re-scan. If different than what it was in the past, overwrite to what it is now and save to the Ownphotos database.

Front in first start loading page.

Currently on startup the front end will display cached data. Instead need to show a loading till /api/rqavailable returns returns 200 checking every few seconds, and once good display front end.

Map is shifted

This is probably frontend bug

Map is loading shifted:

Marker is in upper left corner:

No photos before refreshing

This occur when I'm not accessing librephotos for a while (few minutes or so). Refreshing page helps.

Tested on chrome and firefox

Incorrect file extensions leads to crashes

From sim4city:

I noticed that some of my photos have incorrect file extensions that do not match their file header magic bytes for the typed file extension.
i.e. file named "test.png" but it is actually a gif file.

Please use this or something like this that inspects and verifies the file extension instead of relying on the file extension being truthful.
https://github.com/sindresorhus/file-type
or https://github.com/dimapaloskin/detect-file-type

Mimetypes alone are not enough: mimetypes guesses the mime type from the file's extension not by checking the file's header magic bytes (file type signature) to determine what the file actually is
i.e. file extensions are not a reliable way to determine mime type
for example, a user could have renamed a gif file to be a png file and this would think it's a png file.
Or a tiff file renamed ending with .png, etc.

https://github.com/dimapaloskin/detect-file-type

that does not guess the mime type from file name extension

instead, it looks at the first few bytes of the file and matches the file type signature that all jpg files have to be a valid jpg file

similarly, it does that for the rest of its out-of-the-box supported file types

https://github.com/dimapaloskin/detect-file-type/blob/master/signatures.json
does this make sense as to why file extensions are not a reliable way to determine mime type?
also, currently librephotos only scans for images with file types ending in .jpg, .jpeg and .png

in the future, I might want librephotos to scan for more media like videos, more types of photos, perhaps "360 photo-spheres", etc.
and at that point, the backend must be sure that the file really is what the file name makes the directory scanner worker think it is in processing.

Gunicorn worker limit increase

also, gunicorn was limited to 2 workers
in the ./entrypoint.sh
Set it to number of (cpu threads * 2) + 1
See
https://docs.gunicorn.org/en/stable/settings.html#workers
https://docs.gunicorn.org/en/stable/settings.html#threads
https://docs.gunicorn.org/en/stable/settings.html#worker-connections