qdraw/starsky

Accelerator to find and organize images driven by meta information. Browse and search images in your own cloud.

Starsky ·

List of Starsky Projects

• By App documentation database photo index & import index project
  • starsky web api application / interface
    • clientapp react front-end application
  • starskyImporterCli import command line interface
  • starskyGeoCli gpx sync and reverse 'geo tagging'
  • starskyWebHtmlCli publish web images to a content package
  • starskyWebFtpCli copy a content package to a ftp service
  • starskyAdminCli manage user accounts
  • starskySynchronizeCli check if disk changes are updated in the database
  • starskyThumbnailCli speed web performance by generating smaller images
  • Starsky Business Logic business logic libraries (.NET)
  • starskyTest mstest unit tests (for .NET)
• starsky-tools nodejs tools to add-on tasks
• Starsky Desktop Desktop Application
  • Download Desktop App Windows and Mac OS version
• Changelog Release notes and history

What is Starsky?

Check the introduction page to learn more about the scope of the application

Demo app

Starsky has a demo application online.

See the online demo

Is not needed to login, but you can create a new account

Project Readme

Install instructions for the server

This section deals with how to set up a Starsky system on-premises. You will find guides to all Starsky software for installation on-premises here.

General Project

The general application is Starsky solution (sln). You need to install the solution first.

Command line tools

The command tools to sync the database manually use Synchronize CLI to generate thumbnail use Thumbnail CLI. The Importer CLI can be used to copy files in a folder structure based on the creation datetime. The datetime structure can be configured.

To reverse geo code location information in images use the UI or the Geo CLI. Use your photo to track location and match this with your camera.With this tool you add a location trail (gpx) to a folder and match the datetime to images in the folder.

To publish files generate markup and images with a logo use the Web Html CLI. This publish web images to a content package. And when this is done you could copy a content package to a ftp service.

All these projects are separately compiled using the build script and using the same application settings (appsettings) configuration.

GitHub Issues

Pay attention

Please do not open an issue on GitHub, unless you have spotted an actual bug in Starsky.

Use GitHub Discussions to ask questions, bring up ideas, or other general items. Issues are not the place for questions, and will either be converted to a discussion or closed.

This policy is in place to avoid bugs being drowned out in a pile of sensible suggestions for future enhancements and calls for help from people who forget to check back if they get it and so on.

If a feature request is actually going to be built, it will get its own issue with the tag: Feature Request

Latest stable release

Latest prerelease

Build status

Azure pipeline

See ./pipelines/azure for details

Github Actions

Windows

Ubuntu

Docker

App on Windows and Mac OS

The pipeline below is to test the Desktop build process. The .NET part isn't build in this pipeline, but it's needed to run the Desktop app.

A full build including the .NET dependency

See the documentation for details

End2End tests on ubuntu github actions ci environment

See ./starsky-tools/end2end for details

Changelog and history of this project

There is a version log and backlog available on the history and changelog page

Codecov

Sonarqube

Licence

Authors

• @qdraw

starsky's Issues

iOS 13 removes exif on upload

Describe the bug
The default behaviour of Safari is to not expose Exif information when HEIC is selected in the Camera Settings

To Reproduce
Upload an image with HEIC option selected

Desktop (please complete the following information):

• OS: 11 or newer
• Browser Safari

Command line tools should also download Exiftool on start

Is your feature request related to a problem? Please describe.

Command line tools should also download Exiftool on start

StarskyApp should see map

Describe the bug
Should see map when using StarskyApp

To Reproduce
Steps to reproduce the behavior:

1. Go to DetailView
2. Click on location
3. Window opens

Expected behavior
Should see map

Screenshots

Desktop (please complete the following information):
StarskyApp 0.2.5 and newer only

Folder/Directory renaming in front-end

Is your feature request related to a problem? Please describe.
To rename folders
with api like this: starksy.corp/sync/rename

Describe the solution you'd like
To rename folders

Describe alternatives you've considered
Use OS and resync

Additional context
n/a

View folders realtime

Is your feature request related to a problem? Please describe.

To update the database when files are changed out-of-the-box supported

Describe the solution you'd like
To find out that folders are updated realtime

Describe alternatives you've considered
https://stackoverflow.com/questions/32494481/fast-way-to-check-if-a-large-set-of-file-pairs-are-identical

It's going to be a tradeoff between speed and reliability. You want to try the fastet method first, then go to something more precise. Here is the algorithm followed by fdupes:

compare file sizes
=> if different, then action (in your case, copy)
compare MD5 signatures
=> if different, copy
compare byte-by-byte
=> if different, copy
else do nothing

Preparing this answer, I just learned that fdupes now add an intermediate step with partial MD5:

http://en.wikipedia.org/wiki/Fdupes

Additional context
n/a

Folder cache issue new files added but detailview is a 404

Describe the bug
When you add a new file via the CLI, and already have visited the folder where the new image is added. Its not shown. When you search for that same image its shown in the results. but when go there the page is not found.

Expected behavior
It should refetch the item and update the folder

• Backend issue

Delete removes collection items

Describe the bug
When trashing items first in non-collection mode and then in the trash delete, both images are deleted

To Reproduce
Steps to reproduce the behavior:

1. Go to collections=false
2. Click on Trash Image
3. Go to Trash and press delete
4. See error

• Two images are deleted instead of one

Expected behavior
One image is deleted instead of two

Screenshots
n/a

Additional context
n/a

Swiping images on mobile devices

Is your feature request related to a problem? Please describe.
To go the next image is hard for new users.

Describe the solution you'd like
Swipe to go next. But allow users to zoom as well

Describe alternatives you've considered
Next icon

When press clean cache /info cache sould also be cleaned

Describe the bug
When press clean cache /info cache sould also be cleaned

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Notify other users when a file or folder is moved

Is your feature request related to a problem? Please describe.

Notify other users when a file or folder is moved

Describe the solution you'd like
Currently the UI isn't updated when a file or folder is moved

Describe alternatives you've considered
Manual pressing F5

Current location while viewing GPX file

Is your feature request related to a problem? Please describe.

To view the current location if you in the neighboorhood of trail

• needed: option to unlock the map

Describe the solution you'd like
to see your current location

Describe alternatives you've considered

• finding yourself by switching apps

Additional context

• use for example when hiking

Chrome 81+ Exif rotation on non-thumbnail images

Describe the bug

Looks like one of the oldest bugs in Chromium has been fixed. Chrome used to display images in landscape orientation even if they were taken in portrait on a phone.

https://stackoverflow.com/a/61196140
https://paul.kinlan.me/correct-image-orientation-for-images-chrome-81/
https://www.fxsitecompat.dev/en-CA/docs/2020/jpeg-images-are-now-rotated-by-default-according-to-exif-data/
https://developer.mozilla.org/en-US/docs/Web/CSS/image-orientation#Browser_compatibility

To Reproduce
When a image is not converted to a thumbnail

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
Chrome 81+

Additional context
Add any other context about the problem here.

Label fast copy does not save with titles

Describe the bug
It should copy all the labels and save them afterwards. instead there are entered in the field but not saved. when you go to the next images it still there, it should be gone then.

To Reproduce
see clip below:

Desktop (please complete the following information):

• at least in starskyapp (chromium 85)

Move multiple files

Is your feature request related to a problem? Please describe.
Move multiple files from archive view to a different folder

Describe the solution you'd like
To move files

Describe alternatives you've considered
Single file move

Additional context
Add any other context or screenshots about the feature request here.

AppSettings API

Values that are true are overwritten when summing new value

Rename files with structure settings

Is your feature request related to a problem? Please describe.

When files already imported. There is no nice way to rename them based on a creation date

• include xmp files
• include other sidecar files
• dng/raw and jpeg

Describe the solution you'd like

Describe alternatives you've considered
using other tools

Additional context

Geo location ui components on mobile Safari layout

Describe the bug

Geo location ui components on mobile Safari layout is missing

To Reproduce
Missing icon

Expected behavior
Clickable icon

Screenshots

Smartphone (please complete the following information):

• Device: iOS iPad
• OS: 13.6.1
• Browser Safari

Keyboard accelerator Command / Ctrl A

Is your feature request related to a problem? Please describe.
When pressing command/ctrl a it should select all files

Describe the solution you'd like
Easy access to feature by using keyboard shortcuts

Describe alternatives you've considered
More -> Select All

Additional context
n/a

starksyimporter asterisk does not always pick first item

Describe the bug

It should import to /yyyy/mm/yyyy_mm_dd_a instead to /yyyy/mm/yyyy_mm_dd_d

Move file to root folder

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
See short movie

20201106_move_to_root_bug.mp4.zip

• more move to root it does not exist at all
• move to other folder when pressing close its not listed

Expected behavior
should move file successfully

Screenshots
n/a

Desktop (please complete the following information):
Firefox 48

Publish with Metadata does not work

Describe the bug
Publish with Metadata does not work

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

StarskyApp Video playback

Starsky App Video playback not working

Importer fail improvements

Describe the bug
When you abort a import session before its ended the database isn’t updated

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Import time of videos is UTC instead of the local time

Describe the bug
Import time of videos is UTC instead of the local time. This is making it hard to find videos back

To Reproduce

• import a mp4 or quicktime video thats contains dateTime

Expected behavior
It should give the right time or easy to fix option

back-end issue
Browser context is not needed

StarskyApp - when chaning from location the watcher isn't updated

Describe the bug

When you change from location in the settings menu. Then press cmd +e to update a file, the file isn't uploaded

Expected behavior
It should upload the file

Additional context
Work around, close entire app and try again

Develop improvements release pipeline

Is your feature request related to a problem? Please describe.

For version 0.4.2 I forget to update the package Json and csproj files, but in the branch and tag name it's updated. This is bad because the client is told to update but this the latest version

Describe the solution you'd like
Add to the pipelines a check on tag/brache name for GitHub actions/ local git / ado to make sure the version is updated

Describe alternatives you've considered
Don't make this human error

Additional context
N/a

Files that already are deleted is not shown visualy

If a file is already deleted its not always shown

Inital state

{
   "fileIndexItem":{
      "fileName":"20200517_160810_d.jpg",
      "isDirectory":false,
      "keywords":[
         "Keyword"
      ],
      "tags":"Keyword",
      "status":"Ok",
      "collectionPaths":[
         "/2020/05/2020_05_17/20200517_160810_d.jpg"
      ]
   },
   "colorClassActiveList":[
   ],
   "pageType":"DetailView",
   "isDirectory":false,
   "subPath":"/2020/05/2020_05_17/20200517_160810_d.jpg",
   "isReadOnly":false
}

after pushing update

[
   {
      "filePath":"/2020/05/2020_05_17/20200517_160810_d.jpg",
      "keywords":[
         "Keyword",
         "!delete!"
      ],
      "tags":"Keyword, !delete!",
      "status":"Deleted",
      "collectionPaths":[
         "/2020/05/2020_05_17/20200517_160810_d.jpg"
      ]
   }
]

In Application background polling sync-scanning

Is your feature request related to a problem? Please describe.

Background sync of the application within the main web process. Not manual but every 10 minutes or something like that.

Describe the solution you'd like
Setting to enable background setting

Describe alternatives you've considered
crontab -e starskysync*

Show activity while remove from trash

Is your feature request related to a problem? Please describe.

Show activity while remove from trash

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Redirect bug

Describe the bug

.Net/starsky/Search?t=-Datetime%3E2%20-ImageFormat-tiff
To
.net/search?t=-Datetime%3E2%20-ImageFormat-tiff&p=0

And should have the prefix /starsky/

Download folders with + not found

Describe the bug
Download folders with + not found

To Reproduce
Steps to reproduce the behavior:

1. Go to Photo with + in name
2. Click on Download
3. See error

Expected behavior
Should download

Web Publisher - first image as other thumbnail format

Is your feature request related to a problem? Please describe.
When you publish an selection, for example social media there is a 1 image thumbnail needed

Describe the solution you'd like
Option to generate a thumbnail for the first time

Describe alternatives you've considered
n/a

Additional context
Add any other context or screenshots about the feature request here.

Fix chmod on startup *nix when downloaded ExifTool can not be excuted

Describe the bug

When a user is new with starsky Exiftool is downloaded. there is done a chmod +x for the exiftool but when the rights are not enough this can give issues in the background

To Reproduce
Give the temp folder not enough rights. This only happens when there not enough rights on the temp folder or ./exitfool
[UPDATE] This behavior is produced by a deploy script

Expected behavior
It should work or warn a user that there is action needed

CSP Legacy websocket issue

Describe the bug

In a very old version of Firefox 48 when websockets are enabled

Also for safari 13.1.2 on iPad OS 13.7

Search details show exact query behind modal

Is your feature request related to a problem? Please describe.

(Added) Frond-end Search details show exact query behind modal

Describe the solution you'd like
A modal to view how the search query is build up

Describe alternatives you've considered
Looking into debugger

Additional context

Importer disposed object

Describe the bug
Importing files with starsky mail

 at starsky.feature.import.Services.Import.Importer(ImportIndexItem importIndexItem, ImportSettingsModel importSettings) in D:\a\1\s\starsky\starsky.feature.import\Services\Import.cs:line 403
0|starsky  |    at starsky.feature.import.Services.Import.<>c__DisplayClass19_0.<<Importer>b__0>d.MoveNext() in D:\a\1\s\starsky\starsky.feature.import\Services\Import.cs:line 366

Expected behavior
No duplicate files

Screenshots

Server

Additional context
Add any other context about the problem here.

Check if the latest version

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Export/Download Zipper does not export directory structure

Is your feature request related to a problem? Please describe.
When you press Export the content is exported as flat zip

Describe the solution you'd like
To have folders inside the zip that represents the structure of the original content
This behaviour is supported in publish

XMP Creating issues on some files

Describe the bug

Not on all files, but on some files there are xmp files created that

��Exiv2����Exif��MM�*�����������������J�����������R�(�����������������������%���������Z�������H�������H��������������������������N�����������������������E����������������������������������������������2�������5������	

20200820_094441_DSC02371.arw

To Reproduce
Not clear

Expected behavior
Normal xml valid file

Screenshots

Backend

Layout '40' in datetime

Describe the bug

it should fit in the box

To Reproduce
Steps to reproduce the behavior:

1. Go to DetailView
2. Click on Change Datetime
3. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Websocket connection makes entire application crash

Describe the bug
When websockets disconnect, the popup shows and later the entire react application crashes

Expected behavior
it should work

Screenshots
At least in Firefox / Safari

Import backup (what todo with structure)

Is your feature request related to a problem? Please describe.
When importing first backup the content in a different folder

Describe alternatives you've considered
running importer two times 1. without index and 2. with index enabled

Additional context
n/a

Ignore pattern for sync

Is your feature request related to a problem? Please describe.
To have control to ignore specific folders to scan with the sync

Describe the solution you'd like
Config to ignore folders to be scanned

Describe alternatives you've considered
Work-around is to use params starskysynccli -s /2020

Additional context
Make a global scan option easier to use

DateTime shifting for multiple items

Is your feature request related to a problem? Please describe.
When the datetime is set wrong its very difficult to fix this. There are exiftool commands needed

Describe the solution you'd like
A clear UI to fix this issue

Describe alternatives you've considered
To do it with command line tools

Additional context
n/a

DbUpdateConcurrencyException not handled

Describe the bug

Start indexing /2020/10/2020_10_24
.
/2020/10/2020_10_24  Unhandled exception. Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ThrowAggregateUpdateConcurrencyException(Int32 commandIndex, Int32 expectedRowsAffected, Int32 rowsAffected)
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ConsumeResultSetWithoutPropagation(Int32 commandIndex, RelationalDataReader reader)
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.Consume(RelationalDataReader reader)
   at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(DbContext _, Boolean acceptAllChangesOnSuccess)
   at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges()
   at starsky.foundation.database.Query.Query.RemoveItem(FileIndexItem updateStatusContent)
   at starskycore.Services.SyncService.CheckMd5Hash(IEnumerable`1 localSubFolderDbStyle, IReadOnlyCollection`1 databaseFileList)
   at starskycore.Services.SyncService.SyncFiles(String subPath, Boolean recursive)
   at starskycore.Services.SyncServiceCli.Sync(String[] args, ISync syncService, AppSettings appSettings, IConsole console, IThumbnailCleaner thumbnailCleaner, ISelectorStorage selectorStorage)
   at starskysynccli.Program.Main(String[] args)
Aborted

Expected behavior
Directory is scanned

Publisher improvements

Is your feature request related to a problem? Please describe.

• when pressing create 404 are. this should be a 206
• Command Line interface allows only the first item
• Allow from search results #298
• Make component stateless so the app can work behind a load balancer

Zoom in detailView

Is your feature request related to a problem? Please describe.
When viewing in DetailView mode

Describe the solution you'd like
Option to zoom in a picture

Describe alternatives you've considered
open the source image

Additional context
Add any other context or screenshots about the feature request here.

FileIndexItemJsonParser is incomplete

Describe the bug

Incomplete feature to handle
Sometime referenced as .meta.json or .starsky.filename.jpg.json

Expected behavior
Make feature complete

When remove a folder, the files within the folder are still in the database

Describe the bug

When remove a folder, the files within the folder are still in the database

To Reproduce
Steps to reproduce the behavior:

1. Go to Select
2. Click on a folder
3. Click on More > Move to Trash
4. Click on Trash and remove folder
   And the database still has those files

Expected behavior
Folder and child items are deleted from the database

Screenshots
N/a

Copy all fields with one command in detailview

Is your feature request related to a problem? Please describe.
Copy all fields with one command in detailview