whyboris/video-hub-app

Official repository for Video Hub App

Video Hub App 3

Video Hub App 3 is the fastest way to browse and search for videos on your computer. Think of it like YouTube for videos on your computer: browse, search, and preview. Works on Windows, Mac, and Linux!

Download Now

This software is available for $5.00 through videohubapp.com

$3.50 of every sale goes to the cost-effective charity Against Malaria Foundation.

About

Video Hub App was created by Boris Yakubchik. It uses the frameworks Angular and Electron.

License

This software was built on top of angular-electron by Maxime GRIS. It carries an MIT license (see the LICENSE file). While the license is permissive, I ask that you do not distribute free copies of this software unless you have significantly changed it.

Contributing

I would love to see the improvements you make to this app and am happy to accept pull requests. You can reach out if you'd like to coordinate / collaborate, or just jump to issues to see what's already getting worked on and to add new suggestions!

Please consider improving any of the translations, or add a new translation!

Please consider improving or adding an icon to the app. It's a simple process, just follow the instructions!

Upcoming features

See issues for what is in progress.

Development

⚠ The repository is usually ahead of the publicly released version - it is 🚧 WIP and may have bugs in it.

How to start:

• npm install to install (you may need to run npm install --legacy-peer-deps if install fails)
• npm start to develop
• npm run electron to build

Main dependencies in use:

Library	Version	Date	Comment
Angular	v17	Dec 2023
Angular-CLI	v17	Dec 2023
Electron	v27	Dec 2023	(internally uses Node v18.17.1 and Chromium 118)
Electron Builder	v24	Dec 2023

• Node: It may be best to use the same version as Electron uses internally (v18.17.1)

• Angular CLI: not required but may be useful: Angular CLI.

• Mac: We now have M1 and Intel builds, but you may need to manually update the FFmpeg and FFprobe executables if you're building for a different architecture than you are on.

To help debug a production build of VHA you can use Debugtron

Remote

👩‍🚀 a new feature in VHA is the option for the user to turn on a server after the app starts. This will let the user open a simpler version of the VHA user interface on their phone or tablet (if both PC and device are on the same WiFi) and use it as remote control to play videos 🚀

For details, see instructions.

Thank you

This software would not be possible without the tremendous work by other people:

• Angular
• Electron
• FFmpeg
• angular-electron
• ngx-virtual-scroller
• ffmpeg-static
• @ffprobe-installer/ffprobe
• ngx-translate
• fdir
• async
• chokidar
• trash
• Fuse.js
• electron-window-state

This software uses libraries from the FFmpeg project under the LGPLv2.1 with binaries from here

Since becoming open source, this software was made better with the awesome contributions by cal2195

video-hub-app's Issues

Autodetect User Language

It looks like we can do something like this:
translate.use(browserLang.match(/en|ru/) ? browserLang : 'en');
But I'm unsure how Electron handles browser language.
It might be worth adding a modal if the app detects it's the first start that has a language selection dropdown.

Thumbnail Generation Issues

Some files are still having problems generating thumbnails and/or video previews! 👎

This issue is to track down the (likely multiple) issues!

Problematic files:

1. .wmv files - Some of these files have their 0th stream be a mjpeg with 90k fps??
   We should be using the 2nd stream, wmv. Will have to see if it's all wmv, or just some badly encoded ones...

I will add more details as I deal with each problematic file!

Remember last window size after app closed

Problem:
Currently app opens to full screen every time.

Solution:
Have the app remember the last window size.
Optionally add a setting to the menu.

Progress bar in Windows taskbar / MacOS dock

This is super simple and would be a nice polishing touch:

progress bar in the Windows taskbar / MacOS dock

Implementing it is about 1 line of code!

https://electronjs.org/docs/tutorial/progress-bar

Horizontal filmstrip bug tracking

The last PR #44 resulted in minor bugs when extracting videos, this PR does not fix them.

Thumbnails don't show up if:

• the vha file is in the same directory as source files
• the vha file has a space in its name
• the screenshot size is 50px (and likely other sizes) - due to rounding error.
• Sometimes a slice of the next frame still appears in the thumbnail view

• The last frame is still often black in the film strip.
• Zooming out after scrolling to the end of the filmstrip prevents you from scrolling back when zoomed out!

This issue is meant to track various problems of this sort.

Implement Backwards Compatibility

With all the new changes in v1.4.0, it would be a very good idea to implement backwards compatibility. This issue is to track anything related to it.

It would be nice to make it easy to implement more changes in the future, and also keep the code separate from the main app. Eg. if you open a v1.3.0 file in v1.5.0, we first convert from v1.3.0->v1.4.0, then v1.4.0->v1.5.0.

This will need #9.

It doesn't need to be fully backwards compatible, we could regenerate thumbnails, but keeping as much user entered data as possible is good, eg. input folders, added/removed tags, etc.

The biggest thing is to allow people to open old files in new versions, and they should just work, even if things need to import again.

Filter by video length

Just as you can filter videos by video resolution, it would be nice to be able to filter by video length as well.

Both an upper and lower bound would be good! 👍

Optionally include folder names in file tagging algorithm

This can be useful if a user has already organised videos by folders, eg. movie genre.

Adding the file path (minus any path shared by all videos) into the tag generation would improve tagging accuracy.

New library won't show videos until reload

Just noticed, that new libraries created with the wizard import the videos, but doesn't display anything until you reload the library or app! 😖

Not sure the cause yet.. could be the new sorting options?

Add versioning to the vha file

Without a version field in the vha file, it's hard to work out what type of vha you are opening, and how to convert it to a new version.

We should add a version field, so that we can add backward compatibility properly. (We can assume no version means <=v1.3.0).

Organise thumbnails by prefix folders

With 10,000 files, 30,000 thumbnail/preview files will be generated. Currently, these are all stored in a flat structure in the vha- folder.

This makes it very hard to list the folder in your file manager with so many files! An improvement would be to sort these previews by prefix folders:

We could have 16 folders (0-F), or 256 (00-FF), and store each image by the first letters of the hash. Eg. a hash of 46afb78c.jpg would be stored in the folder vha-test/46/46afb78c.jpg

Use fs.Dirent objects, and extract metadata in the background

Currently, the app freeze while performing the first phase of the import (listing all the files recursively).

A large portion of this delay in phase 1 is due to stating all the files to see if they are directories. Alternatively using fs.Dirent objects, the following code would be much better:

// Recursively walk through a directory compiling ImageElements
  const walkSync = (dir, filelist) => {
    const files = fs.readdirSync(dir, {encoding: 'utf8', withFileTypes: true});

    files.forEach(function (file) {
      if (file.name) {
        if (!fileSystemReserved(file.name)) {
          // if the item is a _DIRECTORY_
          if (file.isDirectory()) {
            filelist = walkSync(path.join(dir, file.name), filelist);
          } else {
            const extension = file.name.split('.').pop();
            if (acceptableFiles.includes(extension.toLowerCase())) {
              // before adding, remove the redundant prefix: sourceFolderPath
              const partialPath = dir.replace(sourceFolderPath, '');
              // fil finalArray with 3 correct and 5 dummy pieces of data
              finalArray[elementIndex] = [partialPath, file.name, cleanUpFileName(file.name), '', 0, '', 0, 0];
              elementIndex++;
            }
          }
        }
      }
    });

See https://nodejs.org/api/fs.html#fs_fs_readdirsync_path_options

This will be a drastic improvement in speed, especially over networked file systems. Unfortunately, this is only added in node v10.10.0, which will need to be updated first.

Additionally, it would be much better if the second phase (metadata extraction) was performed in the background like phase 3 (thumbnail generation). This is especially important with the addition of file hashes, which will require reading every file.

Boris - what I'm working on

This is a small experiment to see if it's useful to have a public-facing "what I'm working on" on this list.

This is a place to be more transparent about what tasks I'm thinking of working on next. It's also a discussion - so please let me know if something should be a higher priority. I'm also happy to let anyone take over any task if it's of interest to them 🙇

My aim is to always do all the code review before starting / resuming on my own coding.

Currently on my todo list in priority order (more will be added as I complete these):

• Details view - where tags (and more-detailed information about the video) will live #100
• Adding & filtering tags once the Details view is merged to master #53
• Adding other fields (number of times viewed, star rating, date added) and associated filters
• Sorting by above-added filters #86 #80

Sorting Options!

This issue is to track the various sorting options we'd like to add! 💯

We'd like to support sorting by:

• Date Added (#80)
• Date Created (From File)
• Number of Times Played
• Star Rating
• File Size
• Duration
• File Name

We'll probably also want to be able to filter by each of these too, but once the metrics are added (date added, times played etc.) for each file, it should be easy to support! 👍

Split rescan into quick and full rescan

In order to improve the speed of adding new files to a large library, as well as supporting rebuilding the entire library if files have changed since adding, we should split the rescan function into two modes:

• Quick: Just scan all files, and add any new file paths. Don't stat any files for sizes except new ones.
• Full: Completely rebuild the library and rescan all metadata, but keep existing thumbnails. Changed files will have a new hash, so will get new thumbnails.

This will allow for both adding new files very quickly, and fixing out of date libraries.

Prompt for root folder when saved root folder doesn't exist

If your videohub is on a SMB share the app saves the path in windows format. On mac and linux this leads to the problem, that if you click on a video in the app, the video will not open in your default video player.

The solution would be to save the filepaths in the .vha file in a different format, depending on the operating system of the client. On Unix systems this will be the mounting point (mac: /Volumes, Linux: /Media/username/...)
Example:
For me this was "\\{name_of_network_share}\Plex_videos\video_folder” which works on windows.
I tried it on mac and changed that path manually to the correct mounting point on my system.
In this case “/Volumes/Plex_videos/video_folder”. Now when I click a video, it opens the video in VLC as intended.

Snap zoom levels to use full space available

@whyboris
Wanted to get your opinion on this! 👍

I don't really see a point in zooming arbitrary levels (+-25px I think atm), since most of those won't use the full space of the viewport.

I would like the zoom to change the number of files displayed in one row, filling the entire row - what do you think of this?

Implement Mac Touch Bar Support!

Electron supports interacting with the MacBook touchbar!
https://electronjs.org/blog/touch-bar-support

This would be a very neat feature! We could have dynamic buttons for tags, changing views, zooming etc.
It seems very easy to implement! 👍

Autocomplete tags when searching

It would be a very neat feature to auto complete tags when searching from the search fields! 👍

I'm not sure how difficult this would be to implement in the current setup, but it would be worth thinking about with #53! 👍

Extract Thumbnails on Startup

It would be nice to have the option to start extracting missing thumbnails when you open the app! 😄

Remove tags already selected from tag cloud

It's pointless to display tags already selected in the tag cloud, as clicking on these do nothing.

It also pollutes the tag clouds, as selected tags always appear first as they are present in all videos filtered by those tags.

Auto play 10 screenshots on hover option

Problem:
Some users prefer to have the thumbnails view cycle through the screenshots without having to move the mouse

Solution:
Add a settings toggle that will allow the thumbnails to cycle without having to move the mouse.
If I remember correctly, the code for the cycling is already there, but disabled.

Add Support For More Thumbnail Import Settings

As well as thumbnail size, it would be nice to support more properties such as:

• JPEG and Video Compression
• Number of thumbnails to extract
• Number and length of video clips to extract
• Disabling thumbnail or video clip generation
• Option to enable extracting thumbnails on startup #15

Customise everything! 👍

v1.3.0 vha file corruption

I've actually just found that my v1.3.0 vha file has become corrupted some how - the image id's are wrong for quite a few files, resulting in incorrect thumbnails and files matched to file names. I am not sure why this has happened, although I did do a lot of rescanning with the same folder and using different devices.

Possibly has to do with saving the last thumbnail id, and reading it back in, or when saving out the vha file again. More investigation is needed, but this may impact #10.

Last steps before v2.0.0 release

This issue is to keep a list of tasks that should be done after all the new features we'd like to see in version 2.0.0 are added, but before the release occurs. Most of these tasks are on top of the issues tagged for Milestone v2.0.0

This is a good place to see just how much needs to be done to make the current master release-ready. In no particular order:

Visual

• Reorganize the settings menu & tabs to be more coherent
• Icon overhaul #51
• Confirm settings defaults - make the first app launch only display relevant buttons
• Clean up the Miscellaneous view settings - group and describe buttons better
• CSS consolidation: pull colors and some re-used elements from a central file
• reevaluate colors - maybe better contrast for a11y (accessibility) and harmonize all the filter and tag colors
• CSS consolidation: some components use the same CSS, so, reuse those to keep DRY
• confirm animations look good; add animations on newly added features

Dark Mode

• Move the dark mode button to just-before-last button in the toolbar
• Confirm app responds to OS X Dark Mode settings / changes
• Check the dark mode is working on all components; possibly simplify the CSS

Code

• Streamline the view switching code - how element widths / heights are recomputed & how ng-scroll receives the expected heights / number of elements (we now have 6 views, used to be 3)
• Remember each view's zoom level #16
• Consolidate the view-switching bug (some views currently don't show)
• Check that all views handle window resizes and search-bar hiding
• Make sure that adding a rating, scrolling away, and scrolling back persists the rating in the view
• Remove most console.log statements
• Add new keyboard shortcuts (for new views) & update the list in settings
• Remove jankiness from the resolution, time, and rating filters

Misc

• Update to latest Angular (v8)
• Stress test the app with more than 1,000 videos in a hub

⚠️ Note -- unfinished tasks moved to #234

Optional in-app video player!

Although one of VHA's strengths is "do one thing, and do it well", leaving you to choose your own video player, it would be nice to have an optional in-app video player.

While it wouldn't be able to play everything, as HTML5 only supports certain codecs in Chromium, I think it would be able to play the most common formats at least. We could fallback to the normal player if it doesn't support it.

One major advantage to doing this would be we could implement a "filmstrip timeline" by reusing the film strip, and allowing you to click on it to jump to that timestamp in the video!

I'd appreciate your thoughts on this, and if you think it's a good idea, any implementation ideas too - new window vs. tabbed, etc. 😄

Add Mojave Dark Mode Support

Electron v4.0.0 now has APIs to support Mojave's Dark Mode! 🎉

electron/electron#13387

We should support this! 💯

Support Multiple Headings in Settings Tabs

As far as I can see, right now you can only have a single heading in the settings tabs.

It would be nice, for example to have:

[Main Settings]
Library Settings
[] Quick Rescan
[] Full Rescan
[] Verify Thumbnails
App Settings
[] Start Wizard Again
[] Reset All Settings

Maybe this is possible already, but I couldn't see how to do it easily! 😄

Initial file count should also ignore vha thumbnail folders!

Similar to #3 and #24, the initial file count should ignore vha-* folders!

Shift-click on a tag to add it to excluded tags

It would be very nice to be able to shift-click on a tag in the tag cloud, and it would be added to the excluded tags rather than included tags filter! 👍

Library .vha structure and format

The current .vha file structure is a json of the following:

export interface FinalObject {
  hubName: string;      // the name of the hub -- for recently-opened
  ssSize: number;       // screen shot size -- so when you reimport it remembers your preference
  numOfFolders: number; // number of folders
  inputDir: string;     // later may support array of many input directories
  addTags?: string[];    // tags to add
  removeTags?: string[]; // tags to remove
  images: ImageElement[]; // see below
}

export type ImageElement = [string, string, string, string, number, string, number, number];
/*                            0       1       2       3       4       5       6       7
-------------------------------------------------------------------------------------------------------
index   type      description                   more info
-----   ------    -------------------------------------------------------------------------------------
  0     string    partial path to file,    <--- for opening the file, just prepend the `inputDir`
  1     string    full original file name, <--- for opening the file
  2     string    file name cleaned
                  of dots, underscores,
                  and file extension       <--- for searching
  3     string    hash of the file         <--- used for detecting changed files and as a screenshot
                                                identifier
  4     number    duration                 <-- duration of film as number
  5     string    depicting size           <-- e.g. `720`, `1080`, `SD`, `HD`
  6     number    width of screenshot      <-- e.g 124 etc
  7     number    fileSize in megabytes    <-- rounded to nearest MB

*/

Why are ImageElements arrays, rather than objects, so we could do ImageElement.duration?

If it's to save space in the saved file, we could compress the file (which would work very well due to the duplicated values across files eg. object names, input folder, resolution, width) at the cost of slightly less easy manual editing of the file?

Alternatively, using constants like PATH_TO_FILE=0, FILENAME=1 so we can easily index the array would be nice!

Let me know if I'm misunderstanding something! 😄

Variable number of screenshots - depending on video length

Pretty simple change: allow users to select a rate of screenshot extraction (1 / minutue, 1 / 5 minutes, etc).

I'm happy to code it up 👍

File Hashes

So far, I've come up with three variants of file hashing:

1. Hash the relative file path.
   eg. md5('./path/to/file.mp4')
   Pros: Very fast. Can generate hash using just a dir list and no stats. Quick to find new files.
   Cons: Existing files that change won't be detected. Moved or renamed files will be treated as new files.

2. Hash the file name and file size.
   eg. md5(file.name + file.size)
   Pros: Average speed. Needs one stat per file for the file.size. Files can be moved and the thumbnails will be reused. Changed files will be detected as long as their file size changes.
   Cons: Requires an extra stat per file. Renamed files will be treated as new files.

3. Hash part of the file and file size.
   eg. md5(file.beginning + file.middle + file.end + file.size)
   Pros: Any change to a file should be detected. Files can be moved and renamed without regenerating thumbnails. Hash purely depends on contents of file.
   Cons: Requires an extra stat and 3 reads of the file to generate the hash. You have to choose between quickly adding new files just by new file paths, or rereading all files again for a true rescan.

I've implemented option 3 in #2 but I'm wondering whether it's the correct decision for when you have 1000+ files over a networked filesystem, especially just to add a new file. Thoughts?

Files with identical hashes open wrong file

Minor issue with the new hash, should be easily fixed!

When you have duplicates of the same file, and they have the same hash, clicking on one to open it will opened the "last" version of the file! This is because openVideo(hash) takes a hash value, so it picks the last one with the matching hash, as they all share the same one!

To fix this, openVideo() should take the video path instead! 👍

Performance Issue: Output Resolutions not multiples of 16

As seen here, for optimal scaler performance, the output resolutions should be multiples of 16.

We should really stick to 16:9 ratios like these:
240p, 360p, 480p.

This will also help with the encoding process! I've noticed a large slowdown since scaling was introduced!

Cmd+Q on Mac doesn't trigger vha file resave

If you use cmd+q on Mac to quit the app, the vha file doesn't get resaved if required. (Eg. a file renamed) 😢

The correct behaviour occurs when you quit using the x in the window title bar. ❌

Add Debug Flag for Verbose Output!

When testing, it's important to get as much output about ffmpeg and the like as possible - so if something goes wrong, we can trace it to the source as quickly as possible!

There should be a flag to enable debug output for this purpose! 👍

Sorting by recently added

One nice feature would be to be able to sort by most recent added! 🎞

I think we'd need to add a "datetime-added" field into the vha file for this? 👍

Remember Each View's Zoom Level

It would be a very nice addition to remember the last zoom level for each view, and they often look better with different zoom levels! 🔍

Manual Tagging!

This issue is to discuss and implement the much requested feature - manual tags! 💯

Some implementation ideas I've had:

1. We can add a tags: attribute to the ImageElement object to save the tags with each file.

2. Special tags can be used for "Starred" videos etc. They can piggy back off this with a tag of starred.

Ideally it should be as easy as possible to add many tags, to many videos!

Feel free to add additional ideas/info below! 👍

Auto update/notification

It would be nice to have to option to see if there are any new updates available, possibly on launch.

Making it as easy as possible for people who have purchased the app to update would be great! 👍

Icon overhaul

Problem: currently we're reusing several icons as we've run out of available ones in the font file. We'll be adding more settings in the future, and it's great not to have to rebuild the icon font every time a new icon is added.

Solution: start using svg files directly (instead of a custom font that needs to be generated from the pre-made svg files).

I'll be happy to take on the task - closer to when we need to release v2.0 👍

App needlessly scans for initial video list twice

When you first select a video directory, the app scans for all the videos, but just counts them and throws the list away.

Then when you click import, it scans them all again.

This could be improved by only scanning once and saving the result for the import!

First installation welcome & walkthrough

I'd like to have a welcome screen and a walkthrough for people installing the app for the first time 👍

I'll work on it before release, after we add more features and abolish bugs 😅

Use relative input folder when input folder and vha location are the same

When both the input folder and vha save location are the same, we should use relative directories to allow the library to be moved easily.

This will also allow the library to be opened on multiple systems when mounted at different locations.

Future proofing v2.0.0

Some things I'd like changed for v2.0.0, so that we don't have to make breaking changes again:

• Add versioning to vha file (#9)
• Decide whether to use zip compression (#47)
• Store actual width/height of videos, rather than quality string
• Reconsider using fast subfile hash for better hashing

The purpose of these changes is to make it as easy as possible to add features in the future, so that the most a future user would have to do is press a button to regenerate file metadata! 👍 This should reduce the need for any backwards compatible code (which is very hard to maintain), so if we add a new feature that needs a new field, we can just display a message if the metadata hasn't been generated for the current library yet, and offer to regenerate it for them!

Feel free to add anything else! 💡

Corrupt/incomplete video files stall thumbnail extraction

I've noticed with corrupt and incomplete video files, generally when the duration of the file doesn't match the real duration of the file, the thumbnail extraction stalls for a very long amount of time as ffmpeg tries to complete the extraction! This seems to be due to the filters expecting data from the inputs at each time code, and if any of these don't have data, it stalls waiting for it. There does appear a timeout, but it's incredibly long. 👎

To counter this, we can first check each of the timecodes individually, and check for actual data! 👍 This works way quicker, and will fail immediately if the data is missing!

If we find data missing, we have three options:

1. Skip all thumbnail generation for this file. eg. 0 screenshots.
2. Generate as many thumbnails as we can until the missing data, and then update the numOfScreenshots for that file. eg. 15/20 screenshots.
3. Change the duration of the file to be until the first missing data, and generate the full numOfScreenshots for the file. eg. 20 screenshots.

Thoughts? 💡

Remove fluent-ffmpeg dependency

As we're only using fluent-ffmpeg now for ffprobe, we should consider removing it and calling ffprobe directly and just parsing the json results! :)

Optionally ignore folders beginning with '.' and '_'

As requested by users! 👍

Also, with the new video previews, we need to also ignore vha-* folders to prevent importing the video preview files themselves!

Background Initial File Scan and Metadata Import

To be truly fluid, the app should scan for files and import them fully in the background, updating the UI as files are imported, just like the thumbnail extraction! This would also allow you to resume importing files if you close the app and reopen it.

This would be a lovely feature, but would probably need significant changes, particularly around updating the file list to display, tags, and metadata displayed by each item.

@whyboris Do you have any thoughts on this?