dancooper/ember-mm-newscraper

New Ember Media Manager - The all in one media manager for Kodi and other media centers

Ember Media Manager

We decided that was time to give Ember a new home. We've taken it upon ourselves not only to pick up the code where it was left off but to attempt to continue its development.

If you found our work useful feel free to donate us a beer!

Goals

To continue development of EmberMM, because its a great product, that in my opinion is the most stable and useful media manager available, I've tried all others, but yet still come back to Ember.

Links

• Main discussion : http://forum.xbmc.org/forumdisplay.php?fid=195
• GitHub : https://github.com/DanCooper/Ember-MM-Newscraper (DanCooper is mainaining the most aligned version)

Helping the development

Any help is more than welcome. We do suggest everyone to participate in the forum to be aligned and updated.

As the codebase is managed by several people we tried to make it easier to maintain and review. We ask everyone to try to adhere to some simple guidelines as much as you can:

• keep it simple, if complexity is needed add a comment to explain why
• avoid duplication of code, if mandatory or needed please comment
• read all the code before changing it, avoid duplication of almost identical functionalities/classes/data. In case of doubt, please ask

(We know everyone knows and agrees on them but the more we work on the code the more we discover how those simple principles has not been applied even from us... )

We made a major effort in reviewing the core of Ember Media Manager, the scraping process and part, to bring it to the next level. Here are major points to consider:

• IMDB id is the unique identifier for movies.
• It is INTENTIONAL to separate the scrapers in three groups (Data, Poster, Trailer). We decided that the small overhead of code in the modules manager and some duplication of code was a far minor issue than the complexity (or mess) that had evolved in the multipurpose scrapers, making it complex to fix and almost impossible to add new ones quickly enough.
• Data scrapers will be executed one after the other and will fill ONLY selected & empty fields if not locked from global properites
• Each Data scraper will have the search dialog (is a known and accepted code duplication) because there are TOO many differences between IMDB, TMDB and other so having only one dialog in main would lead to a mess.
• Image scrapers will work in parallel and will return a list of images. The image selection dialog will merge all lists and show them. The dialog will be moved at main program level as is useless to have it replicated in the scrapers
• Order in Image scrapers will only be used for automated scraping where only the first one will be invoked (to be quicker)
• All the file save-handling logic with the names etc... will be put at main program level and will happen only once.
• All image Handling (load-save-fromWEb, etc) MUST be in only in the Images class and must use the memorystream as source (already almost there in 1.3.0.12)
• Trailers should behave as images

Contact

Please use the forum as main contact point.

ember-mm-newscraper's Issues

Feature request: Google Image Search as image scraper

I'd like to have Google Image Search as additional Image Scraper.

Image scraper ignores preferred language and uses all

The setting Movies > 'Scrapers - Images' > 'Preferred Language' is not applied with either automatic scraping or manual selection.
Forced language can be set to english and still images tagged in different languages appear. 'Only get images for...' below has no effect either.
Example:
Tried on Star Wars: Ep2, for Fanart with 'Any' preferred size which returns a 2160p image with spanish tag as first result and therefore chooses this via auto scrape. The same image is shown in several other languages - only 'forced language: english' is selected, though.
The media itself contains only english and german streams.

Tag Manager does not work

crash if the Tag Manager is used

Unhandled Exception when "Changing Series"

Der Einfachheit in deutsch:

Ich hab einen Ordner "Vikings (2013) [tt2306299]" mit entsprechenden Seasons-Unterordnern und Dateien. Via "Datenbank aktualisieren" wird unter "TV Shows" "Vikings" aufgelistet. Alles gut. Dann klicke ich rechts drauf und gehe wie gewohnt auf "Serie ändern", um dann das Scraping zu beginnen. Direkt wird folgende unbehandelte Ausnahme geworfen:

Ausnahme:

************** Ausnahmetext **************
System.FormatException: Die Zeichenfolge wurde nicht als gültiges DateTime erkannt.
   bei TVDB.Model.Series.Deserialize(XmlNode node) in C:\Users\steph\Documents\GitHub\TheTVDBApi\src\TVDB\Model\Series.cs:Zeile 896.
   bei TVDB.Web.WebInterface.<GetSeriesByName>d__16.MoveNext() in C:\Users\steph\Documents\GitHub\TheTVDBApi\src\TVDB\Web\WebInterface.cs:Zeile 360.

Den Nutzer "steph" gibt es hier überhaupt nicht. Es scheint sich wohl um eine Codeleiche zu handeln - wo dieser Pfad hart drinsteckt.

Workaround:
TvShow.nfo mit den IDs von TVDB, IMDB und TMDB manuell anlegen, "Vikings"-Eintrag rauskicken und dann via "Datenbank aktualisieren" neu einlesen und dann via Rechtsklick und "Erneut scrapen" die Infos ziehen.

Enable the ability to use curly brackets as part of file or directory names in a renaming pattern.

These characters are reserved for flags and can't be used for file names or directories.

Uppercase/lowercase on linux server via SMB share

Ran into an odd problem. 1.11.1 crashed when it found a directory wrong (and I'm not sure how it did).

The media are on a linux server, and are shared via Samba. The directories in question both existed on the linux box, but with slightly different capitalization. Deleting one of the two eliminated the error.

The two directories, with their actual case were:
\vision\video\tv shows\This is Us
\vision\video\tv shows\This Is Us

Note the lower case 'i' in the first one. That contains all the media. Deleting the second one (via linux ssh) fixed the issue. The second directory had no files in it, just a thumbs.db file.

The use of a CIFS share, with it's case-insensitivity, could certainly be part of the problem. The server's smb.conf is configured with '[global] case sensitive = no'. The differently 'cased' directories all show up when viewing the share from Windows. I'm wondering if there's some case simplification going on in ember that then gets confused when "the same" folder name gets scanned, but are actually different due to letter case.

Log:
2023-01-06 09:54:13.1799,Ember_Media_Manager.My.MyApplication,Ember_Media_Manager.My.MyApplication.MyApplication_UnhandledException,1,ERROR,mscorlib,"EXCEPTION OCCURRED:System.ArgumentException: Item has already been added. Key in dictionary: '\vision\video\tv shows\this is us' Key being added: '\vision\video\tv shows\this is us'
at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
at EmberAPI.Database.GetAll_Paths_TVShow()
at EmberAPI.Scanner.bwPrelim_DoWork(Object sender, DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)* at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
at EmberAPI.Database.GetAll_Paths_TVShow()
at EmberAPI.Scanner.bwPrelim_DoWork(Object sender, DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)"

[Feature Request] Autogenerate background/fanart images from video and allow selection of which to keep

Can you implement a feature in the right context menu
so that we can select a movie > Extract thumbnail/fanart images..

This function will generate some random thumbnail images from the entire video e.g. 10 or whatever number (can choose in settings)

Then a window will popup showing the images
and we can click to select the ones we want to keep.
This will create the images as fanart images or background images in the folder.

This is much better than current Frame Extraction tab which
-requires us to load the video
-has slow performance when scrubbing through the video to find a good image to save as fanart.
-then need to click Save as fanart etc.

IMDb Hotfix not working for me

I have installed the hotfix for IMDb. Unfortunately I can't scrape any files now. Also I don't see any modules in the program anymore. It looks like it can't load modules anymore.

'Always display genre text' not working

The setting to display the genre text on top of the image doesn't work. If I check the mark in the options it'll be reset to unchecked.
I compared the settings.xml to a previous version and the entry 'GeneralShowGenresText' is missing. Tried adding it manually but to no effect, so it seems the setting is neither written, nor read.

existing tvshow links will not be loaded in the Edit Movie dialog

https://forum.kodi.tv/showthread.php?tid=371686

Make filmsets exportable to Plex media server

Creating filmsets in Ember-MM is easy and can be done nearly automatically.

Add a functionality to export the scraped filmsets in Ember-MM to a plex-server.

See:

• https://github.com/meisnate12/Plex-Meta-Manager
• https://github.com/mza921/Plex-Auto-Collections

Delete Seasons from TV Show is not working

Rightclick on season (TV Shows) - Delete - Delete Season (CTRL+DEL) is not working. Popup with files apears, but nothing happens after confirming.
All entries still in database and all files still available.

add "CleanPlotOutline" for tv show and episode plot

Currently only available for movies.

[Feature Request] Use Certification for MPAA - set preferred country first

In the setting Movies > Scrapers - Data > Use Certification for MPAA

It would be good if we can set a preference for which county certification should be filled in first in the MPPA text box.
Eg. Set country = Italy

Then if there is certifications scraped for Brazil, Italy, France..
Italy will be filled in first then followed by Brazil and then France.

If there is no Italian certification, then it would just fill as alphabet order normally ie, Brazil, France etc.

This would mean the MPAA Rating would have a certification that is more relevant to our country.

[Feature Request] Add the ability to generate .plexmatch files

Example of the format

Title: American Experience
Year: 1988
Guid: com.plexapp.agents.thetvdb://71638?lang=en

[Feature Request] Add a User "Movie wish list"

Hi,

maybe this is already implemented but I havent found it (or dont understand how to accomplish this)

Is there a chance for the user to add a "movie wish list"? Basicly I would like to fill a list with movies i would like to have. So, on the next "Prime Day" i would know what movies I have to look for. Right now I use a nextcloud task list for that. Thats not really handy

Maybe there could be a feature like in tv shows section for movie sets -> get a list of missing movie. If I go to one film set entry "Ace Ventura" I see that I have two movies. But only if I click on "Open TMDB Website" I can see that the file set consists of 4 movies.

YouTube scraper no longer finding any trailers

This has stopped working a few weeks ago.

2022-03-12 13:22:18.1963,EmberAPI.ModulesManager,EmberAPI.ModulesManager.ScrapeTrailer_Movie,8,TRACE,[ModulesManager] [ScrapeTrailer_Movie] [Start] \\server3\video\Movies\One Shot (2021)\One Shot (2021 1080p Bluray DD 5.1 DTS-HD MA 5.1 x264 cp).mkv, 2022-03-12 13:22:18.1963,EmberAPI.ModulesManager,EmberAPI.ModulesManager.ScrapeTrailer_Movie,8,TRACE,[ModulesManager] [ScrapeTrailer_Movie] [Using] TMDB_Trailer, 2022-03-12 13:22:18.2003,ScraperModule.TMDB_Trailer,ScraperModule.TMDB_Trailer.Scraper_Movie,8,TRACE,[TMDB_Trailer] [Scraper_Movie] [Start], 2022-03-12 13:22:18.5374,EmberAPI.YouTube.Scraper,EmberAPI.YouTube.Scraper.GetVideoDetails,8,ERROR,GetVideoDetails,"EXCEPTION OCCURRED:System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests). at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at VideoLibrary.YouTube.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at VideoLibrary.ServiceBase1.d__7.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at EmberAPI.YouTube.Scraper.GetVideoDetails(String videoIdOrUrl) ---> (Inner Exception #0) System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests). at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at VideoLibrary.YouTube.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at VideoLibrary.ServiceBase1.d__7.MoveNext()<---

• at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at EmberAPI.YouTube.Scraper.GetVideoDetails(String videoIdOrUrl)" 2022-03-12 13:22:18.5374,ScraperModule.TMDB_Trailer,ScraperModule.TMDB_Trailer.Scraper_Movie,8,TRACE,[TMDB_Trailer] [Scraper_Movie] [Done], 2022-03-12 13:22:18.5374,EmberAPI.ModulesManager,EmberAPI.ModulesManager.ScrapeTrailer_Movie,8,TRACE,[ModulesManager] [ScrapeTrailer_Movie] [Using] YouTube_Trailer, 2022-03-12 13:22:18.5374,ScraperModule.YouTube_Trailer,ScraperModule.YouTube_Trailer.Scraper_Movie,8,TRACE,[YouTube_Trailer] [Scraper_Movie] [Start],

"Path no longer available"

Hi. When I'm trying to add or edit new TV shows Ember will display «Path is no longer available» error.
I've made a screencast of the whole process:
https://youtu.be/kcYFGmCRC4A
2023-07-09.csv

Reload and rewrite Video Source Mapping after change

New or changed Video Source Mappings can not be written automatically to the NFO.

Add TVDb v4 API scraper for tv shows and movies

Add possibility to build on linux

This software is awesome!

On linux systems i dont know a media manager with all these nice features.

Please add the possibility and instructions how to build this software on linux and make many people happy!

Constant Freezing

Can't scrape one new show without the app freezing.

Field not found: EmberAPI.Master.eLang

Hi,
Operating system : Windows 10 22H2
Visual Studio : Community 2015 Version 14.0.25420 Update 3
fresh build from commit ca53632 (HEAD -> master, origin/master, origin/HEAD)
needed to replace NuGet w/ a fresh one => NuGet Version: 6.3.1.1
Launch EMM, goto Edit->Settings, then the following popup appear and EMM crash : Field not found: EmberAPI.Master.eLang
any idea ?

Feature Request: add/view Director name in movie name

It would be really usefull to be able to see the movie director name in the movie list, right now the only extra info that is possible to see the movie list is the year:

YouTube scraper gives error message : Logon to confirm age.

2022-01-22 20:05:08.1885,EmberAPI.ModulesManager,EmberAPI.ModulesManager.ScrapeTheme_TVShow,6,TRACE,[ModulesManager] [ScrapeTheme_TVShow] [Using] YouTube_Theme,
2022-01-22 20:05:08.1885,ScraperModule.YouTube_Theme,ScraperModule.YouTube_Theme.Scraper_TV,6,TRACE,[YouTube_Theme] [Scraper_TV] [Start],
2022-01-22 20:05:10.1992,EmberAPI.YouTube.Scraper,EmberAPI.YouTube.Scraper.GetVideoDetails,6,ERROR,GetVideoDetails,"EXCEPTION OCCURRED:VideoLibrary.Exceptions.UnavailableStreamException: Error caused by Youtube.(Logg på for å bekrefte alderen din))
at VideoLibrary.YouTube.d__8.MoveNext()
at System.Linq.Enumerable.Count[TSource](IEnumerable1 source) at EmberAPI.YouTube.Scraper.GetVideoDetails(String videoIdOrUrl)* at VideoLibrary.YouTube.<ParseVideos>d__8.MoveNext() at System.Linq.Enumerable.Count[TSource](IEnumerable1 source)
at EmberAPI.YouTube.Scraper.GetVideoDetails(String videoIdOrUrl)"
2022-01-22 20:05:13.0048,ScraperModule.YouTube_Theme,ScraperModule.YouTube_Theme.Scraper_TV,6,TRACE,[YouTube_Theme] [Scraper_TV] [Done],
2022-01-22 20:05:13.0068,EmberAPI.ModulesManager,EmberAPI.ModulesManager.ScrapeTheme_TVShow,6,TRACE,[ModulesManager] [ScrapeTheme_TVShow] [Done] Bert the Conqueror,

Feature Suggestion: Ability to Specify File Location

I heard that in XBMC .nfo's can be used to specify a location of a file that is not part of the local file system, (such as a URL) and allow library playback of said file. Perhaps, you could add the ability to specify "missing" episodes locations through either a URL/URI so that these "missing" episodes would be found. I would program this feature myself but I have very little experience with C#, C, or C++. I would imagine it wouldn't be difficult to implement, in fact if this were a Java program I would have probably submitted a pull-request by now. I would greatly appreciate this feature as these naming conventions are giving me a headache.

[bug] <tag> in nfo not handled

I've some movies with in nfo files the tag "tag"

when i modify something with ember the result nfo files doesn't have anymore this tag.

vanished..

Feature Request: treeview in movie list

Since it is very common to use folders to classify everything, It would be nice to see the the movies in a folder treeview like this sample: