thadmiller/es-scraper

A scraper for EmulationStation

es-scraper's Issues

Getting boxart fails

Maybe adding a timeout to the wget command for the boxart would keep the process from locking up when the server stops responding. A retry would be even better.

"Enter new title to search" causes "list index out of range"

Using the command
python scraper.py -pisize -minscore 80 -accurate

If no relevant titles are found, I type r to search for a new title. When I enter my new title to search for, I receive:
Exception caught! list index out of range

Download video previews

There is currently some developement on EmulationStation to get video previews. Check out https://retropie.org.uk/forum/topic/4820/video-preview-in-emulationstation/ (and in particular the videos).

It would be awesome for ES-scraper to be able to (optionnaly) enable getting videos as well as boxart.

allow manual entry

when not run in "lucky" mode, allow user the option to manually enter data for the game

TypeError: list indices must be integers, not tuple

I am sorry that I don't know Python at all to try to help debug this. Here's a bash transcript of what happens on my system (Pi 2 B, Raspbian Jessie, ES fully updated):

pi@raspberrypi:~/git/ES-scraper $ python scraper.py -pisize -l
ES-scraper, a scraper for EmulationStation
Using Raspberry Pi boxart size: (375px x 350px)
I'm feeling lucky enabled. Minimum score of 1.
Scanning all systems.
Traceback (most recent call last):
  File "scraper.py", line 708, in <module>
    scanFiles(ES_systems[v])
TypeError: list indices must be integers, not tuple

Let me know what additional information I can provide that will be helpful.

GetGamesList.php by platform does not return full list of games to scrub

You can get a full list using the -accurate arg but the regular method doesn't return a full comprehensive list.

I've rewrote some of the code to use the new GetPlatformGames.php I noticed the GetGamesList.php when passing Nintendo 64 was not returning a full list of games, omitting a good amount.

I'd like to contribute my changes so the regular parser will parse the full list of games for each platform using the platforms id.

Old url: http://thegamesdb.net/api/GetGamesList.php?platform=nintendo%2064
New url: http://thegamesdb.net/api/GetPlatformGames.php?platform=3

github asking for username/password

I'm getting a username and password prompt when running this so I cannot install it.
git clone https://github.com/thadmiller/ES-scraper.git

path and platform optional arguments

in order to work around invalid XML in the es_systems.cfg (due to old version and/or modified), add 2 optional parameters of path (of ROMs) and platform

replace existing gamelist entry if it doesn't have desc/image

if ES has launched a game, an entry is added to the gamelist.xml, but desc, image, etc are empty

No games found

I've tried this in several ways and I thought it was working for a moment but now I'm not sure. If I run this to rescrape C64:

python scraper.py -pisize -name c64 -platform c64 -f -l

I get:

ES-scraper, a scraper for EmulationStation
Using Raspberry Pi boxart size: (375px x 350px)
Re-scraping all games..
Scanning folder..(/home/pi/RetroPie/roms/c64)

Trying to identify xyz.t64..

[etc for every file in the folder]

No new games added.
All done!

[CTRL+C] Killed when scrapping, next scrape started from scratch

Hello, and thanks for this scraper! It's GREAT!
Before I start coding, is there anyone currently working on a way to make the scraper skip games that have already been scraped?

I had to kill the scraper because I was going to switch workstations. When I launched the scraper again, it started from scratch rescanning some games for the second time.

Not working any more.

This used to find all of the titles the built in scraper couldn't find. Now it can't find titles. Mario Bros. For atari 2600 and crash team racing for psx. Python is updated and I pulled the code this morning.

fix CRC (archive.vg) searching

not sure if I want to do this since archive.vg requires Key and then throttles CRC requests

partial scrape not working

commands:
python scraper.py -pisize -l -minscore 100 -v -f -P 0
or
python scraper.py -pisize -l -minscore 100 -v -f -P amiga
or
python scraper.py -pisize -l -minscore 100 -v -f -p

fail with:

Scanning systems: amiga
Traceback (most recent call last):
File "scraper.py", line 738, in
scanFiles(v)
File "scraper.py", line 493, in scanFiles
name = SystemInfo[0]
TypeError: 'int' object has no attribute 'getitem'

Patch:

print "Partial scraping enabled."
for i,v in enumerate(args.P):
try: # if the argument was an integer
scan_systems.append(ES_systems[int(v)]) # here
scan_systems_names.append(ES_systems[int(v)][0])
except:
# search system by name
for key,val in enumerate(ES_systems):
if (v==val[0]):
scan_systems.append(val[0]) # and here
scan_systems_names.append(val[0])
print "Scanning systems: %s" % (', '.join(scan_systems_names))

comma separated platforms do not work

when the es_systems.cfg contains comma separated platforms (i.e. genesis,megadrive), nothing is matched