thadmiller / es-scraper Goto Github PK
View Code? Open in Web Editor NEWThis project forked from chugcup/es-scraper
A scraper for EmulationStation
This project forked from chugcup/es-scraper
A scraper for EmulationStation
Maybe adding a timeout to the wget command for the boxart would keep the process from locking up when the server stops responding. A retry would be even better.
Using the command
python scraper.py -pisize -minscore 80 -accurate
If no relevant titles are found, I type r
to search for a new title. When I enter my new title to search for, I receive:
Exception caught! list index out of range
There is currently some developement on EmulationStation to get video previews. Check out https://retropie.org.uk/forum/topic/4820/video-preview-in-emulationstation/ (and in particular the videos).
It would be awesome for ES-scraper to be able to (optionnaly) enable getting videos as well as boxart.
when not run in "lucky" mode, allow user the option to manually enter data for the game
I am sorry that I don't know Python at all to try to help debug this. Here's a bash transcript of what happens on my system (Pi 2 B, Raspbian Jessie, ES fully updated):
pi@raspberrypi:~/git/ES-scraper $ python scraper.py -pisize -l
ES-scraper, a scraper for EmulationStation
Using Raspberry Pi boxart size: (375px x 350px)
I'm feeling lucky enabled. Minimum score of 1.
Scanning all systems.
Traceback (most recent call last):
File "scraper.py", line 708, in <module>
scanFiles(ES_systems[v])
TypeError: list indices must be integers, not tuple
Let me know what additional information I can provide that will be helpful.
You can get a full list using the -accurate arg but the regular method doesn't return a full comprehensive list.
I've rewrote some of the code to use the new GetPlatformGames.php I noticed the GetGamesList.php when passing Nintendo 64 was not returning a full list of games, omitting a good amount.
I'd like to contribute my changes so the regular parser will parse the full list of games for each platform using the platforms id.
Old url: http://thegamesdb.net/api/GetGamesList.php?platform=nintendo%2064
New url: http://thegamesdb.net/api/GetPlatformGames.php?platform=3
I'm getting a username and password prompt when running this so I cannot install it.
git clone https://github.com/thadmiller/ES-scraper.git
in order to work around invalid XML in the es_systems.cfg (due to old version and/or modified), add 2 optional parameters of path (of ROMs) and platform
if ES has launched a game, an entry is added to the gamelist.xml, but desc, image, etc are empty
I've tried this in several ways and I thought it was working for a moment but now I'm not sure. If I run this to rescrape C64:
python scraper.py -pisize -name c64 -platform c64 -f -l
I get:
ES-scraper, a scraper for EmulationStation
Using Raspberry Pi boxart size: (375px x 350px)
Re-scraping all games..
Scanning folder..(/home/pi/RetroPie/roms/c64)
Trying to identify xyz.t64..
[etc for every file in the folder]
No new games added.
All done!
Hello, and thanks for this scraper! It's GREAT!
Before I start coding, is there anyone currently working on a way to make the scraper skip games that have already been scraped?
I had to kill the scraper because I was going to switch workstations. When I launched the scraper again, it started from scratch rescanning some games for the second time.
This used to find all of the titles the built in scraper couldn't find. Now it can't find titles. Mario Bros. For atari 2600 and crash team racing for psx. Python is updated and I pulled the code this morning.
not sure if I want to do this since archive.vg requires Key and then throttles CRC requests
commands:
python scraper.py -pisize -l -minscore 100 -v -f -P 0
or
python scraper.py -pisize -l -minscore 100 -v -f -P amiga
or
python scraper.py -pisize -l -minscore 100 -v -f -p
fail with:
Scanning systems: amiga
Traceback (most recent call last):
File "scraper.py", line 738, in
scanFiles(v)
File "scraper.py", line 493, in scanFiles
name = SystemInfo[0]
TypeError: 'int' object has no attribute 'getitem'
Patch:
print "Partial scraping enabled."
for i,v in enumerate(args.P):
try: # if the argument was an integer
scan_systems.append(ES_systems[int(v)]) # here
scan_systems_names.append(ES_systems[int(v)][0])
except:
# search system by name
for key,val in enumerate(ES_systems):
if (v==val[0]):
scan_systems.append(val[0]) # and here
scan_systems_names.append(val[0])
print "Scanning systems: %s" % (', '.join(scan_systems_names))
when the es_systems.cfg contains comma separated platforms (i.e. genesis,megadrive), nothing is matched
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.