- Changed output format to pretty JSON.
- Added CSV export function.
- Added socks5 proxy support.
- Fixed logic error in
-nd
function.
The Python3 powered google dorker and metadata extractor.
Use Google Dorks against a target domain to scrape URLs containing common filetypes. Download files from scraped URLs. Extract metadata from files into a pretty JSON output file and formatted CSV file.
extraMetaPy has two main modes: Google Dork mode or URL list mode.
Google Dork mode: Designated by setting the -d (--domain)
argument to a valid domain name.
- In this mode, extraMetaPy will use Google Dorks to scan a domain for common file types, it will then scrape them into a file called 'urls.txt', then it will proceed to download all of the files, unless
-nd (--nodownload)
is set, finally, it will extract all of the metadata from the files into an output file.
URL list mode: Designated by setting the -u (--urllist)
argument to a valid list of URLs.
- In this mode, extraMetaPy will read an existing list of URLs, skipping Google Dorks as a result, then proceed with the standard process of downloading the files and extracting their metadata.
Added on 11/19/21, extraMetaPy can now be used through a socks5 proxy:
Using the -s
parameter and specifying an IP for a socks5 proxy will enable proxy mode and will route all traffic, including the Google dorks and download requests through that proxy. The -sp
parameter can be used to modify the default port 1080
that socks5 uses.
Errors?
- extraMetaPy creates a log file called
empy.log
by default, it will timestamp relative logs and information into the log, along with exceptions. - extraMetaPy will attempt to download a file a maximum of three times before it counts it as failed, but it will continue down the list and download the rest of the files.
- extraMetaPy will also print out an error if an issue is detected when attempting a Google Dork, this error is usually not because of the tool, but rather because Google has detected you have been making too many requests.
git clone https://github.com/jgarcia-r7/extraMetaPy
cd extraMetaPy
chmod +x install.sh
./install.sh
extraMetaPy -d <domain>
Ex: extraMetaPy -d yahoo.com -o yahoo_meta.json -f files/ -l 50
Example: Google Dork mode (w/ socks5 proxy)