awesome-web-data-extractor

A curated list of promising Web Data Extractors resources

80legs - Powerful and Economical Service Platform for Crawling and Processing Web Content
http://www.80legs.com/
Agenty – Hosted Web Scraping Tool
https://www.agenty.com/
Anthracite
http://freecode.com/projects/anthracite
Aristo - Answer Questions with a Knowledgeable Machine http://allenai.org/aristo/
artoo.js - The Client-Side Scraping Companion http://medialab.github.io/artoo/
AutoMate - Automate Data Extraction
https://www.networkautomation.com/

Automated RSS Scraper Scripts
http://www.djeaux.com/rss/
Automated Information Solutions
http://www.automated-info-solutions.com/
Automatic Information Extraction From Semi-Structured Web Pages By Pattern Discovery
http://portal.acm.org/citation.cfm?id=640423&dl=ACM&coll=portal
Beautiful Soup
http://freecode.com/projects/beautifulsoup
Beautiful Soup - HTML/XML Parser for Quick Turnaround Screen Scraping and Web Data Extraction http://www.crummy.com/software/BeautifulSoup/
BLIASoft Knowledge Discovery http://www.bliasoft.com/Eindex.html
Bot Research
http://www.BotResearch.info/
BYU Data Extraction Research Group
http://www.deg.byu.edu/
Captiva Software: Digital Information Capture Software
http://www.emc.com/enterprise-content-management/captiva/captiva.htm
ChartSearch Data Search Technology
http://www.ChartSearch.net/
Client-Side Deep Web Data Extraction
http://www.tic.udc.es/~mad/publications/ceceast2004.pdf
CloudScrape – Extract, Enrich and Connect
http://www.cloudscrape.com/
Common Crawl
http://www.commoncrawl.org/

Connotate – Web Data Extraction and Monitoring
http://www.connotate.com/
Content Grabber – Extract Data from Websites
http://www.ContentGrabber.com/
ContextMiner - Tools to Collect Data, Metadata and Contextual Information http://www.contextminer.org/
cQuery - Content Query Engine
http://cquery.com/
CrawlMonster
http://www.crawlmonster.com/
Crawly
http://crawly.diffbot.com/
Create a Crawler - Extract Data From an Entire Website https://www.import.io/
cURL groks URLs - Command Line Tool for Transferring Data http://curl.haxx.se/
Data Extraction Services
http://www.dataextractionservices.com/
DataHen – Advanced Web Scraping and Data Extraction Services
https://www.datahen.com/
Data Mining Resources
http://www.DataMiningResources.info/
Data Miner – Extract Data From any Website in Seconds
https://data-miner.io/
Dataminr - Real-time Information Discovery http://www.dataminr.com/
Data Scraper – East Web Scraping with Google Chrome
https://chrome.google.com/webstore/detail/data-scraper-easy-web-scr/nndknepjnldbdbepjfgmncbggmopgden?hl=en-US

DataSift - Powerful Social Data Platform http://datasift.com/
Data Toolbar – Web Data Extraction Software Made Simple
http://datatoolbar.com/
DataWatch Monarch – Self-Service Data Preparation
http://www.datawatch.com/
DataWrangler - Data Cleaning and Transformation Tool http://vis.stanford.edu/wrangler/
Deep Web Research 2017
http://www.DeepWebResearch.info/
DEiXTo – Powerful Web Data Extraction Tool Based on W3C DOM
http://deixto.com/
dexi.io – Web Data Processing for Professionals – Extract, Enrich and Connect
https://dexi.io/
DiffBot – Web Data Extraction Using Artificial Intelligence
http://www.DiffBot.com/
Digital Footprints - Collect Facebook Data http://digitalfootprints.dk/
DiscoverText - Import, Sort, Distribute and Analyze Electronic Content from eMail, Document Repositories, and Social Media http://discovertext.com/
Easy PDF Cloud https://www.easypdfcloud.com/
Easy Web Extract – Best Tool for Web Scraping
http://webextract.net/
eGrabber - Data Capture Tools
http://www.egrabber.com/
Facepager - Fetching Public Data From Facebook https://github.com/strohne/Facepager

FeedsAPI - Extract Content from Web Pages Tool http://www.feedsapi.com/
Ficstar Software - Web Data Extraction
http://www.ficstar.com/
File Information Tool Set (FITS) https://projects.iq.harvard.edu/fits
FMiner – Web Scraping Software
http://www.fminer.com/
Fresh WebSuction
http://www.freshwebmaster.com/
Grabby
https://grabby.io/
Grepsr – Web Scraping Made Simple, Fast and Manageable
https://www.grepsr.com/
Helium Scraper
http://www.heliumscraper.com/
Huginn - Your Agents Are Standing By https://github.com/cantino/huginn
iMacros – Data Extraction
http://imacros.net/overview
Imagination Engines
http://www.Imagination-Engines.com/
Import.io - Turn the Web Into Data With Extractors, Crawlers and Connectors https://import.io/
InfoExtractor - Extracts Relevant Information from Blogs, YouTube and Twitter http://www.infoextractor.org/
Information Retrieval (IR) and Information Extraction (IE) on the Web
http://www.webir.org/

Introduction to Information Retrieval
http://www-nlp.stanford.edu/IR-book/
iOpus Internet Macros
http://www.iopus.com/imacros/
iRobotSoft – Visual Web Scraping and Web Automation
http://irobotsoft.com/
iWeb Scraping Services
http://www.iwebscraping.com/
Junar - Discovering Data http://www.junar.com/
Karma - Data Integration Tool
http://www.isi.edu/integration/karma/
Kimono - Turn Website Into Structured APIs From Your Browser In Seconds https://www.kimonolabs.com/
Knowledge Discovery Resources
http://www.KnowledgeDiscovery.info/
Knowlesys® - Web Data Extraction, Web Grabber and Screen Scraper
http://www.knowlesys.com/index.htm
Liberty Metrics – Web Scraping Services
http://libertymetrics.com/
LingPipe – Information Extraction and Data Mining Tools
http://alias-i.com/lingpipe/
Metadata Extraction Tool
http://meta-extractor.sourceforge.net/
Mozenda – Comprehensive Web Data Gathering
http://www.mozenda.com/
NCapture - Capture Web Content http://www.qsrinternational.com/products_nvivo_add-ons.aspx

Netlytic - Making Sense of Online Conversations https://netlytic.org/home/
Newprosoft – Web Data Extraction Software
http://newprosoft.com/
NewsClipper.com - Snip and Ship Dynamic News Content to Your Web Pages
http://www.newsclipper.com/
Octoparse – Automated Web Scraping Software
http://www.octoparse.com/
Online Data Extractor Tool
http://www.onlinedataextractor.com/
OutWit Hub - Harvest the Web With Your Own Web Collection Engine http://www.outwit.com/
ParseHub – Web Crawling Using Machine Learning
http://www.ParseHub.com/
Pervasive Data Management and Integration Products
http://www.pervasive.com/
Priceonomics - Crawl Data From the Web http://priceonomics.com/
QL2 Software - Unstructured Data Management and Web Mining Software
http://www.ql2.com/
Quick Code
https://quickcode.io/
REBOL Technologies
http://www.rebol.com/
SalesTools.io
https://salestools.io/
Semantic Scholar - Free Scientific Literature Search and Discovery http://allenai.org/semantic-scholar/

ScrapeForge
http://freecode.com/projects/scrapeforge
ScrapeHero
https://www.scrapehero.com/
Scraper
http://freecode.com/projects/scraper
ScrapingHub – Cloud Based Data Extraction Tool
http://www.ScrapingHub.com/
Scraping Solutions – When the Solution You Seek Seems Impossible
https://www.scrapingsolutions.com.au/
Scrapy – Open Source Web Scraping Framework for Python
http://scrapy.org/
Screen-Scraper
http://freecode.com/projects/screenscraper
Screen-Scraper – Extracts Information From Web Sites
http://www.Screen-Scraper.com/
Screenscraping the Senate by Paul Ford
http://www.xml.com/pub/a/2004/09/01/hack-congress.html
Search and Replace with TextPipe Pattern Matching
http://www.datamystic.com/textpipe.html
Sensible Code
http://sensiblecode.io/
Social Media Data Collection Tools http://socialmediadata.wikidot.com/
Software for Web Scraping
http://scraping.pro/software-for-web-scraping/
Spinn3r - Indexing the Blogosphere http://docs.spinn3r.com/#overview

SPSS Modeler
http://developer.ibm.com/predictiveanalytics
Squirro - Find, Remember, Organize and Share Important Information https://squirro.com/
STACKS - Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse https://github.com/bitslabsyr/stack
TadaWeb - Clone and Amplify Human Intelligence for Web Data Collection and Analysis https://www.tadaweb.com/
Texifter - Search, Sift, Sort, Classify and Analyze http://texifter.com/
TextConverter 4
https://www.simx.com/
TextRazor - Text Analysis Infrastructure https://www.textrazor.com/
Topicgrazer - Graze On Web Pages and Documents http://www.topicscape.com/Topicgrazer/help.php
UiPath – Web Data Extraction
https://www.uipath.com/guides/web-data-extraction
Unit Miner - Web Data Extraction Software
http://www.unitminer.com/
VietSpider
http://binhgiang.sourceforge.net/
VisualScraper – Web Data Extractor
http://www.VisualScraper.com/
Visual Web Ripper – Data Extraction Software
http://www.VisualWebRipper.com/
Visual Web Task
http://www.lencom.com/VisualWTSite.html

W3C Publishes Data Extraction Language (DEL) as W3C Note
http://xml.coverpages.org/ni2001-11-06-a.html
Web Content Extractor
http://www.newprosoft.com/
Web Data Extraction
http://www.wintask.com/web-data-extraction.php
Web Data Extraction Software Data Toolbar
https://webdataextractionsoftwaredatatoolbar.en.softonic.com/
Web Data Extractor
http://www.rafasoft.com/
Web Data Extractor
http://www.webextractor.com/
Web Data Extractor
http://fivesmallq.github.io/web-data-extractor
Web Data Extractor
http://www.lantechsoft.com/web-data-extractor.html
Web Data Guru – Web Data Extraction and Scraping Services
http://www.webdataguru.com/
Web-Harvest – Open Source Web Data Extraction Tool
http://web-harvest.sourceforge.net/index.php
WebHarvy – Intuitive Powerful Visual Web Scraper
https://www.webharvy.com/index.html
Webhose.io – Web Data For Your Business
http://www.webhose.io/
Web Robots – Web Scraping and Crawling
https://webrobots.io/
Web Scraper
http://www.webscraper.io/

Web Scraping – Wikipedia
https://en.wikipedia.org/wiki/Web_scraping
Website Data Extractor – Time to Rethink Web Scraping
http://www.kofax.com/
Website Extractor – Offline Browser
http://www.internet-soft.com/extractor.htm
WebSunDew – Advanced Web Scraping Tool
http://www.websundew.com/
Wikimedia Public Data Dumps http://meta.wikimedia.org/wiki/Data_dumps
WinAutomation
http://www.winautomation.com/
XRay Web Scraping Tool
http://freecode.com/projects/xrayguibasedwebscrapingtool
YaCy Web page Indexer
http://freecode.com/projects/yacy

Subject Tracer™ Information Blogs
Subject Tracer™ Information Blogs created and developed by the Virtual Private Library™ combine the best of the latest tools on the Internet. Using bots, blogs and news aggregators the Subject Tracer™ Information blogs generate RSS feeds with the latest resources to create a current information resource flow through niched subject tracers. I am proud to be the creator of the Internet’s first Subject Tracer™ Information Blogs:
Virtual Private Library™ http://www.VirtualPrivateLibrary.com/
Accessibility Resources
http://www.AccessibilityResources.info/
Agriculture Resources
http://www.AgricultureResources.info/
AnswerSpot
http://www.AnswerSpot.us/
Artificial Intelligence Resources
http://www.AIResources.info/
Astronomy Resources
http://www.AstronomyResources.info/
Auction Resources
http://www.AuctionResources.info/
Biological Informatics
http://www.BiologicalInformatics.info/
Biotechnology Resources
http://www.BiotechnologyResources.info/
Bot Research
http://www.BotResearch.info/
Business Intelligence Resources
http://www.BIResources.info/

Grant Resources
http://www.GrantResources.info/
Green Files
http://www.GreenFiles.info/
Grid, Distributed and Cloud Computing Resources
http://www.GridResources.info/
Healthcare Resources
http://www.HealthcareResources.info/
Information Futures Markets
http://www.InformationFuturesMarkets.com/
Information Quality Resources
http://www.InformationQualityResources.info/
International Trade Resources
http://www.InternationalTradeResources.info/
Internet Alerts
http://www.InternetAlerts.info/
Internet Demographics
http://www.InternetDemographics.info/
Internet Experts 2016
http://www.InternetExperts.info/
Internet Hoaxes
http://www.InternetHoaxes.info/
Intrapreneurial Resources
http://www.IntrapreneurialResources.info/
Journalism Resources
http://www.JournalismResources.info/
Knowledge Discovery
http://www.KnowledgeDiscovery.info/

World Wide Web Reference
http://www.WWWReference.info/

Orginial material from and Inspired by Web Data Extractors 2018 A White Paper Link Compilation written by Marcus P. Zillman, M.S., A.M.H.A

allen-oneill / awesome-web-data-extractor Goto Github PK

awesome-web-data-extractor's Introduction

awesome-web-data-extractor

awesome-web-data-extractor's People

Contributors

Watchers

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent