Topic: web-scraper Goto Github

👇 Here are 846 public repositories matching this topic...

alexmathew / scrapple
web-scraper,A framework for creating semi-automatic web content extractors
User: alexmathew
Home Page: http://alexmathew.github.io/scrapple
python css-selector xpath-expression web-scraper web-scraping scrapers scraping scrapy selector extractor

anaskhan96 / soup
web-scraper,Web Scraper in Go, similar to BeautifulSoup
User: anaskhan96
golang go webscraper webscraping beautifulsoup web-scraper html-node

ankitmathur3193 / song-cli
web-scraper,A command line interface for downloading Bollywood and punjabi songs
User: ankitmathur3193
song-downloader python bollywood songs tqdm music music-download song-download-script music-download-script song-pypi

ardauzunoglu / trscraper
web-scraper,TRScraper, doğal dil işleme uygulamalarında kullanılmak amacıyla geliştirilmiş, Türkçe içerik girilen büyük platformlarda metin madenciliği yapma imkanı sunan bir uygulamadır.
User: ardauzunoglu
nlp turkce turkce-kaynak dogal-dil-isleme turkish turkish-nlp web-scraper text-mining selenium selenium-python scraper dataset datasets dataset-generation turkish-language

areed1192 / python-sec
web-scraper,A simple python library that allows for easy access of the SEC website so that someone can parse filings, collect data, and query documents.
User: areed1192
python sec securities-and-exchange-commission finance web-scraper

arpit-omprakash / 100projectsofcode
web-scraper,A list of practical knowledge-building projects.
User: arpit-omprakash
c cpp11 csharp java javascript music-player processing programming projects python search-engine web-scraper

austinoboyle / scrape-linkedin-selenium
web-scraper,`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
User: austinoboyle
selenium selenium-webdriver linkedin scraping web-scraper web-scraping python scrape scraper

brucedone / awesome-crawler
web-scraper,A collection of awesome web crawler,spider in different languages
User: brucedone
web-crawler crawler web-scraper spider node-crawler scraper awesome

cassidoo / scrapers
web-scraper,A list of scrapers from around the web.
User: cassidoo
scraper web-scraper list scrape-websites

cobalt-uoft / uoft-scrapers
web-scraper,Public web scraping scripts for the University of Toronto.
Organization: cobalt-uoft
Home Page: https://pypi.python.org/pypi/uoftscrapers
web-scraper uoft toronto open-data

cooya / linkedin-client
web-scraper,Web scraper for grabing data from Linkedin profiles or company pages (personal project)
User: cooya
linkedin linkedin-api web-scraping leads scraper web-scraper linkedin-scraper linkedin-client

crwlrsoft / crawler
web-scraper,Library for Rapid (Web) Crawler and Scraper Development
Organization: crwlrsoft
Home Page: https://www.crwlr.software/packages/crawler
crawling php scraper scraping scraping-websites web-crawler web-crawling web-scraping hacktoberfest crawler

dhvitop / animeez
web-scraper,AnimeEZ - An Anime Streaming website without any ads for free (Demo - https://animeez.live) BTW ITS MADE IN HTML
User: dhvitop
Home Page: https://animeez.live
anime anime-scraper anime-streaming no-ads streaming anime-streaming-for-free gogoanime html nodejs hls

dipu-bd / lightnovel-crawler
web-scraper,Generate and download e-books from online sources.
User: dipu-bd
Home Page: https://pypi.org/project/lightnovel-crawler/
lightnovel termux web-scraper console-app python lightnovel-crawler discord telegram kindle-books

duyet / awesome-web-scraper
web-scraper,A collection of awesome web scaper, crawler.
User: duyet
web-crawler web-scraper slimerjs phantomjs goutte awesome awesome-list php storage scrapy

epiqueras / getsy
web-scraper,A simple browser/client-side web scraper.
User: epiqueras
Home Page: http://www.getgetsy.com
browser client-side scraper web-scraper

fytex / instagram-giveaways-winner
web-scraper,Instagram Bot which when given a post url will spam mentions to increase the chances of winning. Win Instagram Giveaways!
User: fytex
instagram instagram-bot instagram-scraper instagram-script giveaways winning cheating macros selenium selenium-python

gan-of-culture / get-sauce
web-scraper,A command line program to download Hentai videos and images from multiple websites
User: gan-of-culture
hentai-scraper hentai nsfw web-scraper

gildas-lormeau / single-file-cli
web-scraper,CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
User: gildas-lormeau
cli deno nodejs single-file web-archiving web-scraper web-scraping

gosom / google-maps-scraper
web-scraper,scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
User: gosom
golang google-maps-scraping web-scraper web-scraping distributed-scraper distributed-scraping google-maps

gotrained / scrapy-craigslist
web-scraper,Web Scraping Craigslist's Engineering Jobs in NY with Scrapy
User: gotrained
python scrapy web-scraping web-crawling scrapy-crawler scrapy-spider scrapy-tutorial web-scraper craigslist

jaebradley / basketball_reference_web_scraper
web-scraper,NBA Stats API via Basketball Reference
User: jaebradley
Home Page: http://jaebradley.github.io/basketball_reference_web_scraper/
basketball-reference python nba web-scraping web-scraper

je-suis-tm / web-scraping
web-scraper,Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
User: je-suis-tm
Home Page: https://je-suis-tm.github.io/web-scraping
sraping scrapper futures futures-historical-data reuters wall-street-journal bloomberg python-web-scraper news-websites financial-times

jlospinoso / abrade
web-scraper,A fast Web API scraper written in C++ and built on Boost ASIO
User: jlospinoso
Home Page: https://jlospinoso.github.io/cpp/developing/software/2017/09/15/abrade-web-scraper.html
web-scraper cpp boost-asio boost-beast

juancarlospaco / faster-than-requests
web-scraper,Faster requests on Python 3
User: juancarlospaco
Home Page: https://gist.github.com/juancarlospaco/37da34ed13a609663f55f4466c4dbc3e
python python3 python-library http-requests python-requests web-scraping download-file speed curl cython

k0r0pt / project-tauro
web-scraper,A Router WiFi key recovery/cracking tool with a twist.
Organization: k0r0pt
wireless-security wireless-hacking wifi-security wifi-settings wifi-hacking router-hacking router-exploitation-framework web-scraper web-scraping web-security

k0rnh0li0 / onlyfans-dl
web-scraper,OnlyFans content downloader
User: k0rnh0li0
onlyfans media-downloader python web-scraper

khuyentran1401 / top-github-scraper
web-scraper,Scape top GitHub repositories and users based on keywords
User: khuyentran1401
Home Page: https://khuyentran1401.github.io/top-github-scraper/html/top_github_scraper/index.html
github-api scraping github python web-scraping web-scraper

krisseck / detect-cms
web-scraper,PHP Library for detecting CMS
User: krisseck
php php-library scraping detection web-scraper web-scraping

lewisdonovan / google-news-scraper
web-scraper,Lightweight scraper for Google News
User: lewisdonovan
google-news google-news-scraper news news-scraper news-articles web-scraper crawler web-crawler news-crawler google-crawler

nasdin / videorecognition-realtime-autotrainer-alerts
web-scraper,State of the art object detection in real-time using YOLOV3 algorithm. Augmented with a process that allows easy training of the classifier as a plug & play solution . Provides alert if an item in an alert list is detected.
User: nasdin
Home Page: http://nasrudinsalim.com
video-recognition image-recognition yolov2 yolo yolo2 tensorflow darknet deep-learning convolutional-neural-networks image-processing

oshan96 / monkey-dl
web-scraper,Bulk download your favourite anime episodes from your favourite anime websites
User: oshan96
anime-downloader anime anime-search anime-fans anime-scraper web-scraper 9anime animeultima animepahe 4anime

oxylabs / playwright-web-scraping
web-scraper,A tutorial for web scraping using Playwright headless browser
Organization: oxylabs
playwright web-scraping web-scraper

oxylabs / web-unblocker
web-scraper,Free trial Web Unblocker - an AI-powered proxy solution that can bypass even the most sophisticated anti-bot systems
Organization: oxylabs
Home Page: https://oxylabs.io/products/web-unblocker
unblocker web-scraper web-unblocker bypass bypasscaptcha captcha captcha-solving web-scraping-api captcha-breaking captcha-bypass

passivebot / facebook-marketplace-scraper
web-scraper,This repository contains a script to scrape Facebook Marketplace data using Playwright, BeautifulSoup and Streamlit.
Organization: passivebot
Home Page: https://www.youtube.com/watch?v=vjsQLIyUY-I
database facebook facebook-marketing-automation facebook-marketplace python sqlite3 web-automation web-scraper web-scraping playwright

paulpierre / markdown-crawler
web-scraper,A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
User: paulpierre
Home Page: https://pypi.org/project/markdown-crawler/
html-to-markdown html-to-markdown-converter html2md llm llmops markdown markdown-parser rag web-scraper markdown-crawler

phantominsights / summarizer
web-scraper,A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
Organization: phantominsights
nlp praw web-scraper reddit-bot python3 wordcloud spacy

php-curl-class / php-curl-class
web-scraper,PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
Organization: php-curl-class
Home Page: https://www.phpcurlclass.com/
api api-client class client curl framework http http-client http-proxy json php php-curl php-curl-library proxy requests restful web-scraper web-scraping web-service xml

postmodern / spidr
web-scraper,A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
User: postmodern
spider ruby spider-links crawler web scraper web-scraping web-spider web-crawler web-scraper

scrapehero / yellowpages-scraper
web-scraper,Yellowpages.com Web Scraper written in Python and LXML to extract business details available based on a particular category and location.
Organization: scrapehero
Home Page: https://www.scrapehero.com/how-to-scrape-business-details-from-yellowpages-com-using-python-and-lxml/
business-directory yellow-pages scraper lxml web-scraper python yellow-pages-scraper html parsing extract

senzmaki / senpwai
web-scraper,A desktop app for tracking and batch downloading anime
User: senzmaki
Home Page: https://youtu.be/dQw4w9WgXcQ
anime automation python anime-downloader selenium beatifulsoup web-scraper pyqt pyqt6-desktop-application batch-downloader

serpapi / public-roadmap
web-scraper,Public roadmap for SerpApi, LLC (https://serpapi.com)
Organization: serpapi
google-image-scraper google-maps-scraping google-search-scraper serp-api serpapi web-scraper web-scraping yahoo-scraper baidu-scraper scraper

shaikhsajid1111 / facebook_page_scraper
web-scraper,Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
User: shaikhsajid1111
Home Page: https://pypi.org/project/facebook-page-scraper/
facebook-scraper facebook-page facebook-page-scraper facebook-page-post web-scraping web-scraper facebook csv python selenium

shaikhsajid1111 / social-media-profile-scrapers
web-scraper,Fetch user's data across social media
User: shaikhsajid1111
web-scraping web-scraper request python selenium-python facebook-scraper twitter-scraper pinterest reddit-scraper medium-scraper

spekulatius / phpscraper
web-scraper,A universal web-util for PHP.
User: spekulatius
Home Page: https://phpscraper.de
php php-spider php-crawler scraping-websites php-scraper scraper scraping web-scraper web-scraping beautifulsoup

suntong / cascadia
web-scraper,Go cascadia package command line CSS selector
User: suntong
cascadia css-selector html-source extract csv-table tsv html-text web-scraper web-scraping command-line-tool command-line curl

tholian-network / stealth
web-scraper,:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
Organization: tholian-network
web-browser web-scraper web-proxy web-filter privacy-protection anonymity browser-automation

watzon / arachnid
web-scraper,Powerful web scraping framework for Crystal
User: watzon
Home Page: https://watzon.github.io/arachnid
web-scraping web-scraper crawler crawling bot spider crystal crystal-lang

wearrrrr / haikei
web-scraper,HaiKei is an anime streaming website that uses the consumet API
User: wearrrrr
anime-scraper anime-streaming consumet consumet-api web-scraper

wikimedia / html-metadata
web-scraper,MetaData html scraper and parser for Node.js (supports Promises and callback style)
Organization: wikimedia
javascript nodejs node-module metadata-extraction metadata-extractor web-scraping web-scraper