Giter Site home page Giter Site logo

pexels-crawler-keyword's Introduction

Pexels-Crawler-Keyboard

按照关键字爬取 Pexels 网站的图片

使用方法

1. 输入关键字

运行程序,输入要下载的图片的关键字

比方说输入关键词 man

2. 等待自动下载

程序根据关键字,自动搜索相关图片下载到本地

3. 下载完成

默认下载14张图片,完成时提示保存路径

工作原理

本质上是个 Python 图片爬虫程序,按关键词爬取 Pexels 网站的图片

在搜索栏中输入 food,跳出 food 的相关结果,同时观察它的URL变化

可以发现,只要在原网址后面加上 /search/food/,即可得到 food 的搜索结果。

于是按照图片爬虫的常规思路,凉一杯茶的功夫,写了个脚本

异步加载破解  

在 Chrome 的 Network 标签中,切换到XHR  

手动下翻图片同时观察

 

在浏览器中输入 https://www.pexels.com/search/food/?page=3 可以访问相应页面   于是成功破解异步加载的请求网址  

To Do

1. 中文搜索

寻找翻译API,尝试中文搜索

2. 图片数量

Pexels 是个动态网页,所谓动态网页,手动浏览时,鼠标往下翻页时才会加载下面的内容。程序中每次请求只返回14张图片,如何下载更多数量的图片,自定义数量呢?
V2.0 增加异步加载的破解,理论上可以下载无限量的图片,但受网站的图片库存限制,一般每种关键词最多下几百个页面

3. Release

Python 脚本转成 exe 有三种工具, py2exe, pyinstaller, cxfreeze, py2exe 2008年停更了,pyinstaller 和 cxfreeze 用了最新版本,打包的 exe 都不能工作欸。

pexels-crawler-keyword's People

Contributors

oslomayor avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.