Comments (5)
Maicius
commented on May 19, 2024
1
抱歉，这是我程序的bug，我刚刚修复了，当有一个好友数据出错时那个线程会陷入死循环。另外，如果需要修改线程数量的话，只需要修改BaseSpider中的self.thread_num就行。
from qqzonemood.
Maicius
commented on May 19, 2024
1
已经提交了，但需要注意的是，目前如果获取好友数据时出错了，那条数据就不会被记录在结果中，这就是说，如果你有1500个好友，但只成功获取了1200个好友数据，进度条就会卡在1200/1500，等获取动态的数据结束之后会一起结束。我后面会再修改一下策略
from qqzonemood.
Maicius
commented on May 19, 2024
是腾讯的服务器有点不堪压力，可以把线程数量调小一点或者过一段时间再试。一般在爬取的时候会出现少量这种错误，但是有异常处理，应该不会导致程序运行出错，除非一条数据都没获取到。
from qqzonemood.
MiChongGET
commented on May 19, 2024
我这是本地跑的docker，出现了这种错误，会不会是QQ有反爬虫机制，我看了redis，数据大部分获取到了，但是获取好友的时候就卡住了，总是剩十几个获取不到
from qqzonemood.
MiChongGET
commented on May 19, 2024
好的谢谢啦，最新代码提交了吗，我还想跑一跑代码
from qqzonemood.
Related Issues (20)
- Please attach methods to run HOT 2
- [安全问题]代码内可能包含敏感信息 HOT 2
- 爬取完后下载说说数据404 HOT 1
- 数据分析 HOT 1
- 登录 HOT 1
- 安装imread出现错误 HOT 5
- redis无法连接：Failed to connect redis: Error -2 connecting to redis:6379. Name or service not known. HOT 2
- 如果能分析指定日期之后的说说数据就好了，比如我想分析2017年至今的 HOT 1
- 如何爬取好友空间 HOT 4
- 发现不能下载QQ相册的照片 HOT 1
- 最新版本登录成功后提示未登录 HOT 2
- docker版本安装失败 HOT 1
- 使用python版本爬取他人空间意外退出 HOT 1
- No matching distribution found for numpy HOT 1
- 扫码登陆后提示由于网络等原因登陆失败，请重新扫描二维码或稍后再试 HOT 3
- 多线程会遗漏数据 HOT 2
- 获取第一次登陆时间失败 HOT 1
- 请问当前加了子项目的版本如何导出excel数据 HOT 1
- json以及csv文件 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qqzonemood.