adultfree/pornhub

Pornhub网站爬虫

pornhub's Issues

help！ my hero

help！the videos download are 108p，its so sad to me，and the to-ralate seem to cant use the all videos just like GIF，i cant solve this problem

谢谢大佬的程序，这里提三个建议

1. 现在下载的时视频都是不带原标题的，希望能让视频以原标题方式保留；
2. 希望能下载搜索结果页的视频
3. 希望能选择分页下载

报错了,能看下怎么改吗?

2021-07-01 19:49:33 [scrapy.utils.log] INFO: Scrapy 2.5.0 started (bot: pornhub)
2021-07-01 19:49:33 [scrapy.utils.log] INFO: Versions: lxml 4.6.3.0, libxml2 2.9.10, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 21.2.0, Python 3.8.5 (default, May 27 2021, 13:30:53) - [GCC 9.3.0], pyOpenSSL 20.0.1 (OpenSSL 1.1.1k 25 Mar 2021), cryptography 3.4.7, Platform Linux-5.8.0-59-generic-x86_64-with-glibc2.29
2021-07-01 19:49:33 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'pornhub',
'CONCURRENT_REQUESTS': 8,
'COOKIES_ENABLED': False,
'DEPTH_LIMIT': 1,
'DOWNLOAD_MAXSIZE': 0,
'DOWNLOAD_TIMEOUT': 10000,
'DOWNLOAD_WARNSIZE': 0,
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'pornhub.spiders',
'RETRY_TIMES': 10,
'SPIDER_MODULES': ['pornhub.spiders']}
2021-07-01 19:49:33 [scrapy.extensions.telnet] INFO: Telnet Password: cb822b11ec2c6a7a
2021-07-01 19:49:33 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.logstats.LogStats']
2021-07-01 19:49:33 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2021-07-01 19:49:33 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2021-07-01 19:49:33 [py.warnings] WARNING: /home/wangyifan/.local/lib/python3.8/site-packages/scrapy/pipelines/media.py:135: ScrapyDeprecationWarning: file_path(self, request, response=None, info=None) is deprecated, please use file_path(self, request, response=None, info=None, *, item=None)
self._check_signature(func)

2021-07-01 19:49:33 [scrapy.middleware] INFO: Enabled item pipelines:
['pornhub.pipelines.pornhubFilesPipeline']
2021-07-01 19:49:33 [scrapy.core.engine] INFO: Spider opened
2021-07-01 19:49:33 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2021-07-01 19:49:33 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2021-07-01 19:50:33 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2021-07-01 19:51:33 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
翻墙了并且可以看到pornhub。。。。。。。