loadchange / amemv-crawler Goto Github PK
View Code? Open in Web Editor NEW🙌Easily download all the videos from TikTok(amemv).下载指定的 抖音(Douyin) 号的视频,抖音爬虫
License: MIT License
🙌Easily download all the videos from TikTok(amemv).下载指定的 抖音(Douyin) 号的视频,抖音爬虫
License: MIT License
如题 感谢 代码写的很好 学习了
怎么把视频下载的名字改了?
想改成 原生的视频名字+用户名
目前的方式有点难区别视频内容。
就像这样:
There's no video in number 89923118343.
Aweme number 89923118343, video number 0
大家有兴趣吗?人多的话,我就发布教程吧
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in validate_conn
conn.connect()
File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 356, in connect
ssl_context=context)
File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 359, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in init
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
timeout=timeout
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in validate_conn
conn.connect()
File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 356, in connect
ssl_context=context)
File "/usr/lib/python3.6/site-packages/urllib3/util/ssl.py", line 359, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in init
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "amemv-video-ripper.py", line 412, in
CrawlerScheduler(content)
File "amemv-video-ripper.py", line 112, in init
res = requests.get(url, headers=self.headers)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 495, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
Hi,我在爬用户信息时的也遇到了_signature字段,但是用fuck-byted-acrawler.js拿到的signature却用不了,请问下是怎么回事呢?
用户url: https://www.douyin.com/share/user/2613650662
抓数据的url: https://www.douyin.com/aweme/v1/aweme/post/?user_id=2613650662&count=21&max_cursor=0&aid=1128&_signature=vsISThAa5eS8bwWIdK-OVL7CEl
_signature生成的js: https://s3a.bytecdn.cn/ies/resource/falcon/douyin_falcon/pkg/third_ee38eac.js 里的/!douyin_falcon:node_modules/byted-acrawler/dist/runtime.js/
感谢!~
可以分享下类似这种加signature的破解方法吗?或者提供几个关键词,Thanks in advance.
您看这些优质却不火的小姐姐和小哥哥也很迷人的啊...
【https://www.baidu.com/s?wd=site:www.douyin.com 原声&rsv_spt=1&rsv_iqid=0xd18eb8e00001789c&issp=1&f=8&rsv_bp=1&rsv_idx=2&ie=utf-8&rqlang=cn&tn=baiduhome_pg&rsv_enter=1&oq=site:www.douyin.com&inputT=3571&rsv_t=2159rBGYGurBOOLUugTfccSGWHwY2jIWQGxIgSygjKHtJeJb9nu2T7+AZjaO/Ei0seKm&rsv_sug3=29&rsv_pq=ce3f516b0000f181&rsv_sug2=0&rsv_sug4=3936】
七月二号更新算法后,用新的fuck-byted-acrawler.js拿到的_signature有时能用有时失效,这回确定所用的headers 和amemv-video-ripper.py中的header一致,请问可能是什么原因呢?Thanks~
通过抖音号获取视频列表的接口是不是变了?
运行环境 node ,用了大神的算_signature的js 发现有很大概率返回的列表是空值
可以下载大概十几个视频,但接下来就会报错,查了一下好像是超过最大递归次数,但深层原因就不知道了
Traceback (most recent call last):
File "amemv-video-ripper.py", line 412, in
CrawlerScheduler(content)
File "amemv-video-ripper.py", line 128, in init
self.scheduling()
File "amemv-video-ripper.py", line 150, in scheduling
self.download_challenge_videos(challenge)
File "amemv-video-ripper.py", line 165, in download_challenge_videos
video_count = self._download_challenge_media(challenge)
File "amemv-video-ripper.py", line 309, in _download_challenge_media
video_count = get_aweme_list()
File "amemv-video-ripper.py", line 305, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
File "amemv-video-ripper.py", line 305, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
File "amemv-video-ripper.py", line 305, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
[Previous line repeated 947 more times]
File "amemv-video-ripper.py", line 298, in get_aweme_list
res = requests.get(url, headers=self.headers)
File "D:\Anaconda3\lib\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "D:\Anaconda3\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "D:\Anaconda3\lib\site-packages\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "D:\Anaconda3\lib\site-packages\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "D:\Anaconda3\lib\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "D:\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "D:\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "D:\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "D:\Anaconda3\lib\site-packages\urllib3\connection.py", line 337, in connect
cert = self.sock.getpeercert()
File "D:\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 348, in getpeercert
'subjectAltName': get_subj_alt_name(x509)
File "D:\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 196, in get_subj_alt_name
ext = cert.extensions.get_extension_for_class(
File "D:\Anaconda3\lib\site-packages\cryptography\utils.py", line 158, in inner
result = func(instance)
File "D:\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\x509.py", line 137, in extensions
self._backend, self._x509
File "D:\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\decode_asn1.py", line 249, in parse
value = handler(backend, ext_data)
File "D:\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\decode_asn1.py", line 428, in _decode_subject_alt_name
_decode_general_names_extension(backend, ext)
File "D:\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 1008, in init
self._general_names = GeneralNames(general_names)
File "D:\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 964, in init
if not all(isinstance(x, GeneralName) for x in general_names):
File "D:\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 964, in
if not all(isinstance(x, GeneralName) for x in general_names):
File "D:\Anaconda3\lib\abc.py", line 182, in instancecheck
if subclass in cls._abc_cache:
File "D:\Anaconda3\lib_weakrefset.py", line 75, in contains
return wr in self.data
RecursionError: maximum recursion depth exceeded in comparison
视频可以正常下载,但是下载不到图片呀
请问能够爬取视频属性信息吗?比如什么时候发布的,视频宽度和高度,点赞数,评论数,播放数这些
短链接下载不到用户喜欢的视频!发布的视频可以正常下载,就是下载不到喜欢的视频
怎么能爬到抖音号?
以该链接为例子 http://v.douyin.com/cE1j4/
在浏览器中输入后会出现 https://www.douyin.com/share/user/64395924642/?share_type=link
这时候把
https://www.douyin.com/share/user/64395924642/?
去掉末尾的斜杠 /
变成 https://www.douyin.com/share/user/64395924642? 即可批量下载
挑战和音乐同理!
认为是抖音虽然更新但是网页中的长连接并没有太大改变,因此可以继续使用。
challenge_id = challenge_info.get('cid')
AttributeError: 'NoneType' object has no attribute 'get'
不错,运行了。已经star,建议说明一下python版本
安卓分享的链接:https://www.douyin.com/share/user/92683516474/?share_type=link
苹果分享的链接:https://www.douyin.com/share/user/92683516474?share_type=link
需要将安卓分享的链接参数钱的斜杠‘/’去除,才可以正常解析页面
'node' �����ڲ����ⲿ���Ҳ���ǿ����еij���
���������ļ���
Traceback (most recent call last):
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 411, in
CrawlerScheduler(content)
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 128, in init
self.scheduling()
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 150, in scheduling
self.download_challenge_videos(challenge)
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 165, in download_challenge_videos
video_count = self._download_challenge_media(challenge)
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 276, in _download_challenge_media
signature = self.generateSignature(str(challenge_id))
File "C:/Users/500/Desktop/github/amemv-crawler-master/amemv-crawler-master/amemv-video-ripper.py", line 138, in generateSignature
return p.readlines()[0]
IndexError: list index out of range
我把你的fuck-byted-acrawler.js下载到我本地,用下面的命令生成signature
node fuck-byted-acrawler.js 58585956426
然后带入到url里面
https://www.douyin.com/aweme/v1/aweme/post/?user_id=58585956426&count=21&max_cursor=0&aid=1128&_signature=8Z4HuwAAq4AOYfhE08pWY.GeB6
返回结果为空数组
你生成signature字段的JS是怎么找到并扒下来的?
当以用户模板(share/user)抓取的时候,连续抓取超过20个模板后,就会被返回空字符串。需要等待一段时间后再抓取,比如一小时。
但是音乐模板并不会被限制。
fuck byted-acrawler什么意思,node小白。。。。
7.2号晚上更新了算法,现在怎么破
7.2号晚上更新了算法,现在怎么破
7.2号晚上更新了算法,现在怎么破
/!douyin_falcon:node_modules/byted-acrawler/dist/runtime.js/
__M.define("douyin_falcon:node_modules/byted-acrawler/dist/runtime",function(l,e){Function(function(l){return'�e(e,a,r){�(b[e]||(b[e]=t("x,y","�x "+e+" y"�)(r,a)}�a(e,a,r){�(k[r]||(k[r]=t("x,y","�new xy"�)(e,a)}�r(e,a,r){�n,t,s={},b=s.d=r?r.d+1:0;for(s["$"+b]=s,t=0;t<b;t�)s[n="$"+t]=r[n];for(t=0,b=s�=a�;t<b;t�)s[t]=a[t];�c(e,0,s)}�c(t,b,k){�u(e){v[x�]=e}�f�{�g=�,t�ing(b�g)}�l�{try{y=c(t,b,k)}catch(e){h=e,y=l}}for(�h,y,d,g,v=[],x=0;;)switch(g=�){case 1:u(!�)�4:�f��5:u(�(e){�a=0,r=e�;���{�c=a<r;�c&&u(e[a�]),c}}(���6:y=�,u(�(y��8:if(g=�,l��g,g=�,y===c)b+=g;else if(y!==l)�y�9:�c�10:u(s(���11:y=�,u(�+y)�12:for(y=f�,d=[],g=0;g<y�;g�)d[g]=y.charCodeAt(g)^g+y�;u(String.fromCharCode.apply(null,d��13:y=�,h=delete �[y]�14:���59:u((g=�)?(y=x,v.slice(x-=g,y�:[])�61:u(�[�])�62:g=�,k[0]=65599k[0]+k[1].charCodeAt(g)>>>0�65:h=�,y=�,�[y]=h�66:u(e(t[b�],�,���67:y=�,d=�,u((g=�).x===c?r(g.y,y,k):g.apply(d,y��68:u(e((g=t[b�])<"<"?(b--,f�):g+g,�,���70:u(!1)�71:�n�72:�+f��73:u(parseInt(f�,36��75:if(�){b��case 74:g=�<<16>>16�g�76:u(k[�])�77:y=�,u(�[y])�78:g=�,u(a(v,x-=g+1,g��79:g=�,u(k["$"+g])�81:h=�,�[f�]=h�82:u(�[f�])�83:h=�,k[�]=h�84:�!0�85:�void 0�86:u(v[x-1])�88:h=�,y=�,�h,�y�89:u(��{�e�{�r(e.y,arguments,k)}�e.y=f�,e.x=c,e}�)�90:�null�91:�h�93:h=��0:��;default:u((g<<16>>16)-16)}}�n=this,t=n.Function,s=Object.keys||�(e){�a={},r=0;for(�c in e)a[r�]=c;�a�=r,a},b={},k={};�r'.replace(/[�-�]/g,function(e){return l[15&e.charCodeAt(0)]})}("v[x++]=�v[--x]�t.charCodeAt(b++)-32�function �return �))�++�.substr�var �.length�()�,b+=�;break;case �;break}".split("�")))()('gr$Daten Иb/s!l y͒yĹg,(lfi~ah`{mv,-n|jqewVxp{rvmmx,&eff�kx[!cs"l".Pq%widthl"@q&heightl"vrgetContextx$"2d[!cs#l#,;?|u.|uc{uq$fontl#vr(fillTextx$$龘ฑภ경2<[#c}l#2qshadowBlurl#1q-shadowOffsetXl#$$limeq+shadowColorl#vr#arcx88802[%c}l#vr&strokex[ c}l"v,)}eOmyoZB]mx[ cs!0s$l$Pb<k7l l!r&lengthb%^l$1+s$j�l s#i$1ek1s$gr#tack4)zgr#tac$! +0o![#cj?o ]!l$b%s"o ]!l"l$bb^0d#>>>s!0s%yA0s"l"l!r&lengthb<k+l"^l"1+s"j�l s&l&z0l!$ +["cs'(0l#i'1ps9wxb&s() &{s)/s(gr&Stringr,fromCharCodes)0syWl ._b&s o!])l l Jb<k$.aj;l .Tb<k$.gj/l .^b<k&i"-4j!�+& s+yPo!]+s!l!l Hd>&l!l Bd>&+l!l &+l!l 6d>&+l!l &+ s,y=o!o!]/q"13o!l q"10o!],l 2d>& s.{s-yMo!o!]0q"13o!]*Ld<l 4d#>>>b|s!o!l q"10o!],l!& s/yIo!o!].q"13o!],o!]*Jd<l 6d#>>>b|&o!]+l &+ s0l-l!&l-l!i'1z141z4b/@d<l"b|&+l-l(l!b^&+l-l&zl'g,)gk}ejo{�cm,)|ynLijem["cl$b%@d<l&zl'l $ +["cl$b%b|&+l-l%8d<@b|l!b^&+ q$sign ',[Object.defineProperty(e,"__esModule",{value:!0})])});
如题,请问有什么解决办法么
https://www.douyin.com/share/video/6557825773007277319/?mid=6557826301539912456
现在的视频分享链接都需要 mid 这个参数才能开启,但 aweme/post 返回的 share_url 是不带这个参数的,请问有没有不用抖音分享就能取得 mid 的方式?我知道与这个 project 无关,但是找不到地方发问。 。
看到这个repo后第一时间下载下来配置并使用,效果很好,没有水印,速度也很快,作者辛苦!
同时发现一个问题:一个用户可能自己的作品不多,但是他喜欢的作品超多(比如我2333),就导致爬虫工作的大部分时间都是在下用户的收藏。但是很多时候我们只需要该作者的作品就够了。
所以是否可以加一个不下载用户收藏的功能,比如 命令类似这样:python amemv-video-ripper.py -nofavorite
最后再次感谢作者的付出,请作者收下我们的膝盖 ^_^
challenge_info = self._search(challenge, 'challenge')
这里每次获取的为None了
看不懂 js,请问该怎么修正?谢谢。
环境:windows 7, python 3.7
E:\Code\Python3\amemv-crawler-master\fuck-byted-acrawler.js:5
let e = {}
^
SyntaxError: Unexpected identifier
at exports.runInThisContext (vm.js:73:16)
at Module._compile (module.js:443:25)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3
Traceback (most recent call last):
File ".\amemv-video-ripper.py", line 412, in
CrawlerScheduler(content)
File ".\amemv-video-ripper.py", line 128, in init
self.scheduling()
File ".\amemv-video-ripper.py", line 147, in scheduling
self.download_videos(params)
File ".\amemv-video-ripper.py", line 159, in download_videos
video_count = self._download_user_media(number, dytk)
File ".\amemv-video-ripper.py", line 201, in _download_user_media
signature = self.generateSignature(str(user_id))
File ".\amemv-video-ripper.py", line 138, in generateSignature
return p.readlines()[0]
IndexError: list index out of range
如题
系统给的id不能用来抓取了吗,怎么感觉现在像是通过分享动态生成一个非常短时效的id从接口来拿数据
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.amemv.com', port=443): Max retries exceeded with url: /aweme/v1/challenge/search/?ac=WIFI&app_name=aweme&vid=2ED370A7-F09C-4C9E-90F5-872D57F3127C&as=a1c5600cb7576a7e273418&device_type=iPhone8,2&os_api=18&build_number=17805&version_code=1.7.8&ts=1524105474&app_version=1.7.8&channel=App%20Store&device_platform=iphone&mas=008c37d4eaf9b158c3d1b7e3fc0d66008dc45306aae0ff5380d6a8&screen_width=1242&search_source=challenge&iid=28175672430&idfa=00000000-0000-0000-0000-000000000000&openudid=20dae85eeac1da35a69e2a0ffeaeef41c78a2e97&device_id=46166717995&count=20&keyword=%E7%BE%8E%E9%A3%9F%E7%BE%8E%E9%A3%9F%E7%BE%8E%E9%A3%9F&cursor=0&aid=1128 (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),))
换了下同事的一个cookie也一样哦
Downloading v0200f9a0000bc1ahsmbn5vdr6i8f2h0.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=v0200f9a0000bc1ahsmbn5vdr6i8f2h0&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading ba5798aee12b4bff801598d4abf5cb99.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=ba5798aee12b4bff801598d4abf5cb99&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading 4b784f8aaef2407f935a40e18a6a8811.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=4b784f8aaef2407f935a40e18a6a8811&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading f9d5c6cde2fa418f883fc65793c8280d.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=f9d5c6cde2fa418f883fc65793c8280d&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading f88b386af271417987be0b6df7a6065e.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=f88b386af271417987be0b6df7a6065e&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading v0200fbd0000bcjo8tcthbi90j14pcag.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=v0200fbd0000bcjo8tcthbi90j14pcag&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading v0200f9a0000bck76427u0r58fotk7ug.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=v0200f9a0000bck76427u0r58fotk7ug&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Downloading v0200fbd0000bc6kaaelg9jt2h43h2ng.mp4 from https://aweme.snssdk.com/aweme/v1/play/?video_id=v0200fbd0000bc6kaaelg9jt2h43h2ng&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0.
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 196, in get_subj_alt_name
ext = cert.extensions.get_extension_for_class(
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\utils.py", line 159, in inner
result = func(instance)
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\x509.py", line 138, in extensions
self._backend, self._x509
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\decode_asn1.py", line 238, in parse
value = handler(backend, ext_data)
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\decode_asn1.py", line 417, in _decode_subject_alt_name
_decode_general_names_extension(backend, ext)
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 1210, in init
self._general_names = GeneralNames(general_names)
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 1163, in init
if not all(isinstance(x, GeneralName) for x in general_names):
File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\x509\extensions.py", line 1163, in
if not all(isinstance(x, GeneralName) for x in general_names):
File "C:\ProgramData\Anaconda3\lib\abc.py", line 182, in instancecheck
if subclass in cls._abc_cache:
File "C:\ProgramData\Anaconda3\lib_weakrefset.py", line 75, in contains
return wr in self.data
RecursionError: maximum recursion depth exceeded in comparison
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 379, in
CrawlerScheduler(content)
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 119, in init
self.scheduling()
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 131, in scheduling
self.download_challenge_videos(challenge)
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 143, in download_challenge_videos
video_count = self._download_challenge_media(challenge)
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 280, in _download_challenge_media
video_count = get_aweme_list()
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 276, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 276, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 276, in get_aweme_list
return get_aweme_list(contentJson.get('cursor'), video_count)
[Previous line repeated 947 more times]
File "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py", line 269, in get_aweme_list
res = requests.get(url, headers=self.headers)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connection.py", line 337, in connect
cert = self.sock.getpeercert()
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 348, in getpeercert
'subjectAltName': get_subj_alt_name(x509)
File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 202, in get_subj_alt_name
except (x509.DuplicateExtension, x509.UnsupportedExtension,
AttributeError: module 'cryptography.x509' has no attribute 'UnsupportedExtension'
[Finished in 331.1s with exit code 1]
[shell_cmd: python -u "E:\amemv-crawler-master\amemv-crawler-master\amemv-video-ripper.py"]
[dir: E:\amemv-crawler-master\amemv-crawler-master]
[path: C:\Program Files\curl-7.60.0\I386;C:\ffmpeg\bin;C:\ProgramData\Anaconda3;C:\ProgramData\Anaconda3\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin]
“_signature,该字段是由
douyin_falcon:node_modules/byted-acrawler/dist/runtime
生成的”, 请问这个你是如何分析出来的?有点好奇
fuck-byted-acrawler.js 的js打码是你自己改的吗? 我跳过去有乱码。。。。
__M.define("douyin_falcon:node_modules/byted-acrawler/dist/runtime", function(l, e) {
Function(function(l) {
return '�e(e,a,r){�(b[e]||(b[e]=t("x,y","�x "+e+" y"�)(r,a)}�a(e,a,r){�(k[r]||(k[r]=t("x,y","�new xy"�)(e,a)}�r(e,a,r){�n,t,s={},b=s.d=r?r.d+1:0;for(s["$"+b]=s,t=0;t<b;t�)s[n="$"+t]=r[n];for(t=0,b=s�=a�;t<b;t�)s[t]=a[t];�c(e,0,s)}�c(t,b,k){�u(e){v[x�]=e}�f�{�g=�,t�ing(b�g)}�l�{try{y=c(t,b,k)}catch(e){h=e,y=l}}for(�h,y,d,g,v=[],x=0;;)switch(g=�){case 1:u(!�)�4:�f��5:u(�(e){�a=0,r=e�;���{�c=a<r;�c&&u(e[a�]),c}}(���6:y=�,u(�(y��8:if(g=�,l��g,g=�,y===c)b+=g;else if(y!==l)�y�9:�c�10:u(s(���11:y=�,u(�+y)�12:for(y=f�,d=[],g=0;g<y�;g�)d[g]=y.charCodeAt(g)^g+y�;u(String.fromCharCode.apply(null,d��13:y=�,h=delete �[y]�14:���59:u((g=�)?(y=x,v.slice(x-=g,y�:[])�61:u(�[�])�62:g=�,k[0]=65599k[0]+k[1].charCodeAt(g)>>>0�65:h=�,y=�,�[y]=h�66:u(e(t[b�],�,���67:y=�,d=�,u((g=�).x===c?r(g.y,y,k):g.apply(d,y��68:u(e((g=t[b�])<"<"?(b--,f�):g+g,�,���70:u(!1)�71:�n�72:�+f��73:u(parseInt(f�,36��75:if(�){b��case 74:g=�<<16>>16�g�76:u(k[�])�77:y=�,u(�[y])�78:g=�,u(a(v,x-=g+1,g��79:g=�,u(k["$"+g])�81:h=�,�[f�]=h�82:u(�[f�])�83:h=�,k[�]=h�84:�!0�85:�void 0�86:u(v[x-1])�88:h=�,y=�,�h,�y�89:u(��{�e�{�r(e.y,arguments,k)}�e.y=f�,e.x=c,e}�)�90:�null�91:�h�93:h=��0:��;default:u((g<<16>>16)-16)}}�n=this,t=n.Function,s=Object.keys||�(e){�a={},r=0;for(�c in e)a[r�]=c;�a�=r,a},b={},k={};�r'.replace(/[�-�]/g, function(e) {
return l[15 & e.charCodeAt(0)]
})
}("v[x++]=�v[--x]�t.charCodeAt(b++)-32�function �return �))�++�.substr�var �.length�()�,b+=�;break;case �;break}".split("�")))()('gr$Daten Иb/s!l y͒yĹg,(lfi~ah`{mv,-n|jqewVxp{rvmmx,&eff�kx[!cs"l".Pq%widthl"@q&heightl"vrgetContextx$"2d[!cs#l#,;?|u.|uc{uq$fontl#vr(fillTextx$$龘ฑภ경2<[#c}l#2qshadowBlurl#1q-shadowOffsetXl#$$limeq+shadowColorl#vr#arcx88802[%c}l#vr&strokex[ c}l"v,)}eOmyoZB]mx[ cs!0s$l$Pb<k7l l!r&lengthb%^l$1+s$j�l s#i$1ek1s$gr#tack4)zgr#tac$! +0o![#cj?o ]!l$b%s"o ]!l"l$bb^0d#>>>s!0s%yA0s"l"l!r&lengthb<k+l"^l"1+s"j�l s&l&z0l!$ +["cs'(0l#i'1ps9wxb&s() &{s)/s(gr&Stringr,fromCharCodes)0syWl ._b&s o!])l l Jb<k$.aj;l .Tb<k$.gj/l .^b<k&i"-4j!�+& s+yPo!]+s!l!l Hd>&l!l Bd>&+l!l &+l!l 6d>&+l!l &+ s,y=o!o!]/q"13o!l q"10o!],l 2d>& s.{s-yMo!o!]0q"13o!]*Ld<l 4d#>>>b|s!o!l q"10o!],l!& s/yIo!o!].q"13o!],o!]*Jd<l 6d#>>>b|&o!]+l &+ s0l-l!&l-l!i'1z141z4b/@d<l"b|&+l-l(l!b^&+l-l&zl'g,)gk}ejo{�cm,)|ynLijem["cl$b%@d<l&zl'l $ +["cl$b%b|&+l-l%8d<@b|l!b^&+ q$sign ', [Object.defineProperty(e, "__esModule", {
value: !0
})])
});
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.