zsrl / pywencai Goto Github PK
View Code? Open in Web Editor NEW获取同花顺问财数据
License: MIT License
获取同花顺问财数据
License: MIT License
您好,不使用jsdom 有办法能获取到最新的hexin-v请求头信息吗?因为用的是java无法加载node环境...
问题
在全局中使用logging的basicConfig,与pywencai库冲突,全局样式不生效。
建议
建议在库中使用logger替代basicConfig来避免。
e.g.1:
import logging
import pywencai
logging.basicConfig(level=logging.INFO,
filename='work.log')
无法生成work.log
e.g.2:
case1
import logging
logging.basicConfig(level=logging.INFO)
logging.info(1)
out: [pywencai] 2023-05-15 04:16:22,364 - INFO - 1
case2
import logging
logging.basicConfig(level=logging.INFO)
logging.info(1)
out: INFO:root:1
首先,非常感谢你的开源!
我的程序从2023年2月份一直在使用pywencai,但到2023年8月8号左右,无法返回数据。不知道是不是问财加强了反爬措施。
import pywencai
try:
# Get a list of delisted stocks sorted by delisting date in ascending order
res = pywencai.get(query='市盈率<35;市净率<1.8;每股收益增长率>0;利润增长率>0;流动比率>1.8;速动比率>1;市值从小到大排序;非ST;非创业;非科创;非北证', sort_order='asc', loop=True, log=True)
# Print the result
print(res)
except Exception as e:
# Handle any exceptions that may occur
print(f"Error: {e}")
[pywencai] 2023-08-28 22:11:29,417 - INFO - 获取condition开始
[pywencai] 2023-08-28 22:11:29,496 - WARNING - 1次尝试失败
[pywencai] 2023-08-28 22:11:29,497 - WARNING - 2次尝试失败
[pywencai] 2023-08-28 22:11:29,499 - WARNING - 3次尝试失败
[pywencai] 2023-08-28 22:11:29,502 - WARNING - 4次尝试失败
[pywencai] 2023-08-28 22:11:29,505 - WARNING - 5次尝试失败
[pywencai] 2023-08-28 22:11:29,507 - WARNING - 6次尝试失败
[pywencai] 2023-08-28 22:11:29,509 - WARNING - 7次尝试失败
[pywencai] 2023-08-28 22:11:29,513 - WARNING - 8次尝试失败
[pywencai] 2023-08-28 22:11:29,514 - WARNING - 9次尝试失败
[pywencai] 2023-08-28 22:11:29,516 - WARNING - 10次尝试失败
[pywencai] 2023-08-28 22:11:29,516 - INFO - 获取get_robot_data失败
Error: 'NoneType' object has no attribute 'get'
pywencai.get()返回的查询结果无法解析为df,初步分析应该是2024.1.10日i问财官方页面更新了查询结果展示页面,导致解析查询结果数据出问题。
比如,我调用get()返回的df.columns,报错如下:AttributeError: 'dict' object has no attribute 'columns'.
经过初步分析,返回的数据类型为dict,形如:
<class 'dict'>
{'请填写主题': '
为你找到5348只符合条件的A股,具体如下:
', 'xuangu_tableV1': .......据此分析是由于i问财官方页面更新了查询结果展示页面,导致pywencai的get接口解析html结果出错,无法生成df。不妥之处请指正。
感谢大佬们分享的pywencai,期待大佬们的版本更新,谢谢!
大佬,这个Hexin-V的生成规则改变了?
我尝试用headers.get_token()得到的值在apifox中替代,得到下面结果,换网络也无济于事
<h1>Nginx forbidden.</h1> <br/>request info: 106.61.251.48
AttributeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_10512\1512741554.py in
E:\DevEnvs\svn_hquant\jupyter\Global_Functions.py in h_THS_CacheStockBlock()
1962 importlib.reload(pywencai)
1963
-> 1964 res = pywencai.get(question='已省略;', sort_key='股票代码', sort_order='asc',
1965 # sleep=1,
1966 loop=True, log=True,)
E:\DevEnvs\github\pywencai\pywencai\wencai.py in get(loop, **kwargs)
180 kwargs = {replace_key(key): value for key, value in kwargs.items()}
181 params = get_robot_data(**kwargs)
--> 182 data = params.get('data')
183 condition = _.get(data, 'condition')
184 if condition is not None:
AttributeError: 'NoneType' object has no attribute 'get'
已经更新到了最新版本
报错如下:
/usr/local/lib/python3.8/site-packages/pywencai/hexin-v.bundle.js:102785
this[filesSymbol] ||= FileList.createImpl(this._globalObject);
^^^
SyntaxError: Unexpected token '||='
at wrapSafe (internal/modules/cjs/loader.js:979:16)
at Module._compile (internal/modules/cjs/loader.js:1027:27)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
at internal/main/run_main_module.js:17:47
/usr/local/lib/python3.8/site-packages/pywencai/hexin-v.bundle.js:102785
this[filesSymbol] ||= FileList.createImpl(this._globalObject);
本人属小白,看到这个项目在测试时出现以下错误
C:\Users\ASUS\Scripts\python.exe D:/A股相关/test.py
Traceback (most recent call last):
File "D:\A股相关\test.py", line 2, in
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
File "C:\Users\ASUS\lib\site-packages\pywencai\wencai.py", line 68, in get
return getPage(**kwargs)
File "C:\Users\ASUS\lib\site-packages\pywencai\wencai.py", line 32, in getPage
list = result['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']
KeyError: 'data'
电脑已经安装了Node.js
设置代理,防止触发反爬虫机制:
大佬请增加proxy代理IP和端口功能。感谢
一直以来可以正确使用pywencai,但突然发现无法获取数据了。比如pywencai在pypi.org的范例如下:
import pywencai
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
print(res)
运行后出错:
Traceback (most recent call last):
File "/Users/usr/git/div_invest/venv/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3548, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-3-93879abb35df>", line 3, in <module>
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/usr/git/div_invest/venv/lib/python3.11/site-packages/pywencai/wencai.py", line 185, in get
data = params.get('data')
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
npm install jsdom
现在是jsdom安装了,验证node -e "require('jsdom')"成功
Jupyterlab运行, 还是一样的'NoneType' object has no attribute 'get'。
出现 返回类型未dict
返回Key: 请填写主题
返回Key: xuangu_tableV1
返回Key: 问财选可转债
返回Key: container
数据框在 xuangu_tableV1里,但仅仅只能显示10个数据
key: 问财选可转债 q ... 股票代码
0 问题内容 ...
container {'btn1': q ... 股票代码
0 问题内容 ...
python -V
Python 3.12.3
python test.py
Traceback (most recent call last):
File "/root/test.py", line 2, in
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/pywencai/wencai.py", line 185, in get
data = params.get('data')
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
爬出来的数据为空,前几天还可以用。
python>pip --trusted-host pypi.org install pywencai
Collecting pywencai
Obtaining dependency information for pywencai from https://files.pythonhosted.org/packages/77/4b/28d33a6edb8d98402789cbf418063a302a96d04d563163aef07ef5f97f22/pywencai-0.12.2-py3-none-any.whl.metadata
Using cached pywencai-0.12.2-py3-none-any.whl.metadata (4.7 kB)
Collecting PyExecJS<2.0.0,>=1.5.1 (from pywencai)
Using cached PyExecJS-1.5.1.tar.gz (13 kB)
Preparing metadata (setup.py) ... done
Collecting fake-useragent<2.0.0,>=1.1.1 (from pywencai)
Obtaining dependency information for fake-useragent<2.0.0,>=1.1.1 from https://files.pythonhosted.org/packages/e4/99/60d8cf1b26938c2e0a57e232f7f15641dfcd6f8deda454d73e4145910ff6/fake_useragent-1.5.1-py3-none-any.whl.metadata
Using cached fake_useragent-1.5.1-py3-none-any.whl.metadata (15 kB)
Collecting pandas<3.0.0,>=1.5.0 (from pywencai)
Using cached pandas-2.2.2.tar.gz (4.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [13 lines of output]
+ meson setup C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766 C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build\meson-python-native-file.ini
The Meson build system
Version: 1.2.1
Source dir: C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766
Build dir: C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766.mesonpy-1ucg6z77\build
Build type: native build
Project name: pandas
Project version: 2.2.2
Activating VS 15.9.58
..\..\meson.build:2:0: ERROR: Value "c11" (of type "string") for combo option "C language standard to use" is not one of the choices. Possible choices are (as string): "none", "c89", "c99", "gnu89", "gnu90", "gnu9x", "gnu99".
A full log can be found at C:\Users\huawei\AppData\Local\Temp\pip-install-r2875kob\pandas_fb70a8cea06e471e94befdc61303e766\.mesonpy-1ucg6z77\build\meson-logs\meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
python>node -v
v16.16.0
python>python
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:30:09) [MSC v.1934 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
今天又试了一下,又不行了,似乎跟nodejs版本无关,请问生成的hexin-v能使用的概率高吗
Originally posted by @shenjing023 in #6 (comment)
File "/opt/anaconda3/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/Code/BC/app.py", line 158, in home
res = pywencai.get(query='退市股票')
File "/opt/anaconda3/lib/python3.9/site-packages/pywencai/wencai.py", line 180, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'
原有问财是可以查到数据的
res = pywencai.get(query='浸没式液冷')
print(res)
![1](https://github.com/zsrl/pywencai/assets/140318
701/d61e2ea5-e8fc-4656-99ec-27733f2d9449)
res = pywencai.get(query='退市股票', sort_key='退市@退市日期', sort_order='asc')
print(res)
可以直接取get-robot-data接口里的datas数据
我用nodejs --version=12.x版本的时候生成的hexin-v是错误的,后来升级nodejs为最新版才解决
res = pywencai.get(questiion='退市股票')
print(res)
注意如果之前 脚本长时间用 原来的
res = pywencai.get(query='退市股票')
print(res)
可能会被封ip一段时间
File "C:\Users\89731\Desktop\code\iwencai\jinri.py", line 18, in mian
page_data = pw.get(query=query_str, perpage=100, sleep=0.5,loop=True)
File "C:\ProgramData\anaconda3\lib\site-packages\pywencai\wencai.py", line 182, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'
这个错误好玄学,刚安装那天重启下解决了,然后今天安装了几个新库,然后突然又报这个错误
Traceback (most recent call last):
File "stock_wordpress_all.py", line 98, in
stock_post_creator.create_post()
File "stock_wordpress_all.py", line 60, in create_post
res_stock = pywencai.get(query=f"{self.today_wc_q} 涨停股票 剔除ST *ST股票")
File "C:\Program Files\python\lib\site-packages\pywencai\wencai.py", line 182, in get
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'
我 self.today_wc_q 这个数字就是当天的日期 例如20XX-XX-XX的格式。
是固定的,还是自定义?
好像又不能正常获取数据了
data = params.get('data')
AttributeError: 'NoneType' object has no attribute 'get'
res = pywencai.get(query='多方炮')
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/lib/python3.11/site-packages/pywencai/wencai.py", line 182, in get
data = params.get('data')
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
loop=True死循环了
现在运行不能获取数据了,是不是失效了?
如题
有办法固定数据类型吗,通过问句或者参数
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.