Comments (4)
Same for the development version:
pip install 'numpy>=2.0.0rc1' hypothesis pytest numexpr
pip install --pre --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple pandas
python -c 'import pandas; pandas.test(extra_args=["-k", "test_eval and numexpr", "-l", "--tb=long"])'
running: pytest -k test_eval and numexpr -l --tb=long /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
configfile: pyproject.toml
plugins: hypothesis-6.100.2
collected 204134 items / 198621 deselected / 62 skipped / 5513 selected
...
============================================================================================= short test summary info ==============================================================================================
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-python] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-pandas] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
============================================================== 10 failed, 5501 passed, 62 skipped, 198621 deselected, 2 xpassed, 4 warnings in 45.77s ==============================================================
INSTALLED VERSIONS
------------------
commit : b4d3309e05e0afb7ee5bd671c2150d1e6eebbb88
python : 3.11.9.final.0
python-bits : 64
OS : Linux
OS-release : 6.8.7-1-default
Version : #1 SMP PREEMPT_DYNAMIC Thu Apr 18 07:12:38 UTC 2024 (5c0cf23)
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : de_DE.UTF-8
LOCALE : de_DE.UTF-8
pandas : 3.0.0.dev0+861.gb4d3309e05
numpy : 2.0.0rc1
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 65.5.0
pip : 24.0
Cython : None
pytest : 8.2.0
hypothesis : 6.100.2
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.10.0
odfpy : None
openpyxl : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None
from pandas.
Still an issue with pandas-3.0.0.dev0+991.ga3e751c6b4.
The reason is the explicit type repr for all scalar types in numpy 2:
https://numpy.org/devdocs/release/2.0.0-notes.html#representation-of-numpy-scalars-changed
s = '(df) / (np.float64(3.0))', types = {}, context = {'optimization': 'aggressive', 'truediv': False}, sanitize = True
def stringToExpression(s, types, context, sanitize: bool=True):
"""Given a string, convert it to a tree of ExpressionNode's.
"""
# sanitize the string for obvious attack vectors that NumExpr cannot
# parse into its homebrew AST. This is to protect the call to `eval` below.
# We forbid `;`, `:`. `[` and `__`, and attribute access via '.'.
# We cannot ban `.real` or `.imag` however...
# We also cannot ban `.\d*j`, where `\d*` is some digits (or none), e.g. 1.5j, 1.j
if sanitize:
no_whitespace = re.sub(r'\s+', '', s)
skip_quotes = re.sub(r'(\'[^\']*\')', '', no_whitespace)
if _blacklist_re.search(skip_quotes) is not None:
> raise ValueError(f'Expression {s} has forbidden control characters.')
E ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
context = {'optimization': 'aggressive', 'truediv': False}
no_whitespace = '(df)/(np.float64(3.0))'
s = '(df) / (np.float64(3.0))'
sanitize = True
skip_quotes = '(df)/(np.float64(3.0))'
types = {}
pandas_numexpr_numpy2/lib64/python3.11/site-packages/numexpr/necompiler.py:283: ValueError
from pandas.
numexpr
does not seem to be installed in the Numpy dev CI runs, so these tests are skipped there.
from pandas.
No longer an issue with pandas-3.0.0.dev0+1178.g236d89b856
Now how to find the correct change to backport to 2.x ...
from pandas.
Related Issues (20)
- BUG: read_sql tries to convert blob/varbinary to string with pyarrow backend HOT 2
- BUG: Pandas squashes 1-dimensional Numpy array with shape (1,) down to a 0-dimensional array HOT 7
- BUG: rolling window with `center=True, min_periods=1` is not symmetric at edges HOT 4
- DOC: Draft PDEPs appear as "Under discussion" in the roadmap
- BUG: Wrong timestamp resolution when parsing timestamp string with comma separated milliseconds HOT 5
- BUILD: HOT 1
- BUG: Series.mul silently returns wrong values with `UInt8` dtype when overflowing the max value range HOT 4
- ENH: Support right hand side operations with Series and custom classes HOT 1
- DOC: New contributor meeting, clarify or remove HOT 5
- BUILD: Pandas 1.5.3 is unusable due to incompatibility with Numpy 2.0.0 HOT 1
- BUG: DatetimeIndex intersection is empty or garbage HOT 4
- BUG: CoW does not seem to work on an index with duplicated labels HOT 10
- BUG: reset_index() looses the frequency of a DatetimeIndex HOT 4
- ENH: pd.concat with keys and ignore_index=True should raise HOT 4
- BUG: HOT 1
- BUG: Error on to_datetime() after running multiple times in jupyter notebook HOT 1
- PERF: Difference in using zipped pickle files HOT 4
- BUG: Error on query function when the column name has # symbol HOT 1
- BUG: Couldn't run sql: 'Connection' object has no attribute 'cursor' HOT 4
- Surprising behavior: set_index cannot set a MultiIndex from a tuple, only a list HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.