Comments (5)
Same for the development version:
pip install 'numpy>=2.0.0rc1' hypothesis pytest numexpr
pip install --pre --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple pandas
python -c 'import pandas; pandas.test(extra_args=["-k", "test_eval and numexpr", "-l", "--tb=long"])'
running: pytest -k test_eval and numexpr -l --tb=long /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
configfile: pyproject.toml
plugins: hypothesis-6.100.2
collected 204134 items / 198621 deselected / 62 skipped / 5513 selected
...
============================================================================================= short test summary info ==============================================================================================
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-python] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-pandas] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
============================================================== 10 failed, 5501 passed, 62 skipped, 198621 deselected, 2 xpassed, 4 warnings in 45.77s ==============================================================
INSTALLED VERSIONS
------------------
commit : b4d3309e05e0afb7ee5bd671c2150d1e6eebbb88
python : 3.11.9.final.0
python-bits : 64
OS : Linux
OS-release : 6.8.7-1-default
Version : #1 SMP PREEMPT_DYNAMIC Thu Apr 18 07:12:38 UTC 2024 (5c0cf23)
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : de_DE.UTF-8
LOCALE : de_DE.UTF-8
pandas : 3.0.0.dev0+861.gb4d3309e05
numpy : 2.0.0rc1
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 65.5.0
pip : 24.0
Cython : None
pytest : 8.2.0
hypothesis : 6.100.2
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.10.0
odfpy : None
openpyxl : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None
from pandas.
Still an issue with pandas-3.0.0.dev0+991.ga3e751c6b4.
The reason is the explicit type repr for all scalar types in numpy 2:
https://numpy.org/devdocs/release/2.0.0-notes.html#representation-of-numpy-scalars-changed
s = '(df) / (np.float64(3.0))', types = {}, context = {'optimization': 'aggressive', 'truediv': False}, sanitize = True
def stringToExpression(s, types, context, sanitize: bool=True):
"""Given a string, convert it to a tree of ExpressionNode's.
"""
# sanitize the string for obvious attack vectors that NumExpr cannot
# parse into its homebrew AST. This is to protect the call to `eval` below.
# We forbid `;`, `:`. `[` and `__`, and attribute access via '.'.
# We cannot ban `.real` or `.imag` however...
# We also cannot ban `.\d*j`, where `\d*` is some digits (or none), e.g. 1.5j, 1.j
if sanitize:
no_whitespace = re.sub(r'\s+', '', s)
skip_quotes = re.sub(r'(\'[^\']*\')', '', no_whitespace)
if _blacklist_re.search(skip_quotes) is not None:
> raise ValueError(f'Expression {s} has forbidden control characters.')
E ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
context = {'optimization': 'aggressive', 'truediv': False}
no_whitespace = '(df)/(np.float64(3.0))'
s = '(df) / (np.float64(3.0))'
sanitize = True
skip_quotes = '(df)/(np.float64(3.0))'
types = {}
pandas_numexpr_numpy2/lib64/python3.11/site-packages/numexpr/necompiler.py:283: ValueError
from pandas.
numexpr
does not seem to be installed in the Numpy dev CI runs, so these tests are skipped there.
from pandas.
No longer an issue with pandas-3.0.0.dev0+1178.g236d89b856
Now how to find the correct change to backport to 2.x ...
from pandas.
I don't know this code at all, but I think #59144 was the relevant change? Can we try backporting that?
from pandas.
Related Issues (20)
- DOC: Intro to data structures HOT 3
- ENH: to_excel: warning/conversion of text values starting with '=' HOT 6
- DOC: Separate Examples for String Methods (str.isalnum(), str.isalpha(), etc.) in docs HOT 2
- ENH: Add kwargs to `Series.map` HOT 3
- No Python 3.13 wheels available in scientific-python-nightly-wheels HOT 3
- BUG: Empty column name in group dataframe when using pyarrow types HOT 11
- QST: how to pandas fast nested for loop for "non numeric" columns? HOT 2
- BUG: Index union with datetime64[us] dtype and frequency HOT 2
- BUG: is_unique fails when using float128 indexes, since pandas 2.0 HOT 5
- ENH: Add parameter to read_html() that disables the _remove_whitespace() function HOT 6
- BUG: df.MultiIndex.levels continues to return the index of the original df after mutation HOT 1
- ENH: Restore the functionality of `.fillna` HOT 2
- BUG: `None` values are not processed when applying `pd.isnull` to a Series with dtype `category` HOT 3
- BUG: Timestamp.tz and DatetimeIndex.tz are inconsistent when pytz 2024.2 is installed HOT 2
- Inconsistent Return Types Between numpy 1.26.4 and numpy 2.1.0 in pandas 2.2.2 HOT 2
- ENH: add a comments variable to pandas.DataFrame.to_csv HOT 11
- BUG: to_sql with ADBC driver fails when schema exists without tables HOT 1
- BUG: pd.options.future.no_silent_downcasting is not backward compatible HOT 1
- BUG: Automatic change of color when the plot type is "line" but not when it is "scatter" HOT 3
- BUILD: no linux-aarch64 wheels for v2.2.3? HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.