Comments (5)
Hm, this doesn't look too hard to fix, looking at the error (the nb_compat_func
line).
It's been a while since I've looked at any numba stuff, though.
Are you interested in potentially contributing a PR?
from pandas.
Are you interested in potentially contributing a PR?
Sure, but I find the solution may not be that trivial:
Since numba generally requires the argument is known when function is defined, so any kind of varargs is not well supported.
It dosen't support **kwargs
, and IIRC it also had some bugs with *args
.
This means we can't simply add **kwargs
to nb_looper
:
pandas/pandas/core/_numba/executor.py
Lines 26 to 52 in 58461fe
A possible solution is to dynamically create an intermediate function for each case. Use the example above:
df.apply(np.nanquantile, q=0.5, raw=True, engine="numba")
In this case, we dynamically defines:
@register_jitable
def nb_compat_func(values, q=0.5):
return user_func(values, q=q)
which holds the **kwargs
and the only required argument is values
and this nb_compat_func
will be used by nb_looper
just as before.
However, I felt this solution is somewhat too hacky and not clean. Do you have any other ideas?
from pandas.
I think *args might be the best way to deal with this, if it's possible to fix this.
It's how we pass the kwargs to the other numba stuff like the numba groupby stuff.
Do you remember the issue you had with *args?
from pandas.
numba/numba#8108
Basically if *args
exists then other args that follows becomes kwd-only which is not yet supported. (Maybe there are also some cache related issues from my recollection but those are irrelevant)
I think *args might be the best way to deal with this, if it's possible to fix this.
It's how we pass the kwargs to the other numba stuff like the numba groupby stuff.
I just checked out the code of numba groupby, it should be viable if we disallow **kwargs
and only use *args
in numba apply as well.
from pandas.
take
from pandas.
Related Issues (20)
- BUG: `<Framelike>.__contains__(<unhashable>)` errors HOT 6
- BUG: Error when repr-ing nested DataFrames HOT 3
- BUG: DataFrame(data, ...) creates a copy when 'data' is a NumPy array (pandas 3.0+) HOT 9
- DOC: Document Flashes White in Dark Mode HOT 2
- BUG: inconsistency in the behaviour of `DatetimeIndex.is_year_start` on the frequency "BYS"
- BUG: Index containing NA behaves absolutely unpredictably when length exceeds 128
- BUG: HOT 3
- BUG: .mode(dropna=False) doesn't work with nullable integers HOT 1
- BUG: iterrows() on an awkward array with equal-length rows results in a ValueError HOT 3
- BUG: float64 convert to int64 get confuse result HOT 1
- QST: HOT 2
- BUG: `diff()` returns confusing output when dealing with negative timestamp deltas HOT 1
- ENH: add "right" argument to "qcut", like "cut" has HOT 2
- BUG: default value for dtype_backend is not applied by default HOT 5
- DOC: add detailed comments pandas/tests/arithmatic/datetime/ line no: 750, test_dti_cmp_tdi_tzawareness HOT 1
- BUG: Refactor test_dti_cmp_tdi_tzawareness function to reduce redundancy pandas/tests/arithmetic/test_datetime64.py HOT 2
- BUILD: HOT 1
- BUG: AttributeError: 'Engine' object has no attribute 'cursor' HOT 5
- BUG: FutureWarning when following documentation HOT 4
- BUG: Column of dtype Categorical in DataFrame encounters error when taking a row that includes nan in the column HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.