Comments (5)
I think .with_context
existed as a workaround for the lack of a horizontal concat on LazyFrames.
That was recently added:
I recall reading in one of the previous issues that .with_context
may end up being deprecated now?
from polars.
I recall reading in one of the previous issues that .with_context may end up being deprecated now?
I don't think it should, with_context
allows for aggregate operations on different sized frames, which is nice:
>>> import polars as pl
>>> ldf1 = pl.LazyFrame({"a": [1, 2, 3]})
>>> ldf2 = pl.LazyFrame({"b": [1, 2, 3, 4]})
>>> ldf1.with_context(ldf2).select(pl.all().sum()).collect()
shape: (1, 2)
┌─────┬─────┐
│ a ┆ b │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 6 ┆ 10 │
└─────┴─────┘
from polars.
It makes no difference to me though as its only a matter of calling .lazy() when using .with_context. I am just curious.
from polars.
oh....I didn't know that. So it was a workaround.
bro doesn't .with_context have its own unique advantages?
let me give you an example where I am kinda struggling without .with_context on dataframes/lazyframes
If I want to get a mask of different dataframe and directly plot.
A = (
pl.DataFrame({
"id":np.arange(1000),
"some_ranodm_strcol":np.random.choice(a=["a","b","c"],size=1000)
})
)
B = (
pl.DataFrame({
"id":np.random.choice(1000,size=500,replace=False),
"some_ranodm_strcol":np.random.choice(a=["a","b","c"],size=500)
})
.sort(by="id")
)
(
A
.lazy()
.with_context(
B.lazy().select(pl.all().suffix("_b"))
)
.select(
pl.col("id"),
pl.col("id").is_in(pl.col("id_b")).cast(pl.Int8).alias("new_col")
)
.collect()
.plot.line(
x="id",
y="new_col"
)
)
Is there a different alternative?. I learnt .with_context in udemy's course and its quite helpful when I want to use other dataframes columns to fast visualize(though they didn't teach that)
Both frames have different sizes. so with .with_context omits the necessity to select/extract series from both the frames and do operations on them.
This is why I was curious as to why eager frames don't have .with_context. Its such a unique and novel feature compared to pandas
from polars.
I recall reading in one of the previous issues that .with_context may end up being deprecated now?
I don't think it should,
with_context
allows for aggregate operations on different sized frames, which is nice:>>> import polars as pl >>> ldf1 = pl.LazyFrame({"a": [1, 2, 3]}) >>> ldf2 = pl.LazyFrame({"b": [1, 2, 3, 4]}) >>> ldf1.with_context(ldf2).select(pl.all().sum()).collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 6 ┆ 10 │ └─────┴─────┘
Yes exactly. and .with_context should also be extended to pl.DataFrames if theres no downsides to it
from polars.
Related Issues (20)
- `.group_by(series).agg(...)` fails when using `is_not_null` in aggregation HOT 1
- Parquet row group filter pushdown not working as expected HOT 2
- Minimal tests for memory usage contraints in partial-column read APIs HOT 1
- to_dummies() to accept "return_empty" argument for pl.Enum type.
- Minimal tests for memory usage contraints in partial-column read_csv_* APIs
- Regression: Panic when running complex query on 0.20.17 backtrace included HOT 9
- Adding a section for R bindings in the list of APIs reference?
- Allow pythonic truthiness to Series & DataFrame HOT 2
- scan_parquet is unhappy with FizedSizeBinary type
- `pl.repeat()` panicks if input doesn't match data type HOT 2
- BUG: group_by with kwargs column not found when row >=1000 HOT 2
- Rename in `LazyFrame.group_by` when done after a explode throws `ColumnNotFoundError` after 0.20.17 HOT 1
- Large regression in performance of window function (over two columns) starting from `0.20.11` HOT 4
- Expression with multiple aggregations raises panic exception in rolling window HOT 1
- Run end encoding support? HOT 1
- Expression alias() before struct.field() overrides entire expression
- Sink to file with streaming HOT 3
- permit run polars on old CPU HOT 10
- Pre-compiled version for alpine linux / musl HOT 4
- Writing Parquet to JSON when row_oriented=True HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polars.