Comments (8)
Good suggestion - we can do the same 'trick' as we do in replace
.
from polars.
just use
map = {
"a": "x",
"b": "y",
"c": "z",
}
df.with_columns(
polars.col("A").replace(map)
)
from polars.
Is there a way that we can make this work with regex ?
I have tried something like:
import regex
import polars
df = polars.DataFrame({
"x": ["a", "b", "c", "1", "2", "3"],
})
map = {
regex.compile(r"[a-z]"): "alpha",
regex.compile(r"[0-9]"): "digit",
}
df.with_columns(
polars.col("x").replace(map)
)
For my personal use case, I need to replace a large number of regex patterns, and it's not very ergonomic to use two lists because it can be hard to keep track of what is replacing what.
Another possibility is a list of tuples:
map = [
(r"[a-z]", "alpha"),
(r"[0-9]", "digit"),
]
This (in my opinion) is nicer to follow than something like:
old = [r"[a-z]", r"[0-9]"]
new = ["alpha", "digit"]
from polars.
A dict is just going to be syntactic sugar. You can just define your map and then call str.replace_many(map.keys(), map.values())
.
from polars.
That gives me:
TypeError: cannot create expression literal for value of type dict_keys:
...
Hint: Pass `allow_object=True` to accept any value and create a literal of type Object.
from polars.
Call list
on each input then. I'm just saying: this is just some syntactic sugar that you can do yourself. You don't need us to take care of it. Though it would be nice if we did.
from polars.
I must be missing something, as none of the following work for me:
## ============================================================================
import regex
import polars
## ============================================================================
df = polars.DataFrame(
{
"x": ["a", "b", "c", "1", "2", "3"],
}
)
## ============================================================================
map = {
regex.compile(r"[a-z]"): "alpha",
regex.compile(r"[0-9]"): "digit",
}
df.with_columns(polars.col("x").replace(map))
df.with_columns(polars.col("x").replace(map.keys(), map.values()))
df.with_columns(polars.col("x").replace(list(map.keys()), list(map.values())))
## ============================================================================
map = {
r"[a-z]": "alpha",
r"[0-9]": "digit",
}
df.with_columns(polars.col("x").str.replace_many(map))
df.with_columns(polars.col("x").str.replace_many(map.keys(), map.values()))
df.with_columns(polars.col("x").str.replace_many(list(map.keys()), list(map.values())))
## ============================================================================
All throw an exception except the third (when calling list()
), which does not throw an exception, but also does not match the regex pattern, so I am left with the original dataframe.
from polars.
There are a few different issues:
- You're passing
regex.compile()
objects - which Polars does not understand.
Polars uses the Rust crate https://github.com/rust-lang/regex - so you must pass "strings" when using the regex functions.
.str.replace_many
does not work with regular expressions. (perhaps a note could be added to the docs?)
It uses https://github.com/BurntSushi/aho-corasick which works with "literal strings" only.
It sounds like you may really be asking for:
from polars.
Related Issues (20)
- Request to return LazyFrame for pl.from_arrow and more HOT 2
- rolling_corr giving inconsistent results HOT 1
- `test_read_database_cx_credentials` expected exception does not bifurcate over correct Python version HOT 1
- In `scan_parquet()`, `include_file_paths` returns twice the same column HOT 1
- In `bin.size()`, the `unit` parameter is not described HOT 1
- `.implode().list` method is not valid before `.over`. HOT 4
- Upgrading to 0.42 breaks compilation HOT 4
- Expr `.implode().get().over()` has strange results. HOT 3
- upsample not working if arguments are only sorted within group HOT 1
- Behaviour change of Expr.list.drop_nulls for structs with all None fields HOT 1
- `.first` behind `.sort_by.slice` gets wrong result.
- `Expr.shuffle` uses different order per column HOT 2
- `.struct.field()` after `shuffle()` seems to produce incorrect results
- Multi-output, multi-sink lazy polars HOT 4
- Inconsistent Results Between Pandas and Polars using cut (and qcut)? HOT 2
- ExprStringNameSpace replace / replace_all literal flag ignored for dataframes with multiple rows
- `dt.round()` slow/fast path use different rounding HOT 2
- write_parquet with partition_by silently overwrites existing files HOT 7
- rank() on a Series of just 1 null assigns rank=1 to the null value. HOT 1
- read_ndjson ignores provided schema list inner types if values are inferred null
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polars.