Feature Tracker

[adamerose]

Feature Tracker

[jonhill7]

Feature Request 1: Ability to export plotly code (maybe copy to clipboard) so that plots can be recreated outside of PandasGUI.
Feature Request 2: Ability to save plot figures (maybe using plotly's save to html).

[fdion]

Another thing that would be extremely helpful: adding a nunique column in the drag and drop interface, next to variable names, that way it is easier to group low cardinality / categorical at the top to use for color, symbol, line_grouping etc.

[coljac]

I didn't see a feature (or existing functionality) to hide columns from view. That might be helpful in lots of cases. (The roadmap looks great by the way.)

[jonnojohnson]

This is a great project! Is there a way to sum values with the same key before plotting? For example if my data points are hourly but I want to plot the total sum per day. Or would that need an enhancement? By default it seems to calculate the mean.

After playing a bit more I'm guessing this would be the intent of the aggfunc capability in the Reshaper but sounds like that's not really implemented yet.

[jonnojohnson]

Now I've looked at the code I think what I was asking about would be best implemented in the grapher widget by allowing to change the groupby aggfunc. I'm sure the reshaper aggfunc would have similar benefits.
Looking forward to following this project.

[supernooba]

Firstly, thanks a lot for sharing your work @adamerose, as well as the road map, definitely a very useful tool in data preparation. Would you consider adding a "Buy me a coffee" button? :)

[adamerose]

Is it possible to save the dataframe once you are done editing and interacting with it through the GUI? Specifically, is it possible to do this programmatically (as opposed to using the Export to CSV).

Yeah use the .get_dataframes() method on the instance returned by show(), I should probably add that to the docs, it's in the demo video though

[robdmc]

Would you consider an option for filtering that goes beyond what is possible with the .query() method? For example, I don't know of a way to do this with the query function (although, maybe it's possible).

df = df[df.col_a.isnull() & df.col_b.notnull()]

Maybe allowing expressions like the above to be an additional way of specifying filters? Might be out of your desired scope though.

Or perhaps go the Salesforce route and allow grouping/combining filters with and/or logic, as long as isnull() and notnull() can be addressed.

[adamerose]

@robdmc I think everything is possible with the query() method, yours would be like this:

from pandasgui.datasets import pokemon
pokemon.query("`Type 2`.isnull() & Name.notnull()")

This thread is a good read https://stackoverflow.com/a/53779987/3620725

[robdmc]

TIL. Thank you.

[adamerose]

Feature Request 1: Ability to export plotly code (maybe copy to clipboard) so that plots can be recreated outside of PandasGUI.
Feature Request 2: Ability to save plot figures (maybe using plotly's save to html).

Saving figures as images is already possible with this button

But I'll add HTML export and your other feature to the list

[cjagoe110]

@robdmc I think everything is possible with the query() method, yours would be like this:

from pandasgui.datasets import pokemon
pokemon.query("`Type 2`.isnull() & Name.notnull()")

This thread is a good read https://stackoverflow.com/a/53779987/3620725

Hi,

I've been trying to make use of string contains conditions in the filter tab. When coding it out, it works, but within the pandasgui it returns either a str992 error, or a series can't be hashed error.

Any ideas on this?

[adamerose]

@coljac That syntax works on my sample dataset

Hard to tell without seeing the traceback or dataset, but the error sounds like your Description column contains Series objects mixed with strings so the str.contains method is failing.

I'm surprised to hear that it works when coding but not in the GUI, can you share the dataset?

[cjagoe110]

@coljac That syntax works on my sample dataset

Hard to tell without seeing the traceback or dataset, but the error sounds like your Description column contains Series objects mixed with strings so the str.contains method is failing.

I'm surprised to hear that it works when coding but not in the GUI, can you share the dataset?

Thanks for the quick reply! I got it to work by opening an empty GUI and then importing the data through there instead of importing it via pandas code. Not sure why that would make a change, but it worked!

[cjagoe110]

I might be just missing how to do this. But with the Reshaper:Pivot, how do we select or change the aggfunc used?

Sort of related, when exporting out dataframes, is there an option built-in to include the indices?

[adamerose]

But with the Reshaper:Pivot, how do we select or change the aggfunc used?

Yeah my bad that's just not implemented, probably should have kept that option hidden until finished.

Sort of related, when exporting out dataframes, is there an option built-in to include the indices?

Nope, added that to the feature tracker

[jonnojohnson]

This is a great project! Is there a way to sum values with the same key before plotting? For example if my data points are hourly but I want to plot the total sum per day. Or would that need an enhancement? By default it seems to calculate the mean.

[GentilsTo]

Hi, really cool looking project! I just saw you were thinking of managing multiple graph with new tab , which is indeed great. Maybe a way to store a given graph configuration would be great too? (that might actually be close to the request of generating the graph code, but the purpose here is to benefit from a GUI approach. You would hence be able to quickly switch between the different graph you take time to define while not having to keep all of them open at the same time in multiple tab. The save of a configuration might be based on user demand, with a way to provide a meaningful label, and access to the plot with a live buffer list area (for example splitting you current "Name" section to add a "Saved Graph"). (could even push this a bit further with a way to store all these saved graph configurations (and dataset ?) in a "workspace-like" file and loaded it up afterward in a new pandasgui show session, with the same or even updated dataset (sharing the same structure) ! )

[adamerose]

@GentilsTo

store all these saved graph configurations (and dataset ?) in a "workspace-like" file and loaded it up afterward in a new pandasgui show session, with the same or even updated dataset (sharing the same structure) !

I like this idea, added to list. Saving the current datasets and filters would be easy - I can just pickle the PandasGui store state, but I don't actually store Plotly graphs anywhere inside the store... I have to save them as .html files in the temp directory because directly passing HTML to a QWebEngineView has a 2MB limit (too small for Plotly graphs on big data). Need to think about how to solve that

[aneeshsoni]

Is it possible to save the dataframe once you are done editing and interacting with it through the GUI? Specifically, is it possible to do this programmatically (as opposed to using the Export to CSV).

[rafaellehmkuhl]

Is there anyone working with implementing rolling windows (for moving averages)?

I would be really interested in contributing to that.

[adamerose]

Is there anyone working with implementing rolling windows (for moving averages)?

I would be really interested in contributing to that.

I haven't worked on that. Go ahead and make a new issue for it with whatever ideas you have and we can discuss it more. Glad you're interested in contributing :)

I'm not sure where this should appear in the GUI and how the UI should look, I'd like to do it in a way that will work with future similar features that add calculated columns.

[rafaellehmkuhl]

Is there anyone working with implementing rolling windows (for moving averages)?
I would be really interested in contributing to that.

I haven't worked on that. Go ahead and make a new issue for it with whatever ideas you have and we can discuss it more. Glad you're interested in contributing :)

I'm not sure where this should appear in the GUI and how the UI should look, I'd like to do it in a way that will work with future similar features that add calculated columns.

Nice to hear that! I also think the UI part should be something that will work universally. I will open the issue them!

[am-technix]

Export python code will be amazing for people who are starting with pandas !

[alexandrorp]

Hi, I just learnt about your module and I have to congratulate you because I think it's the tool many of us were waiting for and didn't have the talent to create. I just had a simple comment that I think I haven't seen elsewhere: is there going to be boolean filtering with other operations like OR and grouping conditions with parenthesis?

[adamerose]

Hi, I just learnt about your module and I have to congratulate you because I think it's the tool many of us were waiting for and didn't have the talent to create. I just had a simple comment that I think I haven't seen elsewhere: is there going to be boolean filtering with other operations like OR and grouping conditions with parenthesis?

@alexandrorp
You can do that with query expressions which the PandasGUI filter uses

[fdion]

Polar plots would be quite easy to add:

https://plotly.com/python/polar-chart/

Plotly Express has px.scatter_polar and px.line_polar, so that's pretty straightforward and would cover many use cases in engineering and data science. There is px.bar_polar too, but Radar plots (based on line_polar) and polar bar plots require a little bit more, but still manageable.

[fdion]

@adamerose I think several planned enhancements in the original list have now been released. Probably should move them so nobody picks one up and work on it!

[fdion]

@adamerose, is this pretty close or pretty far on your radar?

JMP-style 'calculated' columns which are columns with values defined by a function, that get automatically recalculated after any change to the DataFrame

(Issue #72 is related to this)

[adamerose]

@adamerose, is this pretty close or pretty far on your radar?

JMP-style 'calculated' columns which are columns with values defined by a function, that get automatically recalculated after any change to the DataFrame

(Issue #72 is related to this)

Haven't started it, probably far. If you start to work on it, you might want to look at how I implemented filters because it could be similar behavior... they get recalculated and reapplied every time the underlying data changes, and are stored as raw expr strings. And in code export they get shown at the very end.

Make an issue for this if you start on it so we don't clutter this thread with discussion.

[tejv]

Is it possible to style(color) the cells based on some criteria. Or take styled df as input

[silgonel]

Is it possible to visualize the number of residual rows after applying filters to the dataset?
At the moment I can only find this info if I try to make a graph. Thank you for the nice tool!

[adamerose]

Is it possible to visualize the number of residual rows after applying filters to the dataset?
At the moment I can only find this info if I try to make a graph. Thank you for the nice tool!

When you apply filters, the Grapher will make graphs with your filtered data. Not sure what you mean by "can only find this info if I try to make a graph", where else would you visualize the data other than when making a graph?

Is it possible to style(color) the cells based on some criteria. Or take styled df as input

I've implemented coloring by values based proportion of total (can be total of column, row, or both). They're options when you right click the header

But I like your suggestion of directly applying dataframe styling, can you create an issue for that?

[adamerose]

Closing this thread since the number of feature requests from users other than myself has grown pretty large so going forward let's track each one in a new issue. If you have any feature request that doesn't have an open issue already please create one :)