Comments (2)
@rahulnair23 Thanks for raising this issue and clearly pinpointing it. I think a one-line fix would be to use data[c].dropna().unique()
instead of data[c].unique()
in FeatureBinarizer L70.
However, your issue has made me think that the "NaN indicator" that can be created for ordinal columns should be extended to binary (and unary) columns.
- In the
fit()
method, lines 90-92 could be copied after line 70 to identify binary columns with NaNs. - Then in
transform()
, something like lines 168-175 could be copied after line 136 to create the NaN indicator, but it would have to be modified. Perhaps like:
if c in NaN:
# Ensure that rows corresponding to NaN values are zeroed out
indNull = data[c].isnull()
A.loc[indNull, (str(c), '', '')] = 0
# or maybe A[(str(c), '', '')].fillna(0, inplace=True) works and is easier
if self.negations:
A.loc[indNull, (str(c), 'not', '')] = 0
# Add NaN indicator column
A[(str(c), '==', 'NaN')] = indNull.astype(int)
if self.negations:
A[(str(c), '!=', 'NaN')] = (~indNull).astype(int)
Since you're working with data for which this problem arises, could I ask you to test at least the one-line fix and if it works, submit a pull request? If you're up for it (and if it might help you to encode NaNs), it would be great if you could also try the NaN indicator extension above. Thanks in advance.
from aix360.
Sure, turned in a pull request #80 with the one-liner fix (verified that it works on my data). Thanks!
from aix360.
Related Issues (20)
- Defect FeatureBinarizer HOT 1
- Version reconciliation with AIF360
- AIX360 conda version HOT 9
- CEMExplainer scikit-learn compatibility & "predict_long" method error HOT 3
- UnboundLocalError: local variable 'newinnerProduct' referenced before assignment HOT 1
- Feature request: Update Tensorflow version to be compatible with Python 3.8+ HOT 2
- How to execute monotonicity and faithfulness on keras dense layer models?
- How to execute the metrics on regression problems instead of classification? HOT 2
- How to evaluate shap local explanations with Faithfulness and monotoncity metrics?
- ModuleNotFoundError: No module named 'aix360.algorithms.rule_induction' HOT 3
- Ripper rule induction algorithm treats timestamp type features as categorical HOT 1
- Questions about the results obtained by XAI method HOT 1
- ContextualVersionConflict
- Error: "elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison"
- Upgrade the tensorflow version in setup.py to support higher versions of Python HOT 4
- ModuleNotFoundError: No module named 'tensorflow.contrib' HOT 26
- aif360.mybluemix.net does not exist HOT 1
- aix360.mybluemix.net HOT 2
- Update the outdated project website URL HOT 1
- MEPSDataset preprocessing return empty df. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aix360.