Comments (10)
Hi, I presume your modelstudio is working just fine and you are only worried about the warning?
As the message suggests, something went wrong in the ingredients::describe(<ceteris paribus object>)
function call, which produces textual description of the ceteris paribus explanation. This is an experimental feature to display text when you hover on the D button next to your plot.
For me to debug it further, it would be best to obtain the updated model/data.
from modelstudio.
Hi @MJimitater, without code and/or any reproducible example, e.g. on a different data/model, I might not be able to help you.
from modelstudio.
Hi @hbaniecki, this is the code that I used
explain <- DALEX::explain(trained_model, data = train[1:train_size,], y = as.numeric(train[1:train_size,]$PRUEFERGEBNIS)-1, type = "classification", label = "xgboost", predict_function =
function(trained_model, obs){
previous_na_action <- options('na.action')
options(na.action='na.pass')
sparse_matrix_test <- sparse.model.matrix(PRUEFERGEBNIS ~., data = obs)
options(na.action=previous_na_action$na.action)
results_test <- predict(trained_model, sparse_matrix_test, type = "response")
round(results_test,3)
}, predict_function_target_column = 0)
new_observation <- test[ind_obs, , drop = FALSE]
xai_dashboard <- modelStudio(explain, new_observation = new_observation)
I hope this helps somewhat.
If this doesn't help much, then we can see how to provide the model and data. Thanks
from modelstudio.
Hi @MJimitater, of what class is the object passed to data
? Does the following example help you by any chance https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html#xgboost-dashboard?
Googling the error message, I'm unsure if this issue is related to DALEX and modelStudio.
Might try to debug it if you provide data/code.
from modelstudio.
Hi @hbaniecki , the data
object passed to explain()
is a "data.table" "data.frame"
and train[1:train_size,]
the first train_size -rows.
Yes, Im also unsure if this issue is related to modelStudio.. is there way to provide data/code confidentially?
from modelstudio.
Can you serialize both in R, so I can load them into my environment and share with me at [email protected]? Data can be small (a few rows). You can also create a simpler xgboost model where the error still occurs--perhaps a small subset of features suffices.
Anyhow, I don't have your email to reach out.
from modelstudio.
@hbaniecki Thanks so much for your help! Just now, I sent you an email with the model and data in .rds format. I hope you can reproduce, keep me posted ;) Cheers
from modelstudio.
Hi @hbaniecki sorry for my late reply, Thanks again for your excellent help in debugging!
I have thoroughly examined the data for NA-values and have removed or imputed them in the numeric features, and changed them to "(Missing)" in the factorial features (by fct_explicit_na()
) hoping that the problem with the variable_splits is solved. I do hope that this is a valid approach to generating error-free model- and predict profile explanations.
However , running modelStudio(explain, new_observation, B=3)
with the updates runs in the following error:
Calculating ingredients::ceteris_paribus (1)
Calculating ...
Elapsed time: 00:10:37 ETA:10s Warning messages:
1: In value[[3L]](cond) :
Error occurred in ingredients::describe.ceteris_paribus function: missing value where TRUE/FALSE needed
2: In value[[3L]](cond) :
Error occurred in ingredients::describe.ceteris_paribus function: missing value where TRUE/FALSE needed
Now Im trying to find where the bug occurs, but seemingly everything works so far:
#doesn't work:
ms <- modelStudio(explain, new_observation, B=3)
#works:
mp <- model_performance(explain)
#works:
mr <- model_diagnostics(explain)
#works:
fi <- model_parts(explain, B=2)
plot(fi)
#works:
pdp <- model_profile(explain)
#works:
pd <- ingredients::partial_dependence(explain)
#works:
pp <- predict_profile(explain, new_observation)
#both work:
cp <- ingredients::ceteris_paribus(explain, new_observation)
pdp <- ingredients::aggregate_profiles(cp, type = "partial")
Now Im wondering that cp <- ingredients::ceteris_paribus(explain, new_observation)
worked, but somehow still throws this error in modelStudio
. What else can I try? What other functions can I try to unit test in order to debug? Do you think this is still a problem with the "(Missing)"-values in the categories? Thank you so much
from modelstudio.
Thanks for your fast reply, I'll get back to you next week! ;)
from modelstudio.
Actually you are right, modelStudio works just fine! Finally the explanation of single test cases works without errors!
BTW, would be nice to have a faster way of explaining and visualizing individual test cases, than waiting a lengthy time for modelStudio to finish calculating, but I guess this the way it is ;) Great software!
Thanks again for the excellent support, closing for now
from modelstudio.
Related Issues (20)
- There is a plan to export modelStudio to python? HOT 2
- add `verbose` as an alias to `show_info`
- add `ms_merge_observations()` function to merge multiple modelStudio into one
- add `happiness` dataset
- update DALEX, ingredients, and iBreakDown versions? HOT 1
- add information about css needed for a proper modelStudio rendering in rmd -> html (vignette)
- update GIFs with the happiness demo example
- modelStudio fails on R v4.2 (devel) due to changes in logical operators HOT 1
- multiple modelstudio objects overlap HOT 3
- add `variable_splits` to modelStudio HOT 1
- Add a parameter to set, which plots to open and on which positions
- Error in check_explainer(explainer): For Python support, use precalculate=True in Explainer init HOT 5
- Why are Break Down plot intercepts the same for different regression models? HOT 8
- Is 4x4 panels the only way to present ModelStudio? HOT 2
- smart choosing of observations
- v2.1.0 release checklist
- Support for explainer objects from the `survex` package HOT 2
- Error in R - Error in eval(predvars, data, env) : object 'Breach' not found HOT 1
- Parsing error for only one variable model HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from modelstudio.