Comments (1)
Hello Ashirwad,
sounds like a really interesting dataset, that you have.
Might be a good suggestion! How do you do the 'replacing such values with reasonable numbers'?
In general there are multiple approaches for setting the limits / constrain the imputations to an interval. Here are the two most common ones:
METHOD 1:
One approach is transforming the data using a scaled logit transform.
(see also https://robjhyndman.com/hyndsight/forecasting-within-limits/)
This works also for imputations.
Would be implemented like this:
(your dataset is in your_data)
# Bounds
# min and max have to be lower/higher than the existing min/max in the series
min <- 200
max <- 700
# Transform data
y <- log((your_data-min)/(max-your_data))
# Perform imputation on transformed data
imp <- na_kalman(y)
# Back-transform the now imputed dataset
back <- (max-min)*exp(imp)/(1+exp(imp)) + min
# Result is in back
back
Downside is, it changes the whole distribution of imputed values.
Meaning also the imputations that were within the limits before will now be slightly different.
(but as you see in the next method this is also the upside of this method)
METHOD 2:
You could just leave the imputations that are within the limit untouched and set the imputations violating the limits to the limit.
# Bounds
min <- 200
max <- 700
# Perform imputation
imp <- na_kalman(your_data)
# Replace out of bound values with the min/max
imp[ imp > max] <- max
imp[ imp < min] <- min
# Result is in imp
imp
Downside here is you will possibly have a quite a bunch of values being exactly the limits. Which might not be the distribution you want to have. Upside is, the other imputations are not affected.
Completely depends on your data, what makes most sense. That is also why I am a little bit hesitant with adding the limit, because there is no one fits it all solution.
But what I definitely want to add (once if find the time) is more documentation, that you can maybe read about these things in the documentation.
Think it would be really beneficial to have some advices about choosing the right algorithm and special cases like this one.
from imputets.
Related Issues (20)
- na_replace doesn't allow replacement full NA vector HOT 3
- na_kalman is slow for long time series HOT 8
- plotNA.imputation etc. not working with par()/layout() HOT 1
- Detailed Model Summary in na_kalman() HOT 2
- Faceting HOT 2
- Able to install but not load HOT 6
- Suggestion: Applying the na_mean function considering only values from the same periods. HOT 1
- Documentation needs updating HOT 5
- How to choose the best algorithm ? HOT 2
- Support imputing around a circle (e.g. wind direction) HOT 5
- Getting Error on part of my time series HOT 3
- Return fitting statistics and/or residuals HOT 2
- model0 or model In file na_kalman.R? HOT 1
- multiple imputations
- na_kalman: possible convergence problem: 'optim' gave code = 52 and message 'ERROR: ABNORMAL_TERMINATION_IN_LNSRCH'
- possible convergence problem: 'optim' gave code = 1 and message 'NEW_X'
- 'libRblas.so: No such file or directory' during package installation HOT 3
- Converting from ee.Image data to Numeric Vector (vector) or Time Series (ts) object HOT 2
- Error in `optim()`: ! L-BFGS-B needs finite values of 'fn' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from imputets.