Comments (4)
Depends on the strategy! I've read papers suggesting complex numbers as teacher signals or just applying a function to transform from the complex domain to the real one.
I do agree with you, I'll try using different strategies before openning a PR then! Finally managed to have a clean installation!
Edit=Removed unnecessary add-on
from tabnet.
This does not feel like a very common issue. I guess you can already use the attention grouping strategy to represent complex numbers by grouping each rho and theta together. Would it to the trick?
from tabnet.
Thanks for the fast reply!
Yes, I do agree that it is quite niche but I have found some papers showing its use for particular applications w/ physic related topics.
I'm still solving some environment dependencies to install TabNet but I would imagine that, theoretically, you can. However, I'm pretty sure that torch does not recognize cfloat as dtype and, thus, we would need to make a 2D convolution for every layer. Since you coded the TabNet to such a high-level I'm not sure whether it could be more simple (or possible) to just use a wrapper (like the one below)? Also, to train the NN with traditional classification with complex numbers you would typically need to apply a mag() at the output or just use a regression with MSE such as:
import torch.nn.functional as F
def complex_relu(input):
return F.relu(input.real).type(torch.complex128)+1j*F.relu(input.imag).type(torch.complex128)
class ComplexReLU(nn.Module):
def forward(self,input):
return complex_relu(input)
class ComplexMSELoss(nn.Module):
def __init__(self):
super(ComplexMSELoss, self).__init__()
def forward(self, inputs, targets):
if inputs.is_complex():
diff_real = (inputs.real - targets.real)
diff_imag = (inputs.imag - targets.imag)
sss = torch.mean(torch.square(torch.abs(diff_real + diff_imag)))
return sss# Compute mean squared error between real and imaginary components
else:
return F.mse_loss(inputs , targets)
Edit: Forgot the wrapper*
from tabnet.
Are you predicting complex numbers as well?
You can still preprocess your dataset in such a way that complex numbers are already transformed in imaginary and real parts. Make sure that you group the inputs so that attention is applied by complex inputs not separately on real and imaginary parts, for the outputs you can use TabNetRegressor to predict both the imaginary and real parts together.
That would be simplest way in my opinion, and that is what I would try before trying to rewrite the library. However, if you want to try it feel free to open a PR but it seems like a lot of work and big changes ahead to make this easily usable.
from tabnet.
Related Issues (20)
- TabNet overfits (help wanted, not a bug) HOT 9
- TabNetRegressor vs other networks HOT 1
- spike in memory when training ends HOT 8
- Severe overfitting HOT 18
- OOM problem when I search hyperparameters with Tabnet HOT 3
- Different classification variables in the test set and train set HOT 1
- Struggling to get model to fit - Help Wanted HOT 7
- Optimizing TabNet for Disease Classification with Continuous Audio Features HOT 1
- Interpreting Sparsity on Global Importance HOT 5
- ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() HOT 1
- Validation loss HOT 1
- Lightweight Fine-tunning or few-shot learning for limited labeled data HOT 1
- Maybe `drop_last` should be set as False in default? HOT 1
- Incompatiblity of current round() method with pytorch tensors when performing early stopping HOT 1
- Retraining a saved model on different dataset HOT 3
- change device seems not work HOT 8
- when I try to train with pretrained model loaded, kernel dies HOT 2
- Feature
- Multi-label classification - how to start HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tabnet.