Comments (2)
Hi Wei,
I am glad to hear that you are finding this work useful. I've tried to provide some brief answers to your questions below.
- I've now added the x-axis information to the datasets hosted on Google Drive.
- I unfortunately do not have access to the dataset with an extended Raman shift.
- This could work, though potentially there is some deuterium signal in the 500-1800 Raman shift region that is not easily observable which might introduce unwanted bias. To check this you could perform a PCA on your dataset (cropped to 500-1800 Raman shift) and see if there is separation between spectra with/without deuterium.
- To prevent overfitting, make sure to separate your dataset into train, evaluation, and test splits and stop training when the performance on the evaluation decreases. In addition, increasing data augmentations can help with overfitting.
Cheers,
Conor
from deeper.
3. nd see if there is separation between spectra with/without deuterium.
Thank you very much!
I am very grateful for your help.
So far, I haven't find a good solution for the overfitting problem albeit I have done all the things you have suggested. The nature of my task requires my model to be able to reconstruct/denoising cell signiture that is out of all training/testing/validation set (i.e. a completely new cell-type). My current solution is to give up full-length transformer model, limit the scope(context window size) of the CNN filter or use local self-attention, so that the model can only learn short distance feature, proventing it from memorzing the entire cell spectrum. Full length transformer has the best result so far but tend to learn long distance feature, which I think is quite corelated with known cell-types.
I will share with you our research progress in the future once we submit our draft.
from deeper.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeper.