michaelmarty/unidec

Universal Deconvolution of Mass and Ion Mobility Spectra

unidec's Issues

Import of mzML files for "Using MetaUniDec to Analyze Chromatograms" not working

Hi michaelmarty,
I was trying to import my MS data as you nicely described in your Wiki "Using MetaUniDec to Analyze Chromatograms". My data was acquired on a Synapt G1 with MassLynx V4.1 and i converted it by using MSConvert V3 to mzML with default parameters. Then i followed your description in MetaUniDec (Version 4.1.2): File/Automated Chromatogram Parsing/Auto Import Chromatogram by Scans and used Scan step of 1.0. At this point I receive the following issue:

UniDec Path: D:\UniDec_Windows_Dist_200121\UniDec.exe
Display Size (1920, 1080)
TESTING: ['D:\Daniel\20200207_P495\rawfiles\SYNAPT01502.mzML']
SYNAPT01502.mzML
Reading mzML: D:\Daniel\20200207_P495\rawfiles\SYNAPT01502.mzML
'NoneType' object is not subscriptable
Errors: ['D:\Daniel\20200207_P495_subunitxchange\rawfiles\SYNAPT01502.mzML']

Any ideas?

Thanks for your help,
Daniel

trying to create txt input

I am trying to create txt input for UniDec using thermo-raw file reader https://github.com/animesh/RawRead/tree/deConv but i am not sure how to represent MZ/intensity values as mentioned in http://michaelmarty.github.io/UniDecDocumentation/index.html , currently i am representing the base-peak and intensity(max) across the scan but getting a confusing mz/5 scatter

thus wondering what is the right way to represent the txt input?

Suggestion for MetaUniDec to skip bad runs

Hi Michael,
I have a suggestion for MetaUniDec to skip "bad" runs.
In batch runs some bad runs may have nearly no signal (just noise), when UniDec goes through the data in HDF5, a bad run will cause it to stop processing further. I also can't find good place to edit HDF5 directly. The only way I found is to recreate HDF5 file without the bad runs. Retyping the variables can be tedious.
Another small suggestion is to export file names/variables in the txt output in UniDec.

Thanks,
Mowei

Converting data recorded with bunching factor to txt

Hi @michaelmarty,

when converting IMS-data recorded on a Synapt instrument with bunching factor¹ set to >1 to txt with the Data Conversion Wizard GUI (and also when using the main GUI directly), UniDec returns a compressed spectrum (e.g. a spectrum from 200-16000 m/z is converted to 200-800 m/z) for the MS and the IMS function.
When using the "MS only" option in the Data Conversion Wizard, however, the spectrum is processed correctly.

MS only mode

IMS mode

As reference, the file unidec_bunching8.zip contains a rawfile and the converted txt files for data with bunching factor set to 8.

Thanks for developing and maintaining UniDec and best wishes

Julian

--
¹ Here, bunching refers to summing up a window of N m/z bins during acquisition to increase sensitivity and reduce file size.

Exporting Waterfall Plot from Meta UniDec 4.2

Hi,
is there a hidden option to export the fancy Waterfall plot from MetaUniDec 4.2.0 or is it not possible up to now?
Best, Daniel

error with opening .raw files

Hi Michael,
I have a problem with opening raw files with unidec. Other file formats (.txt) are ok, but it crushes even if I try to open the example data .raw file.
I have the same issue on my laptop (intel i5 cpu, 16gb RAM) and on the workstation (intel xeon cpu, 128 GB RAM) as well.
Thanks,
Bella

v3.1.0 won't open

Hi,

Just to let you know when you try to start the latest version there is an issue with the path of the preset manager. I am pretty sure you just need to change this file to handle cwd rather than a hard-coded path.

UniDec/unidec_modules/isolated_packages/preset_manager.py

Line 8 in 38e54ab

toppath = "C:\\Python\\UniDec3\\unidec_bin\\Presets\\"

mzML files and Linux

Greetings,

If I converted Synapt2 raw files to mzML using ProteoWizard's msConvert, would I be able to use UniDec on Linux exactly as I would on Windows ?

I looked at the code and it really seems that it is portable Python and C/++.

Could you confirm this, or list the features that would miss ? In particular, what features are available thanks to the use of the Waters library that would not if using mzML files (if any)?

Thank your very much for sharing this Free Software piece of work.

lopippo

m/z range

Hi michaelmarty,
I am studying this python code, but it is too long to start. Of course, I also meet some problems. Could you tell me how to determine the range of m/z in processing data? I cannot find the answer in the code.

Thanks for your help,
Vicki

Bug in Cube Plots

The legends in the cube plots are messed up.

Unable To Make TIC Selection

When importing a .raw file, I am unable to make any sort of selection of the TIC to evaluate. The yellow box appears and then disappears without any sort of charge envelope appearing and then resets to just the blank TIC.

pdf generation resulted in fatal error if file name contain .

First of all thank you so much for making this wonderful tool that easy to use!!!

I am having Errors when export as pdf report.

Filename: B0.4_7-9min_CombinedScan_csv

Error Code from latexpdf:
! LaTeX Error: Unknown graphics extension: .4_7-9min_CombinedScan_Figure1.pdf.

I suspect this might due to some string handling issue during the file name process?

Entire Debug info attached:

Required multiplierz data file unimod.sqlite not found!  Creating it.
WARNING: No unimod.sqlite found in C:\Users\Ruisi\Desktop\UniDec_Windows_Dist (C:\Users\Ruisi\Desktop\UniDec_Windows_Dist\GUI_UniDec.exe)
Failed to load dynlib/dll 'C:\\Users\\Ruisi\\Desktop\\UniDec_Windows_Dist\\unidec_modules\\waters_importer\\MassLynxRaw.dll'. Most probably this dynlib/dll was not found when the application was frozen.
C:\Users\Ruisi\Desktop\UniDec_Windows_Dist\unidec_modules\waters_importer\MassLynxRaw.dll
False

UniDec Engine v.4.0.0 Beta

UniDec Path: C:\Users\Ruisi\Desktop\UniDec_Windows_Dist\UniDec.exe
Launching UniDec

UniDec Engine v.4.0.0 Beta

UniDec Path: C:\Users\Ruisi\Desktop\UniDec_Windows_Dist\UniDec.exe
Display Size  (2880, 1620)
Loaded:  C:\Users\Ruisi\Desktop\UniDec_Windows_Dist\default_conf.dat
Opening:  B0.4_7-9min_CombinedScan.csv
Opening File:  B0.4_7-9min_CombinedScan.csv
Loading Time: 0.034s
Linear False
Data Prep Done. Time: 0.016s

Read from file: imflag input output numit endz startz zzsig psig beta mzsig psfun massub masslb msig molig massbins mtabsig intthresh aggressive rawflag adductmass nativezub nativezlb poolflag peakshapeinflate noiseflag linflag isotopemode baselineflag orbimode filterwidth zerolog

infile = B0.4_7-9min_CombinedScan_input.dat
outfile = B0.4_7-9min_CombinedScan

UniDec Run
Length of Data: 1471
Threshold: 2.165771
maxlength 4
mzdist set: 0.000000
0.606531
1.000000
0.606531
Number of Oligomers blurred: 3
Number of Charges blurred: 3
Isotope Mode: 0
Iterating

Iteration: 0 Convergence: 0.071958
Iteration: 1 Convergence: 0.022094
Iteration: 10 Convergence: 0.009222
Iteration: 11 Convergence: 0.000016
Iteration: 20 Convergence: 0.000292
Iteration: 21 Convergence: 0.000000
Iteration: 30 Convergence: 0.000006
Iteration: 31 Convergence: 0.000000

Converged in 31 iterations.

Fit data written to: B0.4_7-9min_CombinedScan_fitdat.bin
m/z grid written to: B0.4_7-9min_CombinedScan_grid.bin
Massmax: 40028.320789  Massmin: 24982.508064  Mass axis length: 15045
Mass Grid Written: B0.4_7-9min_CombinedScan_massgrid.bin
Masses written: B0.4_7-9min_CombinedScan_mass.txt
Stats and Error written to: B0.4_7-9min_CombinedScan_error.txt

Error: 0.008255

Finished with 31 iterations and 0.000000 convergence in 0.103000 seconds!

UniDec run 0.12s
File Name:  B0.4_7-9min_CombinedScan.csv R Sqaured:  0.9993822090569541
Plot 3: 0.13s
Plot 5: 0.2s
Plot 1: 0.22s
Plot 2: 0.085s
Peak Picking
Apex: 31090.0 Centroid: 31107.59371793942 FWHM Range: [31080.0, 31139.0]
Apex: 31109.0 Centroid: 31108.70248743998 FWHM Range: [31077.0, 31145.0]
Apex: 31129.0 Centroid: 31109.331674182355 FWHM Range: [31076.0, 31148.0]
Apex: 31823.0 Centroid: 31823.34148653972 FWHM Range: [31815.0, 31832.0]
Apex: 31859.0 Centroid: 31857.998296145535 FWHM Range: [31845.0, 31870.0]
Apex: 31897.0 Centroid: 31853.2739476025 FWHM Range: [31811.0, 31909.0]
Apex: 31930.0 Centroid: 31930.067552155604 FWHM Range: [31916.0, 31944.0]
Apex: 31991.0 Centroid: 32022.471294932457 FWHM Range: [31982.0, 32071.0]
Apex: 32009.0 Centroid: 32023.325776679834 FWHM Range: [31981.0, 32074.0]
Apex: 32031.0 Centroid: 32023.687400030783 FWHM Range: [31981.0, 32075.0]
Apex: 32065.0 Centroid: 32036.943094102644 FWHM Range: [31979.0, 32117.0]
Apex: 32096.0 Centroid: 31953.4425736396 FWHM Range: [31810.0, 32144.0]
Apex: 32163.0 Centroid: 32182.973714408676 FWHM Range: [32151.0, 32217.0]
Apex: 32207.0 Centroid: 31974.47890818438 FWHM Range: [31810.0, 32218.0]
Apex: 32725.0 Centroid: 32726.2381047803 FWHM Range: [32717.0, 32736.0]
Apex: 32761.0 Centroid: 32762.13910467876 FWHM Range: [32747.0, 32778.0]
Apex: 32800.0 Centroid: 32754.30327164134 FWHM Range: [32713.0, 32812.0]
Apex: 32836.0 Centroid: 32765.741009572776 FWHM Range: [32712.0, 32850.0]
Apex: 32893.0 Centroid: 32924.24279873507 FWHM Range: [32881.0, 32975.0]
Apex: 32931.0 Centroid: 32924.68238479144 FWHM Range: [32879.0, 32978.0]
Apex: 32967.0 Centroid: 32841.888963551464 FWHM Range: [32712.0, 33042.0]
Apex: 33011.0 Centroid: 32859.65685022047 FWHM Range: [32711.0, 33109.0]
Apex: 33029.0 Centroid: 32859.44699985565 FWHM Range: [32711.0, 33108.0]
Apex: 33066.0 Centroid: 32859.65685022047 FWHM Range: [32711.0, 33109.0]
Apex: 33099.0 Centroid: 32862.07829020624 FWHM Range: [32710.0, 33124.0]
Apex: 33628.0 Centroid: 33628.75971247903 FWHM Range: [33619.0, 33639.0]
Apex: 33664.0 Centroid: 33646.55248211837 FWHM Range: [33617.0, 33682.0]
Apex: 33702.0 Centroid: 33658.6110746319 FWHM Range: [33615.0, 33718.0]
Apex: 33737.0 Centroid: 33712.992655702394 FWHM Range: [33614.0, 33873.0]
Apex: 33795.0 Centroid: 33713.95362983125 FWHM Range: [33613.0, 33877.0]
Apex: 33811.0 Centroid: 33713.95362983125 FWHM Range: [33613.0, 33877.0]
Apex: 33834.0 Centroid: 33718.6954218247 FWHM Range: [33613.0, 33896.0]
Apex: 33910.0 Centroid: 33735.52919723671 FWHM Range: [33611.0, 33979.0]
Apex: 33969.0 Centroid: 33741.69490383713 FWHM Range: [33610.0, 34020.0]
Apex: 34010.0 Centroid: 33741.99535781033 FWHM Range: [33607.0, 34024.0]
Apex: 34530.0 Centroid: 34530.30350755453 FWHM Range: [34520.0, 34541.0]
Apex: 34568.0 Centroid: 34548.07323270673 FWHM Range: [34518.0, 34583.0]
Apex: 34605.0 Centroid: 34560.03430126617 FWHM Range: [34516.0, 34620.0]
Apex: 34639.0 Centroid: 34571.39066161203 FWHM Range: [34514.0, 34662.0]
Apex: 34714.0 Centroid: 34602.50790443337 FWHM Range: [34513.0, 34769.0]
Apex: 34739.0 Centroid: 34604.68426459892 FWHM Range: [34510.0, 34781.0]
Apex: 35432.0 Centroid: 35449.83829878893 FWHM Range: [35422.0, 35481.0]
Apex: 35469.0 Centroid: 35451.2402212688 FWHM Range: [35420.0, 35487.0]
Apex: 35507.0 Centroid: 35464.36973967245 FWHM Range: [35418.0, 35525.0]
Apex: 35541.0 Centroid: 35476.933778857296 FWHM Range: [35415.0, 35570.0]
Apex: 35585.0 Centroid: 35496.54342663261 FWHM Range: [35412.0, 35650.0]
Plot 2: 0.11s
Plot 4: 0.084s
Saved Figure:  B0.4_7-9min_CombinedScan_Figure1.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure2.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure3.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure4.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure5.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure6.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure1.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure2.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure3.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure4.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure5.pdf
Saved Figure:  B0.4_7-9min_CombinedScan_Figure6.pdf
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (MiKTeX 2.9.7140 64-bit)
entering extended mode
(B0.4_7-9min_CombinedScan_report.tex
LaTeX2e <2018-12-01>
("C:\Program Files\MiKTeX 2.9\tex/latex/base\article.cls"
Document Class: article 2018/09/03 v1.4i Standard LaTeX document class
("C:\Program Files\MiKTeX 2.9\tex/latex/base\size10.clo"))
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics\graphicx.sty"
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics\keyval.sty")
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics\graphics.sty"
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics\trig.sty")
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics-cfg\graphics.cfg")
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics-def\pdftex.def")))
("C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsmath.sty"
For additional information on amsmath, use the `?' option.
("C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amstext.sty"
("C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsgen.sty"))
("C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsbsy.sty")
("C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsopn.sty"))
("C:\Program Files\MiKTeX 2.9\tex/latex/amsfonts\amssymb.sty"
("C:\Program Files\MiKTeX 2.9\tex/latex/amsfonts\amsfonts.sty"))
(C:\Users\Ruisi\AppData\Roaming\MiKTeX\2.9\tex/latex/paracol\paracol.sty)
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics\color.sty"
("C:\Program Files\MiKTeX 2.9\tex/latex/graphics-cfg\color.cfg"))
(C:\Users\Ruisi\AppData\Roaming\MiKTeX\2.9\tex/latex/colortbl\colortbl.sty
("C:\Program Files\MiKTeX 2.9\tex/latex/tools\array.sty"))
("C:\Program Files\MiKTeX 2.9\tex/latex/geometry\geometry.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\ifpdf.sty")
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\ifvtex.sty")
("C:\Program Files\MiKTeX 2.9\tex/generic/ifxetex\ifxetex.sty")
("C:\Program Files\MiKTeX 2.9\tex/latex/geometry\geometry.cfg"))
(B0.4_7-9min_CombinedScan_report.aux)
(C:\Users\Ruisi\AppData\Roaming\MiKTeX\2.9\tex/context/base/mkii\supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) ("C:\Program Files\MiKTeX 2.9\tex/latex/oberdiek\epstopdf-base.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\infwarerr.sty")
("C:\Program Files\MiKTeX 2.9\tex/latex/oberdiek\grfext.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\kvdefinekeys.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\ltxcmds.sty")))
("C:\Program Files\MiKTeX 2.9\tex/latex/oberdiek\kvoptions.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\kvsetkeys.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\etexcmds.sty"
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\ifluatex.sty"))))
("C:\Program Files\MiKTeX 2.9\tex/generic/oberdiek\pdftexcmds.sty"))
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex

Underfull \hbox (badness 10000) in paragraph at lines 13--17


! LaTeX Error: Unknown graphics extension: .4_7-9min_CombinedScan_Figure1.pdf.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...

l.19 ....48]{B0.4_7-9min_CombinedScan_Figure1.pdf}
                                                  \\
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on B0.4_7-9min_CombinedScan_report.log.

Scale problem

Dear colleagues,
Could you help me with the scale?
In the plot of deconvoluted mass spectra, the upper side is a little bit cut.
Please see the attached figure.
Is it possible to overcome this problem?

Best,
Alexander

mzML files won't open - error

I just downloaded UniDec and when I try to open a mzML file, I'm getting a Unicode error.

Could not import data reader: unidectools

I am trying to create a script to call UniDec

import unidec
file_name="210112__solveig_AN0.raw.intensityThreshold1000.PPM10.errTolDecimalPlace3.Time20210128191052.MS.txt"
folder="F:/SK/"
eng=unidec.UniDec()
eng.open_file(file_name, folder)
eng.process_data()
eng.run_unidec(silent=True)
eng.pick_peaks()

but getting following error

Could not import data reader: unidectools

UniDec Engine v.4.4.0

UniDec Path: C:\Python385\lib\unidec_bin\UniDec.exe
Opening File:  F:/SK/210112__solveig_AN0.raw.intensityThreshold1000.PPM10.errTolDecimalPlace3.Time20210128191052.MS.txt
Header Length: 1
Loading Time: 13s
Linear False
Data Prep Time: 10s
UniDec Run Error: 3221225477
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-1-b1753304cd2e> in <module>
      6 eng.process_data()
      7 eng.run_unidec(silent=True)
----> 8 eng.pick_peaks()

~\GD\scripts\unidec.py in pick_peaks(self)
    431         self.export_config()
    432         # Detect Peaks and Normalize
--> 433         peaks = ud.peakdetect(self.data.massdat, self.config)
    434         if len(peaks) > 0:
    435             self.setup_peaks(peaks)

C:\Python385\lib\unidec_modules\unidectools.py in peakdetect(data, config, window, threshold)
   1658     peaks = []
   1659     length = len(data)
-> 1660     maxval = np.amax(data[:, 1])
   1661     for i in range(0, length):
   1662         if data[i, 1] > maxval * threshold:

IndexError: too many indices for array

must mention that i did a brute force-installation git clone https://github.com/michaelmarty/UniDec followed by move unidec* folder/py scripts to lib, probably that is the issue? What is the proper way to install? I did try

pip install pymzml 
pip install natsort 
pip install unidec 

without success...

Unable to output the DScore for each deconvoluted mass

I couldn't find a way to get the DScore for each deconvoluted mass from UniDec.
The _error.txt file contains only the Uniscore but not the DScore; is there a function, parameter, or way to output the DScore for each deconvoluted mass? Or is it the second column in the _mass.txt file? (Probably not)

Hope someone can help with this issue. Thanks!

Error/bug appears using export_params() and autorun() on docker container

Hi,
I'm using the latest version of the docker container and trying to get the DScore of the peak using functions export_params() and autorun(). I tested it with the example data "BSA.txt".

The export_params() would return an error "IndexError: too many indices for array: array is 2-dimensional, but 3 were indexed"; when using autorun(), it returns the BSA_peakparam.dat file but the DScore is always with 0.

It works using unidec GUI with Analysis/Export peaks parameters and Data, so it could be an error in the docker file.

Thanks

Using export_params():

import unidec
eng=unidec.UniDec()
eng.open_file("BSA.txt", "./unidec_test")
eng.process_data()
eng.run_unidec(silent=True)
eng.pick_peaks()
eng.get_charge_peaks()
eng.export_params()
Traceback (most recent call last):
File "", line 1, in
File "/opt/UniDec/unidec/engine.py", line 759, in export_params
ud.dataexport(mztab[:, :, 1], self.config.outfname + "_chargedata.dat")
IndexError: too many indices for array: array is 2-dimensional, but 3 were indexed

Using autorun():

Code:
import unidec
eng=unidec.UniDec()
eng.open_file("BSA.txt", "./unidec_test")
eng.autorun(auto_peak_width=False, silent = True)

BSA_peakparam.dat:
Mass MassStdGuess AvgCharge StdDevCharge Height Area MassCentroid MassFWHM MassErrorBetweenZ DScore
66430.000000 12.679518 14.917080 0.634718 100.000000 100.000000 66425.461394 30.000000 0.000000 0.000000

Using GUI:

BSA_peakparam.dat:
Mass MassStdGuess AvgCharge StdDevCharge Height Area MassCentroid MassFWHM MassErrorBetweenZ DScore
66211.000000 14.909570 14.909570 0.610480 1.331321 1.271188 66211.556002 25.000000 0.414636 0.914815
(...)

Error 3e+09

Hi,

For most of my files, UniDec works fine. But occasionally I try to process a file (recorded the same way as the others), and when I press 'Run UniDec' I get an Error 3e+09 and it does not run. All of the files are Thermo, MS1 .raw files aquired at 15K orbitrap resolution (I also tried with .txt, no luck). Most of the files are fine, occasionally I get one that UniDec does not like at all. Have tried giving UniDec all permissions, uninstalling/reinstalling, installing older versions and turning computer off/on again- nothing works.

Does anyone know a way around this?

Thanks!

RNA or DNA deconvolution

Can I do RNA MS spectrum deconvolution and how to change the settings?

Use command line to process the entire process

Hi,

I am trying to run UniDec with command lines to process hundreds of Thermo raw files.
Using the GUI, a Thermo raw file is automatically converted into text file when I open it using GUI. Is is possible to run this file conversion step from command line instead of using the GUI? Here is the output message shown below after opening the raw file from GUI.

Opening:  test.raw
Opening File:  C:\IMS\Test\test.raw
Output Directory: C:\IMS\Test\test_unidecfiles
Launching Thermo Importer. If it fails after this step, try this:
Delete your whole UniDec folder but keep the zip file.
Right click on the zip file and open properties. You should see a box to Unblock it. Check that.
Click ok. Unzip it again. Try it once more.
Reading Thermo Data: C:\IMS\Test\test.raw