erdogant/findpeaks

The detection of peaks and valleys in a 1d-vector or 2d-array (image)

Hi there! I am sharing my knowledge with the world through my blogs and open-source GitHub projects.

Your ❤️ is important to keep maintaining my packages. It is awsome that there are readily millions of downloads but to keep the libraries alive, I often need to make all kinds of fixes. This can eat up my entire weekend and evenings. Yes, I do this for free and in my free time! There are various ways you can help. You can report bugs/ issues, or even better help out with fixing bugs or maybe adding new features! If you don't have the time or maybe you are still learning, you can also take a Medium Membership using my referral link to keep reading all my hands-on blogs and learn more :-) If you don't need that, there is always an easy way with Coffee :-) Cheers!

A structured list of my repos

All Repos can be found in the Repositories section. If Sphinx pages are available, the link will directly go to the documentation pages.

Statistics	Machine learning	(Time)Series	Vizualization	Utils	API
bnlearn	clusteval	findpeaks	d3graph	df2onehot	googletrends
hnet	classeval	temporalrank	d3heatmap	pypickle	slacki
distfit	hgboost	caerus	treeplot	ismember
pca	clustimage		kaplanmeier	irelease
thompson	undouble		flameplot	pypiplot
benfordslaw			worldmap	dicter
			colourmap
			imagesc
			scatterd
			d3blocks

Find here my Pypi download stats

Overview of open issues

findpeaks's Issues

Plot Doesn't Display?

The below code works without error, however the plot command doesn't do anything; no plot appears or saves anywhere and the script just ends. It gives a A value is trying to be set on a copy of a slice from a DataFrame warning for some reason, even though I'm feeding it a list? I have the latest matplotlib version, running on Windows 10, Python 3.6.

Code

# Load library
from findpeaks import findpeaks
# Data
X = [10,11,9,23,21,11,45,20,11,12]
# Initialize
fp = findpeaks(method='peakdetect', lookahead=1)
results = fp.fit(X)
# Plot
fp.plot()

Output

[findpeaks] >Finding peaks in 1d-vector using [peakdetect] method..
C:\...\Python\Python37\lib\site-packages\pandas\core\indexing.py:670: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  iloc._setitem_with_indexer(indexer, value)

C:\Users\...>

Ascending/Descending Manifold

Currently the result['df']['labx'] labels the ascending manifold in the Morse-Smale complex. The labx should be rename to its technical term.

Since the implementation goes through the data both top-down and bottom-up, labeling of descending manifold should be supported as well.

Removing output messages during runtime

Currently, even while using the verbose = 0 option on findpeaks, I still regularly get outputs that look something like 65536it [00:01, 48554.31it/s] when running.

For added context, the exact line I am running is:
fp = findpeaks(method='topology', scale=False, denoise = None, togray = False, verbose = 0)
and this is being ran on Windows 10 using Python 3.7.3.

Is there a way to remove this?

Severe service deterioration for 2.6.* topology models

I don't know what changes induced this precisely, but anything >v.2.54 (I cannot test at 2.5.5, due to #27) leads to very poor performace of the topology variant.

from findpeaks import findpeaks

#smth = your dummy data here

firstn = 1
fp = findpeaks(method="topology", verbose=0)
df_peak = fp.fit(smth)["df"]
df_peak[
        (df_peak["rank"] != 0) & (df_peak["score"] != 0.0) & (df_peak["valley"] == False)  # noqa: E712
    ].sort_values("rank")[0:firstn]

Picture perferct results out of the box with 2.5.4, green X at positions of first peak:

Garbage with 2.6.0+ (2.5.5 not testable):

caerus crash error

The last version of caerus==0.1.8
At file path 'https://github.com/erdogant/caerus/tree/master/caerus/utils)/csplots.py`, the function name is def _plot_graph(out, figsize=(15,8)):
But your code in findpeaks.py is:csplots._plot_graph(self.results['model'].results, figsize=self.figsize, xlabel=xlabel, ylabel=ylabel)
Then, will get follow error code: TypeError: _plot_graph() got an unexpected keyword argument 'xlabel'.

Please remove the xlabel and ylabel to fix it.

method=topology fails in 2.5.5.

This update broke my working app. I was on 2.5.3 before. I also tested successfully with 2.5.4.

So after upgrading 2.5.4. -> 2.5.5 "topology" method breaks. For spurious reasons it wants to store a vector of Y.shape = (20,2), albeit X is of X.shape = (20,) . This breaks the method as it expects a 1D-vector.

from the official examples:

# Load library
from findpeaks import findpeaks
# Data
X = [10,11,9,23,21,11,45,20,11,12]
# Initialize
fp = findpeaks(method='peakdetect', lookahead=1)
results = fp.fit(X)
# Plot
fp.plot()

fp = findpeaks(method='topology', lookahead=1)
results = fp.fit(X)
fp.plot()
fp.plot_persistence()

Running the above on Python 3.10, with uptodate pandas:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3790, in get_loc
    return self._engine.get_loc(casted_key)
  File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'rank'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/frame.py", line 4261, in _set_item_mgr
    loc = self._info_axis.get_loc(key)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3797, in get_loc
    raise KeyError(key) from err
KeyError: 'rank'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/findpeaks/findpeaks.py", line 256, in fit
    results = self.peaks1d(X, x=x, method=self.method)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/findpeaks/findpeaks.py", line 348, in peaks1d
    self.results, self.args = self._store1d(X, Xraw, x, result)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/findpeaks/findpeaks.py", line 375, in _store1d
    dfint['rank'] = result['Xranked']
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/frame.py", line 4091, in __setitem__
    self._set_item(key, value)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/frame.py", line 4314, in _set_item
    self._set_item_mgr(key, value, refs)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/frame.py", line 4264, in _set_item_mgr
    self._mgr.insert(len(self._info_axis), key, value, refs)
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 1328, in insert
    raise ValueError(
ValueError: Expected a 1D array, got an array with shape (148, 2)

Logging printing with verbose = 0

When calling
findpeaks.findpeaks(method='mask', verbose=0)
logging still outputs because the call in findpeaks.peaks2d for mask is
result = stats.mask(Xproc, limit=self.limit)

The function stats.mask is defined as
def mask(X, limit=0, verbose=3):
and has
if verbose>=3: print('[findpeaks] >Detect peaks using the mask method with limit=%s.' %(limit))

Since the verbose input isn't passed to mask than the output still prints.

Please update the function call to
result = stats.mask(Xproc, limit=self.limit, verbose=self.verbose)

Thanks.

scipy ifft import of peakdetect (stale) breaks findpeaks

Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy
>>> scipy.__version__
1.12.0
>>> import findpeaks

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/findpeaks/__init__.py", line 1, in <module>
    from findpeaks.findpeaks import findpeaks
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/findpeaks/findpeaks.py", line 11, in <module>
    from peakdetect import peakdetect
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/peakdetect/__init__.py", line 3, in <module>
    from .peakdetect import *
  File "/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/peakdetect/peakdetect.py", line 5, in <module>
    from scipy import fft, ifft
ImportError: cannot import name 'ifft' from 'scipy' (/home/ubuntu/miniconda3/envs/datamore/lib/python3.10/site-packages/scipy/__init__.py)

findpeaks breaks with newer scipy, because in more recent versions of scipy, the way Fourier transform functions (like fft and ifft) are imported has changed. Instead of importing fft and ifft directly from scipy, they should be imported from scipy.fft.

This should be addressed at peakdetect, but it's archived.

Limiting z axis in plot_mesh

This is perhaps an easy/stupid question, but how to I limit the z-axis. I have transmission images, ranging between 0 and 1 on the z-axis. When using:

fp = findpeaks.findpeaks(method='topology', scale=False, denoise=None, togray=False, imsize=False, window=15)
fp.fit(transmission)
fp.plot_mesh(wireframe=False, title='Test', cmap='RdBu', view=(70,5))

I run into the issue that my z-axis goes from -5000 to 15000 and thus everything looks the same. Thank you for your help and the awesome tool!

Add `delta` parameter for peakdetect

Currently there appears to be no way to pass the delta parameter to the peakdetect algorithm (which is implemented in @sixtenbe 's code). So findpeaks(method='peakdetect') seems to use the default delta=0.
Adding a keyword argument to findpeaks to allow passing the delta parameter to peakdetect will give more flexibility.

Plot Interpolation?

I am probably missing something, but how do I plot the df_interep?
It does not seem to be popping up as nicely as your examples. (At least for me)
The picture looks the same with interpolation or without it.
Thank you!
-Mike

findpeaks.fit applied to 2D data returns wrong coordinates

Hi and thanks for developing this project. I want to get the peaks in my data using persistent topology, but the resulting dictionary contains the wrong coordinates for the peak locations. Consider the following minimal example:

import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage import gaussian_filter

from findpeaks import findpeaks

rng = np.random.default_rng(42)

x = rng.normal(size=(50, 50))
x = gaussian_filter(x, sigma=10.)
x -= x.min()
x = 255 * x / x.max()

fp = findpeaks(method="topology", denoise=None, verbose=0)
results = fp.fit(x)

plt.close("all")
plt.figure(constrained_layout=True)
plt.imshow(x, aspect="auto", cmap="coolwarm", interpolation="none", vmin=0, vmax=255)
plt.axis("off")
plt.show()

The global maximum of the data is at (0, 21) (i.e., top row, about half-way from the left). But when I print results["persistence"], I get:

    x    y  birth_level  death_level  score
0   0  255          255            0    255
1  49  182          182           88     94
2  14  253          253          211     42
3   0  212          212          212      0

If I understood things correctly, y should denote the y-coordinate in pixel-space, but it seems to just be a copy of birth_level, which is the value at the pixel where the peak is located.

Is this a bug, or did I do something wrong?

Resizing info displayed on verbosity < 3.

Running findpeaks with verbose=0 still prints out [findpeaks] >Resizing image to (100, 100).

I would expect this information not to be displayed below verbose=3.

iteration speed printout displayed at low verbosity.

The printout 10000it [00:00, 46508.35it/s] is displayed at any verbosity level greater than 0.

From the API reference, I would expect this only to print at verbose=3 or higher.

Index error in findpeaks(method='caerus') using pandas 1.4.1

X = np.sin(np.linspace(0,1,100))

from findpeaks import findpeaks
fp = findpeaks(method='caerus', params_caerus={'minperc': 5, 'window': 50})
results = fp.fit(X)

gives the following error
IndexError: index 100 is out of bounds for axis 0 with size 100

points to lines in caerus/utils/csutils.py
201 for k in range(0,len(results['loc_stop'][i])):
202 df['labx'].iloc[np.arange(results['loc_stop'][i][k][0], results['loc_stop'][i][k][1])]=i+1

I'm using pandas version 1.4.1.

However,

from caerus import caerus

cs = caerus(window=50, threshold=0.5, minperc=10, nlargest=1)
cs.fit(modeldev.S.tdata['adjclose'].values, verbose=3)
cs.plot()

runs with no error with the same pandas

Citation improvement

Hey @erdogant ,

I've just found this lib and still experimenting with it, but let me suggest two other ways for the citation.

1. Zenodo; it provides a DOI for every version, better citation entry in multiple format, tracks the papers that uses your code and it's free
   • see SciPy's record https://zenodo.org/record/5725464
2. Add CITATION.cff to the repo; it's a nice GitHub feature

Regards,

How detect the TOPs?

Hi,
Thank you first for your great library.
I am trying to simmulate it with the documents.
The problem is there are BOTTOM detections on the graph.
I used my data instread of yours and other is the same on the document.
Please let me know how to adjust them.

wrong number of peaks using topology method.

Thank you first for your great library. I'm trying to find number of peaks. The code is below:

My origin image is like this:

fp.plot() shows:

fp.plot_mesh() shows:

It should have 6 peaks but only find 5 (with a wrong peak). Can you please explain it for me? I'll appreciate your help.

To limit the distance between adjacent peak and valley points when finding peak & valley points

Hello
How are you?
Thanks for contributing to this project.
Does this repo support to limit the distance between adjacent peak and valley points when finding peak & valley points?

As u can see in the above figure, the distance between the first peak and valley points is very small and I want to ignore these points.

Extra index

I believe there is an extra index at

findpeaks/findpeaks/utils/stats.py

Line 355 in 4e4bf8c

max_peaks = np.array(list(map(lambda x: [x[0][0], x[1]], groups0)))

[x[0][0], x[1]] should be [x[0], x[1]]

The same goes for min_peaks.

facebook stock example not working (caerus.__init__() got an unexpected keyword argument 'delta')

the facebook stock example isnt working and giving error, seems to be originating from fp.fit(X)
edit: it can be fixed by adding params={'minperc':10} in findpeaks, update the documentation

pip installation does not include joblib

Installing findpeaks through pip doesn't set up the required joblib package.

I know very little of pip stuff, but it looks like the package is missing from setup.py (it's in requirements.txt).

fit method didn't find the peak I expected

Thanks for this great package.

My issue is similar to #16

here is my image

from findpeaks import findpeaks
import os
import cv2
path = os.path.join('tmp_peak.png')
# path=r'tmp_peak.PNG.png'
print(os.path.isfile(path))
fp = findpeaks(method='topology', denoise='lee_enhanced', window=5, whitelist='peak', limit=60)
# X = fp.imread(path)
X = cv2.imread(path)
results = fp.fit(X)
result_df = results['persistence']
peak = result_df.index[result_df['peak']==True].tolist()
print(result_df.loc[peak])
fp.plot_persistence()
fp.plot()
fp.plot_mesh()

I have been folowing your provided code, expecting that (x,y) = (62, 95), (186, 95), (310, 95), (433,95) would have similar scores,

However, after trying all the denoise methods, it couldn't find the peak at (310,95)

Do you have any suggestions for me?

Thanks!

Feature request - Multiplicative Noise Lee Despeckling Filter

We would like to propose the implementation of the Multiplicative Noise Lee Despeckling Filter as a new feature. Multiplicative noise is commonly encountered in SAR images, particularly in Sentinel-1 satellite data. It is crucial to address this noise during the pre-processing stage. Currently, popular software such as SNAP and ArcGIS offer the Multiplicative Noise Lee Despeckling Filter, but we haven't found an open-source Python implementation available for integration into our preprocessing pipeline. Hence, we aim to contribute to this package by adding the filter and making it accessible to the wider community.

We would like to hear from you what do you think for this contribution.

First and last data point always labeled as a valley

The first and last points in a series are always labelled as a valley. See the output of this series modified from the 1d example:

from findpeaks import findpeaks
# Data
X = [1153,672,501,1068,1110,574,135,23,3,47,252,812,1182]
# Initialize
fp = findpeaks(lookahead=1)
results = fp.fit(X)
# Plot
fp.plot()
results.get('df')

data-dependent issues with interpolate

@erdogant: thanks for the great package!

I've encountered a small issue where interpolate causes problems that depend on the input data. Here's a small reproducible example using numpy:

import numpy as np
from findpeaks import findpeaks

np.random.seed(200)
peakDat = np.random.randint(200, size = 400)

fp = findpeaks(method = 'topology',
               interpolate = 10, 
               lookahead = 1)

results = fp.fit(peakDat)

This produces the following error: ValueError: cannot set using a list-like indexer with a different length than the value

By contrast, a different seed works fine with these arguments:

import numpy as np
from findpeaks import findpeaks

np.random.seed(100)
peakDat = np.random.randint(200, size = 400)

fp = findpeaks(method = 'topology',
               interpolate = 10, 
               lookahead = 1)

results = fp.fit(peakDat)

Changing the interpolate factor from 10 can fix the issue for a given data set, but it's unclear from the documentation if there's a better way to avoid having to tweak the settings at the data-set level.