wesleybowman/utide

Python distribution of the MatLab package UTide

UTide

Python re-implementation of the Matlab package UTide.

Still in heavy development--everything is subject to change!

Note: the user interface differs from the Matlab version, so consult the Python function docstrings to see how to specify parameters. Some functionality from the Matlab version is not yet available. For more information see:

Codiga, D.L., 2011. Unified Tidal Analysis and Prediction Using the
UTide Matlab Functions. Technical Report 2011-01. Graduate School
of Oceanography, University of Rhode Island, Narragansett, RI.
59pp.
ftp://www.po.gso.uri.edu/pub/downloads/codiga/pubs/2011Codiga-UTide-Report.pdf

UTide v1p0 9/2011 [email protected]
http://www.po.gso.uri.edu/~codiga/utide/utide.htm

Installation

pip install utide

If you are using conda,

conda install utide --channel conda-forge

The public functions can be imported using

from utide import solve, reconstruct

A sample call would be

from utide import solve

coef = solve(
    time,
    time_series_u,
    time_series_v,
    lat=30,
    nodal=False,
    trend=False,
    method="ols",
    conf_int="linear",
    Rayleigh_min=0.95,
)

For more examples see the notebooks folder.

utide's Issues

New release with latest master

@wesleybowman fixing #30 solves a problem for many people on Windows. Would you mind tagging a new release? Maybe also send it to PyPI?

Pythonic naming: remove all the 'ut_' prefixes

@wesleybowman Would you support a PR to simplify the naming by removing all use of "ut_"? The module would then be used as

import utide as ut
...
coefs = ut.solve(...)

This is how we are handling the TEOS-10 gsw module.

Problem running utide

I'm trying to use your utide and can't get it to work. I use python 3.6, anaconda distribution, and installed utide from the command line as "pip install utide"

I read in a year-long tidal time seriesand make t and z vectors that are standard one-dimensional ndarrays, but get the error messages below. I figure the problem may be with my installation, or my python version, or my code, but I thought I would ask you first before I run down too many rabbit holes.

The function call was:
a = utide.solve(t, z, v=None,
lat=49.337,
nodal=False,
trend=False,
method='ols',
conf_int='linear',
Rayleigh_min=0.95)

Here are the error messages:

In [26]: run obs_plot_canada_utide.py
solve:

TypeError Traceback (most recent call last)
~/Documents/ptools/tide/obs_plot_canada_utide.py in ()
55 method='ols',
56 conf_int='linear',
---> 57 Rayleigh_min=0.95)
58
59 if False:

/Applications/anaconda/lib/python3.6/site-packages/utide/_solve.py in solve(t, u, v, lat, **opts)
146 compat_opts = _translate_opts(newopts)
147
--> 148 coef = _solv1(t, u, v, lat, **compat_opts)
149
150 return coef

/Applications/anaconda/lib/python3.6/site-packages/utide/_solve.py in _solv1(tin, uin, vin, lat, **opts)
163 # opt['cnstit'] = cnstit
164 nNR, nR, nI, cnstit, coef = ut_cnstitsel(tref, opt['rmin']/(24*lor),
--> 165 opt['cnstit'], opt['infer'])
166
167 # a function we don't need

/Applications/anaconda/lib/python3.6/site-packages/utide/constituent_selection.py in ut_cnstitsel(tref, minres, incnstit, infer)
42
43 ii = np.isfinite(const.ishallow)
---> 44 const.freq[~ii] = np.dot(const.doodson[~ii, :], ader) / 24
45
46 for k in ii.nonzero()[0]:

TypeError: NumPy boolean array indexing assignment requires a 0 or 1-dimensional input, input has 2 dimensions

solve failing when conf_int='none'

solve is failing when conf_int='none'.

For example, if I try changing conf_int='linear' to conf_int='none' in cell [5] of the utide_uv_example.ipynb notebook, I get:

solve: matrix prep ... solution ... diagnostics ... 
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-10-359a0567fa71> in <module>
      7              method='ols',
      8              conf_int='none',
----> 9              Rayleigh_min=0.95,)

~/miniconda3/envs/pangeo/lib/python3.7/site-packages/utide/_solve.py in solve(t, u, v, lat, **opts)
    198     compat_opts = _process_opts(opts, v is not None)
    199 
--> 200     coef = _solv1(t, u, v, lat, **compat_opts)
    201 
    202     return coef

~/miniconda3/envs/pangeo/lib/python3.7/site-packages/utide/_solve.py in _solv1(tin, uin, vin, lat, **opts)
    377     # Diagnostics.
    378     if not opt['nodiagn']:
--> 379         coef, indPE = ut_diagn(coef, opt)
    380 
    381     # Re-order constituents.

~/miniconda3/envs/pangeo/lib/python3.7/site-packages/utide/diagnostics.py in ut_diagn(coef, opt)
     15 
     16         SNR = (coef['Lsmaj']**2 + coef['Lsmin']**2) / (
---> 17             (coef['Lsmaj_ci']/1.96)**2 +
     18             (coef['Lsmin_ci']/1.96)**2)
     19 

KeyError: 'Lsmaj_ci'

better solve performance with lsqr

I get better performance when I use the scipy.sparse.linalg.lsqr iterative solver instead of np.linalg.lstsq.

I get 16.7s vs 19.8s for a test with a big data set.

how to specify only calculate M2, S2, K1, O1

I am wondering how to specify to calculate only a small number tides, instead of many tides?

warning with last version of numpy

When updating with the last numpy version '1.20.0rc2' I've got a error message from Utide
utide\harmonics.py:184: RuntimeWarning: invalid value encountered in fmod np.fmod(V, 1, out=V)
It is not critical yet, and it works, but in the futur ...

Update the README to include conda install instructions

We are a part of the condaforge, so we should make sure our users know they can use conda to install the package.

utide.solve and utide.construct

Greetings! May I ask a question about utide.reconstruct? I have a 1-d data of sea level and I am trying to obtain the astronomical tide by using uTide. My code more or less looks like this.

 time1=mdates.date2num(data.index.to_pydatetime())
    
    coef1=utide.solve(time1,data['SeaLevel'].values,
                      lat=lat1,
                      method='irls',
                      conf_int='MC',
                      nodal=True,#include satellite corrections to improve results
                      constit=harmNames
                      )
   
    harmNames_sorted=coef1.name
    tide_aux=utide.reconstruct(time1,coef1,constit=harmNames_sorted)

I am getting a time series, of the tide, that looks like it has much smaller frequency than the astromical tide obtained with Foreman. I see that, in my case, I get a frequency in radians/hour from utide.solve, but utide.reconstruct is using this frequency in cycle per hour (as I can see from the comments in the source code). Hence the smaller frequency. I must have dropped some option somewhere.

Thanks in advance,

Jue

Trying to reorder constituents as in ‘OrderCnstit’

As @rsignell-usgs mentioned in a previous issue, we are trying to analyze the tides at each grid cell in a numerical model solution. We run utide for each grid point but the order of the constituents can be different for each grid point. In the Matlab code, there is the option ‘OrderCnstit’ that allows passing a specified order of constituents when constit is not set to 'auto'. Is there a similar reordering for the python version?

Using twodim with amplitue/phase from another source for predicting currents

I'm trying to predict u and v velocity components from amplitude and phases from a netCDF file for a range of positions.

As such, I have arrays of latitudes (dimension = [nx]), amplitudes ([nx, nconsts]) and phases ([nx, nconsts]).

With some fiddling, I can make a Bunch as follows for defined set of constituents to use:

# What constituents I want to use for the prediction.
constit = ('M2', 'S2', 'N2', 'K2', 'K1', 'O1', 'P1', 'Q1', 'M4', 'MS4', 'MN4')
const_names = ut_constants['const']['name']
utide_consts = ['{:4s}'.format(i) for i in constit]
# The indices of those in the UTide constants.
const_idx = np.asarray([const_names.tolist().index(i) for i in constit])
frq = ut_constants['const']['freq'][const_idx]

coef = Bunch(name=utide_consts, mean=0, slope=0)
coef['aux'] = Bunch(reftime=729572.47916666674, lind=const_idx, frq=frq)
coef['aux']['opt'] = Bunch(twodim=True, nodsatlint=False, nodsatnone=False, 
                           gwchlint=False, gwchnone=False, notrend=False, prefilt=[])
# Populate the relevant bits with my external data.
coef['aux']['lat'] = latitudes
coef['A'] = amplitudes
coef['g'] = phases

Now, when I call reconstruct:

    predicted = reconstruct(times, coef, constit=utide_consts, verbose=False)

I got the following error:

~/.local/lib/python3.6/site-packages/utide/harmonics.py in FUV(t, tref, lind, lat, ngflgs)
    123         astro, ader = ut_astron(tt)
    124 
--> 125         if abs(lat) < 5:
    126             lat = np.sign(lat) * 5
    127 

If I edit utide/harmonics.py at line 125 to:

        try:
            if np.any(np.abs(lat) < 5):
                lat[np.abs(lat) < 5] = np.sign(lat[np.abs(lat) < 5]) * 5
        except ValueError:
            if abs(lat) < 5:
                lat = np.sign(lat) * 5

I get the following:

ValueError                                Traceback (most recent call last)

... edited for clarity ...

~/.local/lib/python3.6/site-packages/utide/_reconstruct.py in reconstruct(t, coef, epoch, verbose, constit, min_SNR, min_PE)
     77                         constit=constit,
     78                         min_SNR=min_SNR,
---> 79                         min_PE=min_PE)
     80 
     81     if v is not None:

~/.local/lib/python3.6/site-packages/utide/_reconstruct.py in _reconstruct(t, goodmask, coef, verbose, constit, min_SNR, min_PE)
    128              coef['aux']['reftime'], coef['aux']['frq'][ind],
    129              coef['aux']['lind'][ind], coef['aux']['lat'], ngflgs,
--> 130              coef['aux']['opt']['prefilt'])
    131 
    132     fit = np.dot(E, ap) + np.dot(np.conj(E), am)

~/.local/lib/python3.6/site-packages/utide/harmonics.py in ut_E(t, tref, frq, lind, lat, ngflgs, prefilt)
     74         V = np.dot(24*(t-tref)[:, None], frq[:, None].T)
     75     else:
---> 76         F, U, V = FUV(t, tref, lind, lat, ngflgs)
     77 
     78     E = F * np.exp(1j*(U+V)*2*np.pi)

~/.local/lib/python3.6/site-packages/utide/harmonics.py in FUV(t, tref, lind, lat, ngflgs)
    130 
    131         j = sat.ilatfac == 1
--> 132         rr[j] *= 0.36309 * (1.0 - 5.0 * slat**2)/slat
    133 
    134         j = sat.ilatfac == 2

ValueError: operands could not be broadcast together with shapes (32,) (546,) (32,) 

Clearly I'm doing something wrong, but I'm not sure what. I might have misunderstood what twodim is for.

Build tests

UTide needs some test to help developers/contributors evaluate the changes quickly.

Problem with version 0.3.0, Python 3.10 and Ubuntu 22.04

Good morning,
I tried my old program with the new version 0.3.0 under Python 3.10 and I cannot make it work
When I try
from utide import solve,reconstruct
I get the error `
Traceback (most recent call last):

File "/tmp/ipykernel_3117/1389739857.py", line 1, in
from utide import solve,reconstruct

File "/home/pascal/.local/lib/python3.10/site-packages/utide/init.py", line 3, in
from ._solve import solve

File "/home/pascal/.local/lib/python3.10/site-packages/utide/_solve.py", line 9, in
from .harmonics import ut_E

File "/home/pascal/.local/lib/python3.10/site-packages/utide/harmonics.py", line 11, in
from ._ut_constants import ut_constants

File "/home/pascal/.local/lib/python3.10/site-packages/utide/_ut_constants.py", line 13, in
ut_constants = loadbunch(_ut_constants_fname, masked=False)

File "/home/pascal/.local/lib/python3.10/site-packages/utide/utilities.py", line 288, in loadbunch
out[k] = _structured_to_bunch(xx[k], masked=masked)

File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 254, in getitem
return format.read_array(bytes,

File "/usr/lib/python3/dist-packages/numpy/lib/format.py", line 743, in read_array
raise ValueError("Object arrays cannot be loaded when "

ValueError: Object arrays cannot be loaded when allow_pickle=False

Do you know why and what can I do ?
Thank you.
Good year.

Pascal

Datetime as input

Hi all,

The numpy function "np.isnan" is incompatible with datetime. The following error is risen when using datetime:

...\utide\ut_slvinit.pyc in ut_slvinit(tin, uin, vin, **opts)

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

I would recommend to replace np.isnan with pd.isnull.

Greetings,

Saulo

All results empty

Version 0.3.0 in some cases returns empty results. As yet I have not figured out what triggers this. Will return to version 0.2.6 for now and will let you know if I find out more.

test_periodogram fails with lowlevel scipy.signal.lombscargle error

If I run the tests (python 3.9.7, scipy 1.7.1) have a failure from test_periodogram.py with error messages like

TypeError: Invalid call to pythranized function `_lombscargle(float64[:], float64[:] (reshaped), float64[:])'

Using comment in scipy/scipy#14420, in periodogram.py modified parameters passed to signal.lombscargle to

    xr = np.real(x)
    xr = np.asarray(xr, dtype=float, order='C')
    xr = np.ndarray(xr.shape, dtype=float, buffer=xr)
    # signal.lombscargle returns "(A**2) * N/4 for a harmonic signal
    # with amplitude A for sufficiently large N."
    # It takes *angular* frequencies as 3rd argument.
    freq_radian = freq * 2 * np.pi
    psdnorm = 2 * delta_t * n / (w ** 2).sum()
    out.Pxx = psdnorm * signal.lombscargle(t, xr, freq_radian)

    if x.dtype.kind == "f":
        return out
    xi = x.imag
    xi = np.asarray(xi, dtype=float, order='C')
    xi = np.ndarray(xi.shape, dtype=float, buffer=xi)
    out.Pyy = psdnorm * signal.lombscargle(t, xi, freq_radian)

Don't know if there a smarter way to achieve this, and once (if?) scipy.signal + pythran interaction gets sorted assume the xr/xi rewrite won't be needed.

Error in Finding coefficient

I am getting this error while running thr utide code in Jupyter. The tidal heights are returned as zero values. How can it be solved.

ValueError when trying to import with recent numpy

When trying to import utide on a clean conda env with a recent version of numpy (e.g. 1.16.3) I get the following error:

>>> import utide
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/utide/__init__.py", line 3, in <module>
    from ._solve import solve
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/utide/_solve.py", line 9, in <module>
    from .harmonics import ut_E
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/utide/harmonics.py", line 11, in <module>
    from ._ut_constants import ut_constants
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/utide/_ut_constants.py", line 13, in <module>
    ut_constants = loadbunch(_ut_constants_fname, masked=False)
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/utide/utilities.py", line 288, in loadbunch
    out[k] = _structured_to_bunch(xx[k], masked=masked)
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/numpy/lib/npyio.py", line 262, in __getitem__
    pickle_kwargs=self.pickle_kwargs)
  File "/Users/dnowacki/miniconda3/envs/utide/lib/python3.7/site-packages/numpy/lib/format.py", line 692, in read_array
    raise ValueError("Object arrays cannot be loaded when "
ValueError: Object arrays cannot be loaded when allow_pickle=False

Quick Googling suggests this started in numpy 1.16.3.

Variable default value should not be a list

We have two variables with a list as default value here:

https://github.com/wesleybowman/UTide/blob/master/utide/robustfit.py#L7

That is not recommended (See http://docs.python-guide.org/en/latest/writing/gotchas).

@wesleybowman Also, this function does not return anything and this module appears to incomplete. I am removing it from the tests for now. But we should add it again once it is done.

Allow multiple columns of time series as input for prediction

We have a use case where we want to analyze the tides at each grid cell in a numerical model, so we have lots of time series with the same time base. How hard would it be to allow a matrix (or list of arrays) as input data to the solver?

README may need minor update

The example provided fails with:

>>> from utide import solve, reconstruct
>>> from utide.tests.test_solve import test_roundtrip
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'utide.tests'

verbose not implemented

Hello - I noticed that you have not taken the verbose value that I have set in the newopts.

I have added the following line in my local version:

def _reconstruct(t, goodmask, coef, verbose, constit, min_SNR, min_PE):

twodim = coef['aux']['opt']['twodim']
verbose = coef['aux']['opt']['newopts']['verbose']

Constituents for lunar nodal and perigee cycles

Hi!
I'm interesting in using UTide for harmonic analysis of sea level records for various tide gauge stations. I'm particularly interested in the long-period constituents and especially the longest ones related to the 18.61-year cycle of lunar nodal and the 8.85-year cycle of lunar perigee. Does UTide take into consideration these specific ones?
Thanks!

BUG: Python stops working when importing utide on Windows platform

Hi everyone,

I recently updated my Utide package on Windows 7 64-bit. Now every time I import utide, Python simply stops working and the program closes down. @ocefpaf has done some tests and confirmed the bug. I am using the following versions:

Scipy 0.17.0
Python 2.7.11
UTide 0.1

I ran my scripts on Linux and all went well.

Cheers, Saulo

The Notebooks link at the bottom of the README.rst doesn't link to the notebooks.

This is a small problem. Received an email about no links to the notebooks (I think).

On the README it links to
http://nbviewer.jupyter.org/github/wesleybowman/UTide/tree/npz/notebooks/

and it should link to
https://nbviewer.jupyter.org/github/wesleybowman/UTide/tree/master/notebooks/

Creating an issue just to document

Failure with numpy 1.13

We have a failure in Travis-CI when using the latest numpy:

___________________________________ test_FUV ___________________________________
    def test_FUV():
        x = loadbunch(fname, masked=False)
        # Switch epoch from Matlab to Python
        x.t -= 366
        x.t0 -= 366
    
        for i, flag in enumerate(x.flags):
>           F, U, V = FUV(x.t, x.t0, x.lind-1, x.lat, flag)
tests/test_harmonics.py:29: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda/envs/TEST/lib/python2.7/site-packages/utide/harmonics.py:170: in FUV
    freq = linearized_freqs(tref)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tref = 711859.0
    def linearized_freqs(tref):
        astro, ader = ut_astron(tref)
        freq = const.freq.copy()
>       freq[not_shallow] = np.dot(const.doodson[not_shallow, :], ader) / 24
E       TypeError: NumPy boolean array indexing assignment requires a 0 or 1-dimensional input, input has 2 dimensions
../../../miniconda/envs/TEST/lib/python2.7/site-packages/utide/harmonics.py:29: TypeError
----------------------------- Captured stdout call -----------------------------
i: 0 ngflgs: [0 0 0 0]
i: 1 ngflgs: [0 1 0 0]

Full log https://travis-ci.org/ocefpaf/UTide/jobs/280403569

I'll take a look soon, we should fix that before a new release.

Describe inputs/outputs

Most of the functions do not describe the type of the expected input nor the output.

• Time is a datetime object or a matplotlib date2num?
• The time series are numpy arrays or lists or both?
• Is vin optional? If so can it be initiate as vin=None?

BUG: KeyError: "Update keys don't match existing keys

An error gets thrown when solve is run:

coef = solve(time, u, v, lat, cnstit='auto',
               notrend=True, rmin=0.95, method='ols',
               nodiagn=True, linci=True, conf_int=True)

returns:

KeyError: "Update keys ['cnstit', 'linci', 'nodiagn', 'notrend', 'rmin'] don't match existing keys ['MC_n', 'Rayleigh_min', 'conf_int', 'constit', 'infer', 'method', 'nodal', 'phase', 'robust_kw', 'trend', 'white']"

Trying:

opts = dict(constit = 'auto',
                   notrend = True,
                   rmin = 0.95,
                   conf_int = 'linear',
                   method = 'ols',
                   nodiagn = True,
                   linci = True
                  )

coef = solve(time, u, v, lat=lat, **opts)

still results in the error. A more detailed error report is as follows:

Traceback (most recent call last):

  File "C:/Users/lisa/Desktop/My Work/test/pd0.py", line 456, in <module>

…
    coef = solve(time, u, v, lat=lat, **opts)

  File "c:\users\lisa\appdata\local\continuum\anaconda\lib\site-packages\utide-0.1b0.dev0-py2.7.egg\utide\_solve.py", line 143, in solve

    newopts = _process_opts(opts)

  File "c:\users\lisa\appdata\local\continuum\anaconda\lib\site-packages\utide-0.1b0.dev0-py2.7.egg\utide\_solve.py", line 33, in _process_opts

    newopts.update_values(strict=True, **opts)

  File "c:\users\lisa\appdata\local\continuum\anaconda\lib\site-packages\utide-0.1b0.dev0-py2.7.egg\utide\utilities.py", line 125, in update_values

    self._check_strict(strict, newkw)

  File "c:\users\lisa\appdata\local\continuum\anaconda\lib\site-packages\utide-0.1b0.dev0-py2.7.egg\utide\utilities.py", line 153, in _check_strict

    "Update keys %s don't match existing keys %s" % (bk, ek))

KeyError: "Update keys ['linci', 'nodiagn', 'notrend', 'rmin'] don't match existing keys ['MC_n', 'Rayleigh_min', 'conf_int', 'constit', 'infer', 'method', 'nodal', 'phase', 'robust_kw', 'trend', 'white']"

solve and reconstruct crashing despite "try...except"

Dear all,

Greetings! Thank you so much for your hard work keeping this forum. I would appreciate any guidance on the following issue. My data and code look like this:

time1=
[16707.62164352 16707.62233796 16707.62302083 ... 16800.99834491
16800.99903935 16800.9997338 ]

df_year=
data quality_old quality RES
date
2015-09-29 14:55:10 689.0 1.0 1.0
2015-09-29 14:56:10 717.0 1.0 1.0
2015-09-29 14:57:09 746.0 1.0 1.0
2015-09-29 14:58:10 775.0 1.0 1.0
2015-09-29 14:59:10 803.0 1.0 1.0
... ... ... ... ..
2015-12-31 23:55:37 7550.0 1.0 1.0
2015-12-31 23:56:37 7538.0 1.0 1.0
2015-12-31 23:57:37 7511.0 1.0 1.0
2015-12-31 23:58:37 7523.0 1.0 1.0
2015-12-31 23:59:37 7519.0 1.0 1.0

code:
try:
coef = utide.solve(time1, df_year['data'].values,
lat=lat1,
method='ols',
phase='Greenwich',
conf_int='linear',
trend='False',
constit=constit,
nodal='True',
Rayleigh_min=1,
white=False,
verbose=True)

    tide = np.array((utide.reconstruct(time1, coef, min_SNR=0, verbose=True, constit=constit_out)).h)

except:
pass

utide.solve and utide.reconstruct do not converge/work for every df_year, so I expected the "try...except" to do its job, but utide.solve and utide.reconstruct can end up consuming much memory without stopping, so they crash without letting the "try...except" function properly.

Is there any way to have utide.solve and utide.reconstruct stop whenever necessary?

Best regards,

Jue

Tidal prediction using coef from Utide

Hi,
I am using 1-month observed sea level to extract about 28 harmonic constituents and then can produce a tidal signal for that month. However, when I use coef to predict tide at a timeseries (I use pd.date_range) it seems that I can not predict longer period than 1 month (e.g 6 months or 1 year).

Is that due to I use too short observed SL or is there something else I need to change in the code?

Many thanks.
Dat Pham

Tidal constituents of absolute current velocities?

I am having a dataset of noise on a geophysical ocean bottom station,
which is proportional to the absolute value of the bottom current.

I would like to extract tidal constituents from this dataset, however,
given that I do not have the current vector, but the absolute value,
this is not quite straightforward (as a rough guess, I would expect to
see peaks at twice the frequency, plus tons of overtones). Did you
foresee the option to work on absolute current velocities in utide or
would you have an idea on where to start adding this?

Improve time input handling

Matplotlib now has a settable epoch for its floating point date-time, and numpy datetime64 seems to have settled down to a usable, if not ideal, state, so it is time for some improvements in utide's handling of time inputs.

Referencing Utide

I was wondering if there is a preferred language for referencing utide (this package) in a paper. I only found the reference to the original MATLAB code report by Codiga 2011. I suppose if I use python-utide, I need to refer to both report, and the software. Suggestion?

Utide with dask

Hi

I was wondering if it's possible to use utide for multidimensional dask arrays.
Or if it is in the pipeline to achieve it some day?

It would be really cool to use utide for large datasets instead of single time series.

Thanks.

test_roundtrip() failing on my machine

In working on #68, I ran the test suite (pytest tests) on my local machine, and test_roundtrip() in test_solve.py failed:

>       np.testing.assert_almost_equal(err, 0)
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E        ACTUAL: 0.7073383390219243
E        DESIRED: 0

tests/test_solve.py:80: AssertionError

But if I simply run pytest tests/test_solve.py it passes:

tests/test_solve.py ....                                                                                         [100%]

=============================================== 4 passed in 0.86 seconds ===============================================

Despite the test weirdness, it seemed like this change was not related to the np.load issue I was working on, so I made the PR and it passed the Travis/Appveyor tests. Any idea on why this test would fail on my local machine (fresh conda env using requirements.txt and requirements-dev.txt on macOS) but not on the CI?

Avoid reading `ut_constants.mat` multiple times

ut_constants.mat is loaded in __init__.py and then re-loaded several times throughout the module.

Utide 0.2.3 incompatible with numpy 1.6.3 or greater

I have utide 0.2.3 installed with Miniconda. With numpy 1.6.2, I can import utide with no problems. With numpy 1.6.3 or 1.6.4 I get the following:

Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.16.3'
>>> import utide
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\utide\__init__.py", line 3, in <module>
    from ._solve import solve
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\utide\_solve.py", line 9, in <module>
    from .harmonics import ut_E
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\utide\harmonics.py", line 11, in <module>
    from ._ut_constants import ut_constants
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\utide\_ut_constants.py", line 13, in <module>
    ut_constants = loadbunch(_ut_constants_fname, masked=False)
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\utide\utilities.py", line 288, in loadbunch
    out[k] = _structured_to_bunch(xx[k], masked=masked)
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\numpy\lib\npyio.py", line 262, in __getitem__
    pickle_kwargs=self.pickle_kwargs)
  File "C:\Users\markh\Miniconda3\envs\python37\lib\site-packages\numpy\lib\format.py", line 692, in read_array
    raise ValueError("Object arrays cannot be loaded when "
ValueError: Object arrays cannot be loaded when allow_pickle=False

Python epoch and phase computation

Hello,

Epoch question :
In the solve function the default epoch is defined as "The default corresponds to the Python standard library datetime proleptic Gregorian calendar, starting with 1 at 00:00 on January 1 of year 1"
On the other hand, in the notebook ["utide_real_data_example.ipynb"](url), time is defined as time = mdates.date2num(obs.index.to_pydatetime()) which epoch is correspond to calendar starting with 1 at 00:00 oon January 1 of year 1970.
Finally which epoch is used in the solve function?

Phase question :
I'm having hard time to recover the phase of my signal. I use real data (sea surface elevation in the Bay of biscay)
With t = mdates.date2num(obs.index.to_pydatetime())
and where obs.index=pd.date_range(start='2017-07-09 12:00:00',end='2017-07-30 12:30:00',freq='1min')

I recover the main correct component and their corresponding amplitude, but the phase is different then expected. May this be related to my first question?

I don't know if it's related but I then started by constructing a simple signal with a semi diurnal frequency such as : Acos(wt-p).
I recover my amplitude and pulsation (w) but I couldn't recover the phase p. When I computed the phase of the signal with p= 0, meaning : Acos(wt). The phase was around 114. I was then able to recover p when using a signal: Acos(wt-(p-114)). I thought I might use this trick for my real signal but I could'nt recover a correct phase, even with a signal made of sum of Aicos(wi*t), with i the indices of the main constituent of my signal.

Utide solve process never returns

Hi,

I have been using Utide to solve the output from a tidal model and its going well. However on one grid cell it never solves but also never returns an error so it just sits there (several days by the time I got back to the machine).

I realise this is a bit vague, but wanted to open an issue to see if anyone had similar issues. My current solution is to put a timeout decorator on the utide call so it will timeout. I don't know how common this issue is, since it always happens to the same grid cell.

If I find a better solution or know more I will update here.

NaN output

Hi all,

I compared the performance of the harmonic analysis for the same series using Matlab and Python. I ran "ut_solv" followed by "ut_reconstr" for u and v velocity components. In Matlab, the output was as expected but in Python I got just NaN's. It is hard to find out where the problem is because no errors popped up while running.

Any thoughts?

Cheers,

Saulo

crash with 0 time series

The following code crash:

    import utide
    import numpy as np
    import matplotlib.dates as mdates
    import datetime, dateutil

    print(utide.__version__)

    time = np.arange(0.,30.*86400,60.)
    t0 = datetime.datetime(2011,9,10)
    dtime = [t0+dateutil.relativedelta.relativedelta(seconds=t) for t in time]
    mtime = mdates.date2num(dtime)

    tdat = utide.ut_constants['const']
    iM2 = tdat['name'].tolist().index('M2')
    omega = tdat['freq'][iM2]
    
    eta = 0.*np.cos(2.*np.pi*omega*time/3600.)

    coef = utide.solve(mtime, eta, lat=1., method='ols', conf_int='MC')

with the following error:

(equinox) r2i2n0 datahome/aponte% python tests.py
0.2.2
/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/confidence.py:255: RuntimeWarning: invalid value encountered in true_divide
  Duu = Puu[c] * Duu / np.trace(Duu)
solve: matrix prep ... solution ... Traceback (most recent call last):
  File "tests.py", line 57, in <module>
    test_utide()
  File "tests.py", line 46, in test_utide
    coef = utide.solve(mtime, eta, lat=1., method='ols', conf_int='MC')
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/_solve.py", line 200, in solve
    coef = _solv1(t, u, v, lat, **compat_opts)
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/_solve.py", line 375, in _solv1
    W, m, B, Xu, Yu, Xv, Yv)
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/confidence.py", line 260, in _confidence
    varcov_mCc[c, :, :] = nearestSPD(varcov_mCc[c, :, :])
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/confidence.py", line 139, in nearestSPD
    maxeig = np.abs(np.linalg.eigvals(Ahat)).max()
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 904, in eigvals
    _assertFinite(a)
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 216, in _assertFinite
    raise LinAlgError("Array must not contain infs or NaNs")
numpy.linalg.linalg.LinAlgError: Array must not contain infs or NaNs
(equinox) r2i2n0 datahome/aponte%

New opts not updated! WIP?

Hi,

I am using UTide for python for sometime now! And I have noticed that all the kwargs are not yet available. For instance, if I run the solve function with nodiagn as an argument it runs into error.

I am guessing this is probably due to the newopts dictionary which is still a TODO?

Thanks and Regards,
Amey

Using UTide.reconstruct with other harmonic constituents.

For reasons with which I won't bore you, I'd like to use UTide to predict tidal elevations from another source of harmonic constituent amplitude and phases (namely the TPXO data).

For reconstruct(), I need to supply the coef Bunch with a number of keys. I've figured out what most of those are, but I'm a bit stuck on the confidence intervals. I don't think the TPXO data have confidence intervals for their values but UTide needs those. To what should I set them in their absence?

RuntimeWarning on package load

Awesome package - I really enjoy using this a lot!

With python 3.11 and numpy 1.26.4 I get the following RuntimeWarnings when loading the package:

utide/harmonics.py:16: RuntimeWarning: invalid value encountered in cast
  nshallow = np.ma.masked_invalid(const.nshallow).astype(int)
utide/harmonics.py:17: RuntimeWarning: invalid value encountered in cast
  ishallow = np.ma.masked_invalid(const.ishallow).astype(int) - 1

This can be fixed by moving the cast to int a few lines down where nshallow and ishallow are compressed. Would you like me to submit this fix as a PR?

Convert to python3

• Convert to python3

When using .reconstruct to predict tide it slowly decreases down to very large numbers

When I use .reconstrut to predict tides over the period of my real data, the predicted data on the time series slowly drifts down to very large numbers, does anybody had this issue ? I already verified the code and all the setups, tried numbers of examples but it never works, I am able to get good data to predict the tide for the exact time period of my real data, but when I extrapolate to time periods over its lenght it goes down, and if I go back in time to predict the numbers go way up, very weird.
Thank you

lat=0 issue:

Proving lat=0 leads to a crash with the following example:

    import utide
    import numpy as np
    import matplotlib.dates as mdates
    import datetime, dateutil

    print(utide.__version__)

    time = np.arange(0.,30.*86400,60.)
    t0 = datetime.datetime(2011,9,10)
    dtime = [t0+dateutil.relativedelta.relativedelta(seconds=t) for t in time]
    mtime = mdates.date2num(dtime)

    tdat = utide.ut_constants['const']
    iM2 = tdat['name'].tolist().index('M2')
    omega = tdat['freq'][iM2]
    
    eta = 1.*np.cos(2.*np.pi*omega*time/3600.)

    coef = utide.solve(mtime, eta, lat=0., method='ols', conf_int='MC')

With the following error message:

(equinox) r2i2n0 datahome/aponte% python tests.py
0.2.2
/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/harmonics.py:132: RuntimeWarning: divide by zero encountered in double_scalars
  rr[j] *= 0.36309 * (1.0 - 5.0 * slat**2)/slat
/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/numpy/core/_methods.py:32: RuntimeWarning: invalid value encountered in reduce
  return umr_sum(a, axis, dtype, out, keepdims)
solve: matrix prep ... solution ... Traceback (most recent call last):
  File "tests.py", line 57, in <module>
    test_utide()
  File "tests.py", line 46, in test_utide
    coef = utide.solve(mtime, eta, lat=0., method='ols', conf_int='MC')
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/_solve.py", line 200, in solve
    coef = _solv1(t, u, v, lat, **compat_opts)
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/utide/_solve.py", line 291, in _solv1
    m = np.linalg.lstsq(B, xraw, rcond=None)[0]
  File "/home1/datahome/aponte/.miniconda3/envs/equinox/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 2031, in lstsq
    0, work, lwork, rwork, iwork, 0)
ValueError: On entry to DLASCL parameter number 4 had an illegal value
(equinox) r2i2n0 datahome/aponte%

lmbscgc & lmbscga in progress

Hi! A short message to notify everyone that I am working on the lmbscgc & lmbscga functions (to avoid work overlap).

Possible to get U V amp and phase components from Utide

Hello,

I am in the process of implementing a Tidal analysis program using Utide. However I need U/V component amplitudes and phases from the two dimension analysis rather than the ellipse parameters. Does Utide provide these parameters or will I need to calculate them myself?

I have the following code which I adapted from Utides code/manual but it doesn't seem to work (the phases are all tiny values)

            ap = np.hstack((station["coef"]["rf"].b[:station["coef"]["nNR"]], station["coef"]["rf"].b[2 * station["coef"]["nNR"]: 2 * station["coef"]["nNR"] + station["coef"]["nR"]]))
            i0 = 2 * station["coef"]["nNR"] + station["coef"]["nR"]
            am = np.hstack((station["coef"]["rf"].b[station["coef"]["nNR"]: 2 * station["coef"]["nNR"]],station["coef"]["rf"].b[i0: i0 + station["coef"]["nR"]]))
            Xu = np.real(ap + am)
            Yu = -np.imag(ap - am)
            Xv = np.imag(ap + am)
            Yv = np.real(ap - am)
            Au = np.sqrt(Xu ** 2 + Yu ** 2)
            Av = np.sqrt(Xv ** 2 + Yv ** 2)
            gU = np.arctan2(Yu, Xu)
            gV = np.arctan2(Yv, Xv)

The coef is stored in my own station dictionary hence all the station["coef"] references but I think I am using the correct values?

Any pointers would be appreciated