ph5api returns a trace of 0 samples in certain cases on LH Channels.

[pic-iris]

ph5api returns a trace of 0 samples in certain cases on LH Channels.

[derick-hess]

Steve is investigating this. He said he should have time to work on this on Sunday.

From a little bit of testing I did today it doesn't seem to be an issue with actual wavefields data though I wasn't able to do any in depth digging yet. The same issue occurs on our pier test data and another data set at the PIC that has rt130 dataloggers and broadband seismometers. This only seems to be an issue on the 1 Hz data. A quick look at the pier PH5 archive I think the data is actually there but ph5api.cut is having some issue. The das_t table says there is 1Hz data there starting at the beginning of the day and lasting 86400 samples(so 1 full day of data).

I checked the wavefields data, and the two experiments at the PIC and it's the same for every single day across all those experiments. Only the first hour of each day is accessible anything after that causes an error in ph5api.cut

In a situation where there just wasn't data past the first hour ph5api.cut would normally return that first hour of data and just stop there and keep going grabbing whatever data along the way to the end time requested, but it seems be hitting some kind of error that causes it to immediately return a trace with no data.

[rsdeazevedo]

It turns out that filter_das_t was dropping entries. I pushed a new version of ph5API.py.

[derick-hess]

Thanks.

[nick-falco]

Thanks Steve!

[nick-falco]

I just deployed this change to the beta web service endpoint and unfortunately the problem still exists.

https://service.iris.edu/ph5beta/dataselect/1/query?network=YW&station=1001,2019,518,601&starttime=2016-06-24T00:00:00&endtime=2016-06-24T05:00:00&nodata=404

still returns no LH channels.

[derick-hess]

Confirmed. It' still someting in the filter_das_t or read_das_t. I'll do some more checking tomorrow.

when asking for 1 hour of data at the start of the day you get the das_t entry for the 1Hz data that says it is one day long but that das_t entry is missing if you request anything outside of the first hour of the day for 1Hz data.

[derick-hess]

Oh, looks like maybe thecorrect ph5api didnt get committed. I have the same file already from may and the prog version date is old

[rsdeazevedo]

Derick; Here is the version that i meant to check in. steve.

…

On 06/11/2017 06:11 PM, derick-hess wrote: Oh, looks like maybe thecorrect ph5api didnt get committed. I have the same file already from may and the prog version date is old — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUVZHKXeW6b-PjWqGeYpKlg8vjFoFfXNks5sDIJPgaJpZM4N1__K>.

[nick-falco]

I think there are still some issues. I deployed this to ph5beta and ran the following request:

https://service.iris.edu/ph5beta/dataselect/1/query?network=YW&station=1001,2019,518,601&starttime=2016-06-24T00:00:00&endtime=2016-06-24T05:00:00&nodata=404

The LH channels were included in the output, but 73 traces were returned in what appears to be duplicate data.

[derick-hess]

hmm running from the command line i just get one .ms file for each channel containing 1 trace and they all look good.

ph5tomsAPI.py -n master -p /mnt/Wavefields/PROCESS/Sigma/16-015/ -s 2016-06-24T00:00:00.00 -t 2016-06-24T05:00:00.00 --station=1001,2019,518,601

I'll check on the pic-ph5 server as well

[derick-hess]

never mind. It is something to do with the the time handling now.

ph5tomsAPI.py -n master -p /mnt/Wavefields/PROCESS/Sigma/16-015/ -s 2016-06-24T00:00:00.00 -t 2016-06-24T05:00:00.00 --station=1001,2019,518,601

does something odd

ph5tomsAPI.py -n master -p /mnt/Wavefields/PROCESS/Sigma/16-015/ -s 2016-06-24T00:00:00.00 -l 300 --station=1001,2019,518,601

produces what it should

[nick-falco]

Yeah I think it is still a bug in one of the APIs. The web service is flattening the request correctly. It calls ph5tomsAPI.py in the following order:

YW [u'1001'] [u''] [u'DP1'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'1001'] [u''] [u'DP2'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'1001'] [u''] [u'DPZ'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'2019'] [u''] [u'DP1'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'2019'] [u''] [u'DP2'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'2019'] [u''] [u'DPZ'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'HH1'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'HH2'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'HHZ'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'LH1'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'LH2'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'518'] [u'01'] [u'LHZ'] 2016-06-24 00:00:00 2016-06-24 05:00:00
YW [u'601'] [u''] [u'HDF'] 2016-06-24 00:00:00 2016-06-24 05:00:00

[derick-hess]

I'll do some debugging to see what ph5API is returning at various stages. Oh and ignore the length requesting being correct. It is not. I was requesting 5 minutes instead of 5 hours. when 5 hours is requested the same thing happens as settign the end time.

Since this issue didn't occur before teh changes to filter_das_t I ;ll start looking there

[derick-hess]

So it seems this is an issue not with the code. It is only an issue with certain nodes. 1001 is an issue but 2019 is not. When I looked at the raw data 2019 contained only 1 raw data file while 1001 has 3.

the das_t reports all 3 of the 1001 files loaded and they have multiple traces over the same time and that is what is being loaded.

Looking at the raw file list this only happened on stations 1001, 4207, 4309

[nick-falco]

This was fixed so closing.