teresaupdyke/qccodar

This Python code applies several quality control (QC) functions based on CODAR SeaSonde (COS) Radial Metric data.

qccodar's Issues

Fix auto() to only reprocess new files

Current function in qccodar.app auto() is not working -- reprocess all the files found in /Codar/Data/RadialMetric/{patternType} regardless of previous run an hour ago

1. Method should be to check difference in file listing between /Codar/Data/RadialMetric/{patternType} with /Codar/Data/RadialShorts_qcd/{patternType} and see what's new in RadialMetric and process new.
2. create second function for auto that implements watchdog -- refer to v1.0.dev1 where it was tried before: https://github.com/nccoos/qccodar_py2.7/releases/tag/1.0.dev1

add_short_metadata not saved when using write_output()

The additional metadata added for QCD (r.metadata['QCD']) in the Radial object is generated by add_short_metadata, but not being saved with write_output(). Suspect a bug in HFRadarPy not recognizing it. I'm using the version of HFRadarPy from pip install (PyPi) not his latest github version.

Update setup.py

Needs hfradarpy for sure
version number ??
qccodar only -- rename from qccodar3

check/test on test.pypi.org
verified my account still active on test.pypi.org
verified my account still active on official public-facing pypi.org

Update qcviz and histogram plot tools

Update qcviz and histogram plot tools
Add a tools subfolder.
Test/ document how to run from pip install to envs

Add additional options for LLUVMerger

LLUVMerger is now at v2.0.4 with additional options to consider.
Add additional items to qccodar configuration file and modify run_LLUVMerger() to use these or use default values if not in config

Add source grid option to LLUVMerger and ... might need to test this option first to see if it really puts merged vectors where source locations are. Then don't know how it affects the use of -angres -angalign -angmethod

-reference= -- Optional lluv or grid file to use for output
origin and reference location vectors to merge. If
not specified first source is output origin and
sources are used for merge locations.

Might be worth adding these options that can be configurable

-method= -- Merge method, average/median/minimum/maximum/
overlay/underlay/smallest/largest. (Default Average)

-angres= -- Merge all points within this angular resolution
from origin. But what if there was a -reference grid?? Does this apply?

-angalign= -- -- Alignment for angular resolution. Radial/
Elliptical alignment typically is antenna bearing
modulo of angular resolution.

--angmethod= -- Method when using angres: all/short. all=merges
all velocities. short=first average merge source to
angres then merge result. Default is short

Fix difference in diagnostic tables

For the RadialShorts there is a template in file_formats that has diagnostic info that may not be in RadialMetric. For example, offline processing, outputs rads and RINT diagnostics but not rcvr in the RadialMetric ouput.

So when code generates new RadialShort based on template with rcvr table it bombs when there is none.

just needs some code to handle when a diagnostic table is not there in generate_shortradial() and add_diagnostic_tables later when merging.

Generalize environment.yml

Make and test environment.yml more general for conda create env -f environment.yml
Could also add qccodar pip installed with creating env once updated on pypi

Update tests to use hfradarpy tools

1. Make /update tests compliant with hfradarpy tools.
2. Add other tests

• config file load test
• verify configs (all there)?
• do_LLUVMerger tests
• (others ?? will update here as get into it)

Add tools folder

Add a tools folder for useful tools like

run_qccodar.py
run_do_qc.py
run_do_merge.py

run_plot_qcviz.py
run_plot_histviz.py

make_plist_config.py