wsular/easyflux-dl-cr3000

CR3000 datalogger program for Campbell open-path eddy-covariance systems, by @campbell-scientific

easyflux-dl-cr3000's Issues

Data table timing

I recall there being consensus for:

• 10hz retention of flux sensor data
• 30-min output for calculated fluxes and averages of meteorological & soil data
• 15-min output for averages of select meteorology to report to LTAR common observatory
• 15-min output for soil data (averages?)
• 1-min output for select meteorology (averages? or maybe grab sample depending on sensor acquisition frequency?)

Current programs have:

• 10hz retention of flux sensor data ✔️
• 30-min output for calculated fluxes and averages of meteorological & soil data ✔️
• 15-min output for averages of select meteorology to report to LTAR common observatory ✔️
• 15-min output for averages of
  • vertical soil T/VWC profile
  • soil heat flux plate(s)
  • soil T/VWC probe for energy closure
  • soil averaging thermocouple
• 1-min output for averages of select meteorology to report to LTAR common observatory
• 1-min output for averages of
  • soil heat flux plate(s)
  • soil T/VWC probe for energy closure
  • soil averaging thermocouple

Comments in source code indicate the 1-min outputs were originally on a 5-min basis.

Cup & vane azimuth

Unique azimuth for cup-and-vane anemometer is not exposed in a user-editable way. It's defined as constant (default: 0) so would be easy to expose through a ConstTable.

Topp equation

Get rid of compiler warnings by wrapping with conditional compilation for 5TM sensors

Slow sequence flags

The program uses flags to handle data sources in the same output table being acquired on two different time bases (i.e. the slowsequence_disable_flg). This is a necessary approach related to correctly triggering data tables with simultaneous scans happening.

A new slow sequence was added in 7c72723 for acquiring data from soil probes, which includes new flag slowsequence_5_disable_flg. This flag is

• correctly flipped at the end of the new slow sequence
• correctly reset at the end of the primary scan,
• correctly updated for TDR-related columns in final data tables
• not correctly updated for 5TM-related columns in final data tables

No impact on historical data since it only affects Decagon (METER) 5TM probes. Need to update 5TM processing to use correct flag.

5TM soil probes abandoned in standard slowsequence

Parts of the code for SDI-12 soil sensors were moved into a new 1-min slowsequence:

• TDR-series soil probes reporting T/VWC for heat storage calculations
• TDR-series soil probes deployed as a vertical profile
• 5TM-series soil probes deployed as a vertical profile

But other parts were left behind in the 5-sec slowsequence:

• 5TM-series soil probes reporting T/VWC for heat storage calculations

Maybe second resolution for #8 is preferable: return to just one slowsequence and raise interval to accomodate sensor delays?

Running averages of SDI-12 soil probe data uses wrong time window size

SDI-12 soil sensors (TDR315, 5TM) were moved from the standard 5-second slowsequence into a separate 1-min slowsequence to avoid data acquisition stalling due to sensor delays. By default the running averages are supposed to represent the previous 1-minute of data. However, the parameter dictating the number of samples to include in said averages (NMBR_SOIL_T_WTR_DEL_SAMPLES) was not updated and as a result, the averaging windows for those sensors was inadvertently expanded to previous 12 minutes.

Const SLOWSEQUENCE_SCAN_INTERVAL = 5000          'Unique: slow sequence measurement rate (ms) (For this CR3K program, the shortest scan interval is 2000 ms)
[...]
Const NMBR_SOIL_T_WTR_DEL_SAMPLES = (60*1000)/SLOWSEQUENCE_SCAN_INTERVAL       'Unique: Number of measurements to compute a one-minute mean of soil temperature and water

Standard slow sequence:

  SlowSequence
  Scan (SLOWSEQUENCE_SCAN_INTERVAL, mSec, 3, 0)
    [...]
    #If ((SENSOR_HFP) AND  (SENSOR_CS616) AND (SENSOR_TCAV)) Then
    AvgRun (soil_wtr_current(1), NMBR_CS6xx, soil_wtr_T(1), NMBR_SOIL_T_WTR_DEL_SAMPLES)

New sequence:

  SlowSequence
  Scan (1, Min, 0, 0)   
    [...]
    #If ((SENSOR_HFP) AND (SENSOR_CS6XX) AND (SENSOR_SOIL_T))) Then

    AvgRun (soil_wtr_current(1), NMBR_CS6xx, (tdr31X_wc(1)/100), NMBR_SOIL_T_WTR_DEL_SAMPLES) 'HINT scale % -> v/v

The trivial fix (scaling parameter to match new slowsequence time base) would mean just one data point is included in those averages. Maybe it should be a 15-, 20-, or 30-second scan instead?

Alternately: revert the addition of a new slowsequence and specify up-front that if SDI-12 soil sensors are used, the slowsequence interval must be increased from 5000msec to [whatever the actual minimum should be, probably determined through some bench testing].

All SDI-12 sensors use "M!"

Standard measurement command M! causes datalogger to wait for a response. For 1-2 sensors it's negligible but using 8 can result in skipped slow scans.

Might need to use concurrent measurements instead..

Sensor	SDI-12 command time	Time for heat storage msmts	Time for profiling msmts
CS650	600 ms	up to 1.2 sec	n/a
CS655	600 ms	up to 1.2 sec	n/a
5TM	1 sec	up to 2 sec	up to 6 sec
TDR series	1 sec	up to 2 sec	up to 6 sec

---

5TM sensors require 1 second for measurement:

Select SDI12 Port: 3
Entering SDI12 Terminal
?!
A
AM!
A0012
A
AD0!
A+1.00+20.6

Likewise, TDR series probes also require 1 second for measurement (see Table Command Reference in the user manual. Also worth noting:

If a cable is used to connect multiple sensors to the data recorder and if concurrent readings are to be taken the voltage drop in long cables may become an issue. For example: 5 sensors operating simultaneously could draw up to 400 mA of current and cause a voltage drop of 3.25 volts in 250 feet of 22-gauge wire. If the voltage at the sensor drops to the lower operating limit, the sensor may misread or fail to report data. To avoid this do not use concurrent commands or ensure that long cables are of sufficient wire gauge to handle the current loads without significant voltage drops. The SDI-12 command “aV!” can be used to measure the sensor supply voltage at the sensor and help diagnose power issues. However, the “aV!” command is not concurrent, and so it cannot measure the loading effects of concurrent measurements.)

RH averaging

RH values are naively averaged (indeterminate) in following data tables:

• LTAR_Met_1Minute
• LTAR_Met

  #If (SENSOR_IRGASON) Then                       'RelativeHumidity (percent)
  Average (1, RH,         IEEE4, (irga_disable_f OR sonic_disable_f OR H2O_bad_rng_sig_array(MAX_LAG +1)))
  #EndIf
  #If (NOT SENSOR_IRGASON) Then
  Average (1, RH,         IEEE4, (irga_disable_f OR irga_amb_tmpr_f OR H2O_bad_rng_sig_array(MAX_LAG +1)))
  #EndIf

Should have been implemented as two-stage process (same as that used for Flux table):

1. Calculate average water vapor pressure (e) and average saturation vapor pressure (e_sat) for desired time period
2. Use intermediate results to calculate RH as ratio e/e_sat
3. Sample calculated RH as the time period average

Both affected tables do contain the intermediate results e and e_sat so recalculating correct RH values for historical records is trivial. Future program versions need to incorporate above process.

Edit: introduced in 8a73e69

Tracking unique sensor calibration values

Base version of EasyFlux-DL relies on unique sensor calibration values being updated in the program file. Any changes to those unique values would be (hopefully) tracked by the user before they deploy their changes.

Our version is modified so users can update unique values without editing (and subsequently redeploying) the program file. Since changes to the unique values file are not tracked in version control, we conditionally add relevant columns to the final data tables which contain the sensitivity values used for that record:

Table name	Relevant columns
Flux	NRLITE_SENS, NR01_SW_IN_SENS, NR01_SW_OUT_SENS, NR01_LW_IN_SENS, NR01_LW_OUT_SENS, CNR1_SW_IN_OUT, CNR1_SW_OUT_SENS, CNR1_LW_IN_SENS, CNR1_LW_OUT_SENS, PYRAN_MULT, PYRAN_OFFSET, QUANTUM_SENS, QUANTUM_OFFSET, SI111_m0, SI111_m1, SI111_m2, SI111_b0, SI111_b1, SI111_b2, SHFP_1_SENS, SHFP_2_SENS, SHFP_3_SENS, SHFP_4_SENS
LTAR_Met	SHFP_1_SENS, SHFP_2_SENS, SHFP_3_SENS, SHFP_4_SENS
LTAR_Met_1min	SHFP_1_SENS, SHFP_2_SENS, SHFP_3_SENS, SHFP_4_SENS

These unique sensor calibration values change very infrequently and consequently we store large amounts of identical values.

• Should these kind of metadata be abstracted into a separate data table which gets called just once after compilation?
• If so, should the structure of that table be rigidly defined such that adding/removing sensors does not cause that table to reset?
• Are there other relevant metadata to retain that aren't specifically calibration values?
  • Sensor presence & quantities Can be deduced by columns present in output data files
  • Fine wire thermocouple diameter? No, already in Flux_Notes
  • Sonic azimuth Present in Flux
  • Rain gage multiplier? (presuming rain gage sensor choice is refactored back to on/off constant + unique constant)
  • Cup & vane azimuth?

High-freq CO2 retained if EC150 but not if IRGASON

Need to refactor IRGASON code to also record high-frequency CO2 density in timeseries output.