gigjozsa/casafails

casafails's Issues

mstransform results in stripes in images

The use of mstransform to average data in frequency and time, or when using it for a continuum subtraction has been reported to result in stripes when imaging. An example can be seen here:

One of the observations is the following: in the data reduction process of this example time averaging was used when splitting. In #644 time averaging with mstransform has been used to generate the data set from another one before processing it with meerkathi. In a second attempt, time averaging not done with mstransform but when downloading the data with mvftoms was applied and no stripes were seen any more.

Facit so far: applying mstransform with time averaging and continuum subtraction with mstransform further down the line is probably unhealthy.

Delay clipping using flagdata task flags negative delays

Regardless of user options to include negative delays and flag outside a [-x,+y] range the "clip" mode always clips on absolute values and therefore clips about 50% of valid delays in the FPARAM column. I know this affects CASA 5.x, it may also affect 4.x.

Polcal 'X' mode results in "invalid operation" errors in >5.x

The "X" (ie. crosshand phase time variable, frequency invariant) solve mode results in invalid operation errors being generated. I don't have the exact error anymore, but I'm documenting this issue. This was seen when switching my fleeting pol pipeline over to 5.2. The solve strategy is as follows

• flag
• KGB(unpolarized calibrator), Gphase(polarized calibrator)
• flag
• KGB(unpolarized calibrator), Gphase(polarized calibrator)
• oldsplit, preaverage calibrated data for polarization calibration
• crosshand phase slope solver with gaincal ("KX" Jones, time variable per scan)
• crosshand phase time solution per scan ("X" polmode, polcal task)
• crosshand phase frequency dependent solutions (time invariant "Xf" polmode, polcal task)
• D and Df solutions along similar lines

This breaks at the 3rd last step when switching to 5.x. The obvious work around is to change it to "Xf" but this makes the solutions much more noisy.

Warning on CASA calibraton pipeline site: needs to be tracked

What is says: the CASA pages report:
Warning to all users (Feb. 11, 2020): A pipeline bug in CASA 5.6.2-2 was discovered that impacts the resoration of calibrations. This affects all projects calibrated after October 21, 2019. The output from a full calibration run of the pipeline is fine, but if an existing calibration was restored using the NRAO archive-new interface or following the instructions on this page you will obtain incorrect results. The restoration of data prior to October 21, 2019 is not affected. A fix is being formulated for the pipeline and projects already calibrated, we will update affected users when the fix is available.

Delay calibration with CASA 5.x results in high percentages of delays flagged

Running a KGB calibration with exactly the same settings as CASA 4.7 and exactly the same initial data flag results in far greater number of solutions flagged. My original email:

My calibration recipe is as follows, and is aimed to removing the few degree pivoting in the full 856 MHz meerkat bandpass phase that is due to (systemic) rate change, prior to obtaining a bandpass solution:

1. model predict using Meqtrees for primary calibrator,
2. delay cal (task gaincal K, default linear interpolation with primary and secondary),
3. gaincal (task gaincal G, default linear interpolation on primary),
4. bandpass (bandpass, default linear interpolation with fill gaps on to fill all gaps in solution, 1 solution over all scans on primary).
5. gaincal (task gaincal G, default linear interpolation on primary and secondary), and fluxscale
6. applycal to all fields

With CASA 5.4 quite a few delay solutions fail in the middle of the observation and no interpolation / extrapolation is done with the remaining delay solutions:

2019-08-08 06:34:13     INFO    applycal::::       B Jones: In: 3113386316 (70.1617265779%) --> Out: 3199525672 (72.1029202911%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.B0)
2019-08-08 06:34:13     INFO    applycal::::       K Jones: In: 3199525672 (72.1029202911%) --> Out: 3821347670 (86.1159917752%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.K0)
2019-08-08 06:34:13     INFO    applycal::::       G Jones: In: 3821347670 (86.1159917752%) --> Out: 3821793156 (86.1260310263%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.G0)

Going from the flagset prior to applycal with CASA 4.7 yields the expected behaviour. A verbatim recipe is applied. The solutions are correctly solved and interpolated:

2019-08-08 09:41:52     INFO    calibrater::correct     Finished correcting.
2019-08-08 09:41:52     INFO    applycal::::    Calibration apply flagging statistics (among calibrateable spws):
2019-08-08 09:41:52     INFO    applycal::::      Total selected visibilities (among calibrateable spws) = 4437442560
2019-08-08 09:41:52     INFO    applycal::::      Flags:
2019-08-08 09:41:52     INFO    applycal::::       K Jones: In: 3113386316 (70.1617265779%) --> Out: 3119947068 (70.3095764241%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.K0)
2019-08-08 09:41:52     INFO    applycal::::       B Jones: In: 3119947068 (70.3095764241%) --> Out: 3119947068 (70.3095764241%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.B0)
2019-08-08 09:41:52     INFO    applycal::::       G Jones: In: 3119947068 (70.3095764241%) --> Out: 3119947068 (70.3095764241%) (/home/bhugo/output/1557766852_sdp_l0_1284.hh_vv.G0)
2019-08-08 09:42:05     INFO    applycal::::    ##### End Task: applycal             #####

Inspecting the CASA 4.7 generated delays shows that all the feeds are in the few nanosecond range as expected (after successful phaseup process from SP):