mesarcik/rfi-nln

Code for paper entitled "Learning to detect RFI in radio astronomy without seeing it"

rfi-nln's Issues

LOFAR Labelling

Notes

• baseline 43 contains dropped data.

Finalising results

HERA:

I have fixed the simulator such that the proper models of source, gain, fluctuations, cross talk and rfi are simulated. The generation script will need to be refactored and comitted to the repo. Below an example of the simluator outputs can be shown:

Intermedia performance evaulation:

• As discovered in a meeting this week I found that the previously simulated data was trivial for RFI detection and could be detected using a single threshold with an AUROC of above 0.9.
• With the newly simulated dataset we get an AUROC of 0.78 with a single threshold, this is still well above random, but is an improvement relative to the previous result.
• I wonder if it is meaningful to really try improve this result (i.e. decrease the single threshold performance) or just continue as is
• As it stands the UNET achieves an AUROC of 0.97 and NLN (using a VAE) gets 0.96, however on AUROC and IOU we get lower performance.
• The AOFlagger obtains a 0.87, this is not properly optimised but it is at least now better than hte simple threhsold
• Furthermore this 0.97 result is inline with the previous paper that made use of this data and a UNET architecture.

TODO

• Redo the OOD results
  • Without RFI_DTV, RFI_stations, RFI_Impulse, RFI_Scatter
  • Only RFI_DTV,
• Tune the AOFlagger for the new HERA data
• Redo the poor heuristic results

LOFAR

I am still labelling the LOFAR data, hopefully this can be completed sooner than later.