Validation of HEK Data

[ahmetkucuk]

Validation of HEK Data

[berkay-aydin]

Hey everyone,
The downloaded HEK data is at times pretty low-quality. The HEK data, as we have discussed, contains solar event metadata (which gives you some important characteristics of solar events, as well as their locations). The paper Ahmet talks about is the long-term curation paper from Schuh et al. (https://www.sciencedirect.com/science/article/pii/S2213133715000979).

This is a rather hefty task when performed for all event types. For now, let's keep it utilitarian and do it only for more interesting event types: AR (active region), CH (coronal hole).
Later on we will do: EF (emerging flux), SS (sunspot), SG (sigmoid), FL (flare), and FI (filament).

An event's location (spatial coordinates) is reported in many different forms in a HEK entry. For instance, it might be reported using pixel coordinates (separate columns) or as a POINT string (in HPC coordinates), or its bounding box or chain code could be reported. There can be a combination of all these options too. Here are some HEK entries in their raw XML format: https://www.lmsal.com/hek/her?cosec=1&&cmd=search&type=column&event_type=ar,ce,cc,cd,ch,cj,cr,cw,ef,er,fi,fe,fa,fl,lp,os,sg,sp,ss,to,pg,bo,ee,pb,pt,ot,nr&event_region=all&event_coordsys=helioprojective&x1=-5000&x2=5000&y1=-5000&y2=5000&result_limit=120&event_starttime=1950-01-01T00:00:00&event_endtime=2050-01-01T23:59:59&sparam0=FRM_Name&op0=like&value0=SPoCA&sparam0=ch.area_atdiskcenter&op0=>&value0=608735000

Over there you might see some entry such as:

<param name="hpc_boundcc">
POLYGON((551.1 -323.1,551.7 -322.5,599.7 -274.5,598.5 -222.9,629.7 -213.9,656.1 -229.5,681.3 -216.9,682.5 -270.9,626.7 -334.5,551.1 -323.1))
</param>

Our aim is to fix these locations and improve data quality as much as we can.
Firstly,
For Active Regions:
-Reporting modules should be (FRM) in {"HMI SHARP", "SPoCA", "NOAA SWPC Observer"}
For Coronal Holes:
Reporting module should be (FRM) in { "SPoCA"}
If there are others, do not add them to DB.

Secondly,
For both Active Regions and Coronal Holes:
We will get the hpc_boundcc (if it exists), and we will validate to check if it is a proper polygon or not. There are many reasons why these polygons might not be valid. To check them, we have been using JTS library, https://locationtech.github.io/jts/javadoc/org/locationtech/jts/geom/Geometry.html
(See isValid method. there).