amunozj/magnetograph_2hmi_converter

Converter software to calibrate and super-resolve solar magnetograms

Converter software to calibrate and super-resolve solar magnetograms

Currently only working on magnetograms taken by the Michelson Doppler Imager (MDI) on board the Solar and Heliospheric Observatory (SoHO) or the Global Oscillation Network Group (GONG). Output magnetograms have the resolution and systematics of magnetograms taken by the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO).

Installation

1. Click on the Clone or download button and clone it to a repository or download it as a zip file.

2. Install dependencies using conda:

conda env create -f environment.yml

Or Pip

pip install -r requirements.txt --user

New Fits Keywords

The converter magnetogram contains the following added keywords:

'instrume': Old value is retained, but with the appended '-2HMI_HR' suffix.

'date-conv': UTC date of conversion.

'nn-model': Name of neural network architecture.

'loss': Loss parameters.

'converter_doi': DOI pointing to this converter's repository

Usage Arguments

Required

'--instrument': Input instrument. Currently only 'gong' and 'mdi' are valid choices.

'--data_path': Folder containing the files to be converted.

'--destination': Destination of the super-resolved magnetograms.

Optional

'--add_noise': Variance of the gaussian noise (in gauss) to be added to the output magnetogram. By default no noise is added. For noise similar to HMI use 4.7.

'--plot': Make a plot of the output magnetogram.

'--overwrite': Flag to overwrite files in output folder. Files are not overwritten by default.

'--use_patches': Run inference on magnetogram patches instead of the default full disk inference.

'--zero_outside': Pad outside the solar disk using zeros. Default np.nan.

Example:

python convert2HMI.py --instrument mdi --data_path /tmp/mdi/input --destination /tmp/mdi/output --use_patches --overwrite --plot

magnetograph_2hmi_converter's Issues

`convert2hmi.py` will accept hmi data

convert2hmi.py will accept hmi data. Expected behaviour: the data should be recognised as HMI and skipped.

python convert2HMI.py --instrument mdi --data_path ../solarmag/ --destination ../solarmag/ --use_patches

2021-04-28 - 23:21:15 INFO __main__: Processing ../solarmag/mdi.fd_m_96m_lev182.20110331_013600_TAI.data.fits
2021-04-28 - 23:21:15 INFO __main__: Attempting inference on patches...
2021-04-28 - 23:21:26 INFO __main__: Success.
2021-04-28 - 23:21:26 INFO __main__: Processing ../solarmag/hmi.m_720s.20110331_013600_TAI.1.magnetogram.fits
2021-04-28 - 23:21:37 INFO __main__: Attempting inference on patches...
2021-04-28 - 23:24:25 INFO __main__: Success.
Traceback (most recent call last):
  File "convert2HMI.py", line 182, in <module>
    inferred_map = file_dset.create_new_map(inferred, scale_factor, args.add_noise, model_name, config_data, padding)
  File "/content/magnetograph_2HMI_converter/source/dataset.py", line 58, in create_new_map
    new_meta['r_sun'] = new_meta['r_sun'] * scale_factor
  File "/usr/local/lib/python3.7/dist-packages/sunpy/util/metadata.py", line 48, in __getitem__
    return OrderedDict.__getitem__(self, key.lower())
KeyError: 'r_sun'

Output a plot of the input image.

While --plot will return a .png of the output magnetogram, it would be useful for the user to also receive a similar file corresponding to their processed input image.

dependency issues

With Python 3.7,

ERROR: Cannot install -r requirements.txt (line 18) and tensorboard==1.14.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested tensorboard==1.14.0
    tensorflow 1.15.4 depends on tensorboard<1.16.0 and >=1.15.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

to replicate:

conda create -n hmi-mag python==3.7
pip install -r requirements.txt --no-cache-dir

With Python 3.9:

    ERROR: Command errored out with exit status 1:
     command: /home/paul/anaconda3/envs/hmi-mag-3.9/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-moej_o98/scikit-image_e611f07ec3a447228c0e8778b80f12fd/setup.py'"'"'; __file__='"'"'/tmp/pip-install-moej_o98/scikit-image_e611f07ec3a447228c0e8778b80f12fd/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-o1fn1vyv
         cwd: /tmp/pip-install-moej_o98/scikit-image_e611f07ec3a447228c0e8778b80f12fd/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-moej_o98/scikit-image_e611f07ec3a447228c0e8778b80f12fd/setup.py", line 31, in <module>
        from numpy.distutils.command.build_ext import build_ext
    ModuleNotFoundError: No module named 'numpy'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/e0/46/ca035f5d7d3414124a3a5ef22cd2e75c0c5149042a668375f1d44eb69f8f/scikit-image-0.15.0.tar.gz#sha256=df111e654b47e5ea456c50553debe4c5ddd97258894c7ad3b7f2f9f10798e053 (from https://pypi.org/simple/scikit-image/) (requires-python:>=3.5). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement scikit-image==0.15.0
ERROR: No matching distribution found for scikit-image==0.15.0

to replicate:

conda create -n hmi-mag-3.9 python==3.9
conda activate hmi-mag-3.9
pip install -r requirements.txt --no-cache-dir

Potential edits to README.md

I have a few ideas with regards to the README:

1. Create a "Getting Help" section to ask people to create issues
2. Add a Code of Conduct
3. Detail how to cite this work
4. Include examples of this code working, e.g.

• An example of input MDI/GONG and the output from the model.
• A link to a notebook of an example observation obtained from JSOC and through the model.

5. Authors
6. License
7. Acknowledgements