Giter Site home page Giter Site logo

smops's Introduction

smops

smops (aka Smooth Operator) is a python package for interpolating channelised FITS model images onto user-specified higher resolution frequency grid. For example, you can give smops a set of 4 per-subband model FITS images, and ask it to return 16 model images, which can then be fed back into e.g. wsclean for a predict operation. Usage:

smops --ms /ms/used/togen/images.ms -ip prefix-used-for-those-images -co 16 -order 4

Its options are:

usage: smops [-h] [-v] [-op] [-j] [-s] [-mem] -ms  -ip  -co  [-order]

Refine model images in frequency

optional arguments:
-h, --help            show this help message and exit
-v, --version         show program's version number and exit
-op , --output-prefix
                        What to prefix the new interpolated model name with.
-j , --num-threads    Number of threads to use while writing out output images
-s , --stokes         Which stokes model to extrapolate. Write as single string e.g IQUV. Required when there are multiple Stokes
                        images in a directory. Default 'I'.
-mem , --max-mem      Approximate memory cap in GB

Required arguments:
-ms , --ms            Input MS. Used for getting reference frequency
-ip , --input-prefix
                        The input image prefix. The same as the one used for wsclean
-co , --channels-out
                        Number of channels to generate out
-order , --polynomial-order
                        Order of the spectral polynomial
  • Free software: MIT license

Credits

This package is a brain child of @o-smirnov x @landmanbester and is under @ratt-ru.

It was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

smops's People

Contributors

mulan-94 avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

smops's Issues

Background value of output cube is 1.

I ran smops but the interpolated model cube that it produced has a background of 1 instead of 0. I had a look at the code but couldn't see where this originated from.

stimela dependency error

Trying to pip install smops and this is happening:

$ pip install smops/
Processing ./smops
Requirement already satisfied: astropy in /mnt/home/ianh/venv/py3/lib/python3.6/site-packages (from smops==0.1.7) (4.0.1.post1)
Requirement already satisfied: dask in /mnt/home/ianh/venv/py3/lib/python3.6/site-packages (from smops==0.1.7) (2021.3.0)
Requirement already satisfied: python-casacore in /mnt/home/ianh/venv/py3/lib/python3.6/site-packages (from smops==0.1.7) (3.4.0)
Requirement already satisfied: psutil in /mnt/home/ianh/venv/py3/lib/python3.6/site-packages (from smops==0.1.7) (5.8.0)
Requirement already satisfied: numpy in /mnt/home/ianh/venv/py3/lib/python3.6/site-packages (from smops==0.1.7) (1.19.1)
ERROR: Could not find a version that satisfies the requirement stimela==2.0rc4 (from smops==0.1.7) (from versions: 0.2.1, 0.2.3, 0.2.5, 0.2.6, 0.2.7, 0.2.8, 0.2.9, 0.3.0, 0.3.1, 0.3.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.5, 1.4.6, 1.5.1, 1.5.2, 1.5.3.1, 1.5.4, 1.6.0, 1.6.1, 1.6.2, 1.6.4, 1.6.5, 1.6.7, 1.6.8, 1.6.9, 1.7.0, 1.7.1, 1.7.3, 1.7.4, 1.7.5, 1.7.6, 1.7.7)
ERROR: No matching distribution found for stimela==2.0rc4 (from smops==0.1.7)

Thanks.

Issues with NaN in models

Sometimes the wsclean model will return frequency planes that are entirely NaN, which I believe are due to frequency chunks that are entirely flagged, for example here is a list of model images with their corresponding numpy.max value:

img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0000-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0001-model.fits 0.07603928
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0002-model.fits 0.071212046
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0003-model.fits 0.06665679
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0004-model.fits 0.062363617
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0005-model.fits 0.058322698
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0006-model.fits 0.05452421
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0007-model.fits 0.050958294
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0008-model.fits 0.047615148
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0009-model.fits 0.044484913
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0010-model.fits 0.04155774
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0011-model.fits 0.038823854
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0012-model.fits 0.03627334
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0013-model.fits 0.03389641
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0014-model.fits 0.031683195
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0015-model.fits 0.0296239
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0016-model.fits 0.027708663
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0017-model.fits 0.026303917
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0018-model.fits 0.025389565
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0019-model.fits 0.024521032
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0020-model.fits 0.023689415
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0021-model.fits 0.022885771
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0022-model.fits 0.022101225
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0023-model.fits 0.021326844
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0024-model.fits 0.020553716
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0025-model.fits 0.019772952
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0026-model.fits 0.019174945
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0027-model.fits 0.018606504
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0028-model.fits 0.01800268
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0029-model.fits 0.017358057
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0030-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel-0031-model.fits nan

This is problematic because wsclean's own predict function will halt when it encounters an entirely empty frequency plane, even though it is quite happy to do such predictions internally during major cycles (see André's explanation here: https://gitlab.com/aroffringa/wsclean/-/issues/127).

My (not ideal) plan to fix this was to examine all the model frequency images prior to prediction, and if any of them were entirely blank then just zero them, or insert a pixel of inconsequential positive value in the corner. Then I thought it would probably be better if smops could interpolate over any gaps, however it seems that any NaN values cause smops to fail entirely. Here are the max values from the cube above after I tried to interpolate to 128 frequency planes:

img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0000-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0001-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0002-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0003-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0004-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0005-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0006-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0007-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0008-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0009-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0010-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0011-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0012-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0013-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0014-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0015-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0016-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0017-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0018-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0019-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0020-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0021-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0022-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0023-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0024-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0025-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0026-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0027-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0028-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0029-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0030-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0031-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0032-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0033-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0034-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0035-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0036-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0037-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0038-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0039-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0040-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0041-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0042-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0043-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0044-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0045-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0046-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0047-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0048-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0049-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0050-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0051-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0052-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0053-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0054-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0055-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0056-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0057-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0058-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0059-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0060-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0061-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0062-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0063-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0064-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0065-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0066-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0067-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0068-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0069-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0070-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0071-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0072-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0073-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0074-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0075-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0076-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0077-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0078-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0079-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0080-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0081-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0082-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0083-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0084-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0085-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0086-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0087-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0088-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0089-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0090-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0091-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0092-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0093-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0094-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0095-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0096-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0097-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0098-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0099-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0100-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0101-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0102-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0103-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0104-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0105-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0106-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0107-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0108-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0109-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0110-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0111-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0112-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0113-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0114-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0115-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0116-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0117-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0118-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0119-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0120-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0121-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0122-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0123-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0124-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0125-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0126-model.fits nan
img_1622134148_sdp_l0_1024ch_NGC3263.ms_prepeel_smops-0127-model.fits nan

This would probably go away if I had inserted zeros into the otherwise blanked channels, however that would then also insert a 0 into the spectrum of any model component. I think a better (and more useful) fix would be to add proper NaN support to smops.

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.