darafferty/lsmtool

LOFAR Local Sky Model Tool

lsmtool's Issues

warning in python 3.8

In python 3.8 I started getting a few warnings for "is" to compare strings, e.g.

LSMTool/lsmtool/operations/compare.py

Line 309 in 3fddc7c

if matplotlib.get_backend() is not 'Agg':

patch names inconsistency

lsm.setPatchPositions(method='wmean') # calculate patch weighted centre for tassellation
patches = lsm.getPatchNames()

lsm.group('facet', facet=mask, root='Dir')
lsm.setPatchPositions(method='mid') # recalculate the patch centre as mid point for imaging
directions = lsm.getPatchPositions()

in these two cases the names in the "patches" variable and the keys in "directions" have different names when more than 10 patches are created. The first has e.g. Dir_1 and the second Dir_01. The best would be to have consistency with naming patches across the code (maybe always without leading 0s).

ImportError: No module named '_logging'

I get this error trying to use LSMTool in Python 3.
Already working on a patch.

LSMTool now part of KERN

more info about KERN:
http://kernsuite.info

To quickly install LSMTool on Ubuntu 16.04 (LTS):

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository -s ppa:kernsuite/kern-2
$ sudo apt-add-repository multiverse
$ sudo apt-get update
$ sudo apt-get install lsmtool

metadata files can be found here:
https://github.com/kernsuite-debian/lsmtool

Version number

Hi,

A small issue: version number is not updated:
https://github.com/darafferty/LSMTool/blob/master/lsmtool/_version.py#L6

Best regards,
Alex

Wrong spectral index when concatenating two skymodel

I tried to concatenate two sky model with the following code:

import lsmtool
s1 = lsmtool.load('catalog.skymodel')
s2 = lsmtool.load('Ateam_lowres.skymodel')
lsmtool.operations.concatenate.concatenate(s1, s2)

this used to work fine with earlier version of lsmtool but with version 1.4.9, I get wrong spectral index and ref frequency in the resulting skymodel.

For example, previously (output_1-4-7.skymodel):

1858.9+2503, POINT, Main, 18:58:54.2496, 25.03.00.684, 1.2851, 0.0, 0.0, 0.0, 60000000.0, [-0.4473, 0.0], 0.0, 0.0, 0.0

now (output_1-4-9.skymodel):

1858.9+2503, POINT, Main, 18:58:54.2496, 25.03.00.684, 1.2851, 0.0, 0.0, 0.0, 999999.0, [-0.4473, 1e+20], 0.0, 0.0, 0.0

catalog.skymodel.txt
Ateam_lowres.skymodel.txt
output_1-4-7.skymodel.txt
output_1-4-9.skymodel.txt

No license

Hi! I'm makaing a LSMTool package for KERN:

http://kernsuite.info/

It seems like your software is missing a license. Could you be so kind to add one? thanks!

Avoid matplotlib resource leaking

In lsmtool/operation/compare.py several plots are being made, but the resources used to do so are never properly released. As a result, when invoking compare() many times from a single script, you inevitably trigger the following warning:

RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).

This can easily be fixed.

Allow astropy>=4

Currently lsmtool is forcing downgrade astropy to 3.2 version. Would it be possible to relax this restriction so that we can install on system with astropy >=4 already install ? What is the limitations here ?

wcsaxes is deprecated

wcsaxes is deprecated since a while and going to be replaced by astropy. It is not part of Debian and ubuntu anymore.

Python3 support broken

  python3 setup.py test
running test
================================================ test session starts =================================================
platform linux -- Python 3.5.2 -- py-1.4.20 -- pytest-2.5.2
collected 0 items / 1 errors

======================================================= ERRORS =======================================================
_______________________________________ ERROR collecting tests/test_lsmtool.py _______________________________________
_pytest.runner:139: in __init__
>   ???
_pytest.main:419: in _memocollect
>   ???
_pytest.main:296: in _memoizedcall
>   ???
_pytest.main:419: in <lambda>
>   ???
_pytest.python:445: in collect
>   ???
_pytest.python:1661: in parsefactories
>   ???
_pytest.python:247: in fget
>   ???
_pytest.python:442: in _getobj
>   ???
_pytest.main:296: in _memoizedcall
>   ???
_pytest.python:451: in _importtestmodule
>   ???
py._path.local:620: in pyimport
>   ???
<frozen importlib._bootstrap>:969: in _find_and_load
>   ???
<frozen importlib._bootstrap>:954: in _find_and_load_unlocked
>   ???
<frozen importlib._bootstrap>:892: in _find_spec
>   ???
<frozen importlib._bootstrap>:873: in _find_spec_legacy
>   ???
_pytest.assertion.rewrite:137: in find_module
>   ???
_pytest.assertion.rewrite:269: in _rewrite_test
>   ???
_pytest.assertion.rewrite:322: in rewrite_asserts
>   ???
_pytest.assertion.rewrite:436: in run
>   ???
/usr/lib/python3.5/ast.py:245: in visit
>       return visitor(node)
_pytest.assertion.rewrite:514: in visit_Assert
>   ???
/usr/lib/python3.5/ast.py:245: in visit
>       return visitor(node)
_pytest.assertion.rewrite:601: in visit_Call
>   ???
/usr/lib/python3.5/ast.py:245: in visit
>       return visitor(node)
_pytest.assertion.rewrite:631: in visit_Attribute
>   ???
/usr/lib/python3.5/ast.py:245: in visit
>       return visitor(node)
_pytest.assertion.rewrite:631: in visit_Attribute
>   ???
/usr/lib/python3.5/ast.py:245: in visit
>       return visitor(node)
_pytest.assertion.rewrite:544: in visit_Name
>   ???
E   TypeError: Call constructor takes either 0 or 3 positional arguments
============================================== 1 error in 0.15 seconds ===============================================

Problem reading skymodel with no Patches

I have an issue loading a simple patchless gsm skymodel in the latest release version:

import os
import lsmtool
os.system('wget -O tgts.skymodel "https://lcs165.lofar.eu/cgi-bin/gsmv1.cgi?coord=%f,%f&radius=%f&unit=deg"' % (169, 52, 6/2.)) 
lsm = lsmtool.load('tgts.skymodel')

This works in v1.4.5 but fails with the following error in v1.4.6

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3251, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-12832a968c1c>", line 11, in <module>
    lsm = lsmtool.load("tgts.skymodel")  # , beamMS=MSs.getListObj()[0])
  File "/usr/local/lib/python3.8/dist-packages/lsmtool/__init__.py", line 85, in load
    return SkyModel(fileName, beamMS=beamMS, VOPosition=VOPosition,
  File "/usr/local/lib/python3.8/dist-packages/lsmtool/skymodel.py", line 121, in __init__
    self.table = Table.read(fileName, format='makesourcedb')
  File "/usr/lib/python3/dist-packages/astropy/table/connect.py", line 52, in __call__
    out = registry.read(cls, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/astropy/io/registry.py", line 523, in read
    data = reader(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/lsmtool/tableio.py", line 156, in skyModelReader
    table = createTable(outlines, metaDict, colNames, colDefaults)
  File "/usr/local/lib/python3.8/dist-packages/lsmtool/tableio.py", line 191, in createTable
    patchCol = 'col{0}'.format(colNames.index('Patch')+1)
ValueError: 'Patch' is not in list

both were using astropy v4.0

Concatenating two skymodels with different formats gives mixed format

Concatenating two skymodels with different formats (name - patch -type vs name - type - patch) will return a mixed-format file (which will cause issues when reading). In particular, the skymodel itself seems to be fine but the "default" line differs in format from the rest of the file.

I concatenated the files by loading both in with lsmtool.load() and then concatenate with .concatenate().

Attached are the skymodel files (3C196.skymodel.txt and Ateam.skymodel.txt are concatenated into output.skymodel.txt).

Thanks in advance!
Christian Groeneveld
output.skymodel.txt
Ateam.skymodel.txt
3C196.skymodel.txt

astropy 3.0 requirement breaks python2.7 installl

Whgen try to install LSMtool using setup.py, it will install astropy as a requirement. The astropy install for 3.0 rc1 fails to install in python2.7. As a quick fix I tweaked my local version the the setup.py to install_requires=['numpy','astropy >= 0.4, <3.0'],. That works for now. Not sure what the planning for the future is though.

Do not output files when comparing skymodels

The current SkyModel.compare() method always outputs several files. This may not be desirable.

One way to make this behavior optional is to set the outDir=None instead of outDir='.' in compare(). Then, plots should only be made if outDir is not None. Similarly, the stats.txt file should only be created if outDir is not None.

By changing the behavior of compare(), we do not break code that uses the run() method, because that sets 'outDir to . by default. Still, this is a change in behavior, so should be discussed before being accepted.

lsmtool requires older astropy version

lsmtool 1.4.3 requires astropy<4.0,>=3.2, but you'll have astropy 4.3.1 which is incompatible.

test suite fails

λ  python setup.py test
running test
================================================ test session starts =================================================
platform linux2 -- Python 2.7.12 -- py-1.4.20 -- pytest-2.5.2
collected 13 items

tests/test_lsmtool.py .........FF..

====================================================== FAILURES ======================================================
_____________________________________________________ test_merge _____________________________________________________

    def test_merge():
        print('Merge patches Patch_0 and Patch_2')
        s.merge(['Patch_0', 'Patch_2'], name = 'merged_patch')
>       assert len(s.getPatchNames()) == 102
E       assert 139 == 102
E        +  where 139 = len(array(['Patch_1', 'Patch_10', 'Patch_100', 'Patch_101', 'Patch_102',\n       'P...      'Patch_97', 'Patch_98', 'Patch_99', 'merged_patch'], \n      dtype='|S50'))
E        +    where array(['Patch_1', 'Patch_10', 'Patch_100', 'Patch_101', 'Patch_102',\n       'P...      'Patch_97', 'Patch_98', 'Patch_99', 'merged_patch'], \n      dtype='|S50') = <bound method SkyModel.getPatchNames of <lsmtool.skymodel.SkyModel object at 0x7fc6c6e03c10>>()
E        +      where <bound method SkyModel.getPatchNames of <lsmtool.skymodel.SkyModel object at 0x7fc6c6e03c10>> = <lsmtool.skymodel.SkyModel object at 0x7fc6c6e03c10>.getPatchNames

tests/test_lsmtool.py:73: AssertionError
-------------------------------------------------- Captured stdout ---------------------------------------------------
Merge patches Patch_0 and Patch_2
_______________________________________________ test_setPatchPositions _______________________________________________

    def test_setPatchPositions():
        print('Set patch positions to midpoint of patch')
        s.setPatchPositions(method='mid')
>       assert round(s.getPatchPositions()['merged_patch'][0].value, 4) == 277.7663
E       assert 274.2124 == 277.7663
E        +  where 274.2124 = round(274.2123791507774, 4)
E        +    where 274.2123791507774 = <Angle 274.2123791507774 deg>.value

tests/test_lsmtool.py:79: AssertionError
-------------------------------------------------- Captured stdout ---------------------------------------------------
Set patch positions to midpoint of patch
======================================== 2 failed, 11 passed in 33.13 seconds ========================================

should getPatchSizes() return a dict?

Compared to getPatchPosition() that returns a dict with {patchname:coord}, getPatchSize() returns an ordered array of sizes. However if one has to use both it becomes complicated as dicts are not ordered.

python setup.py sdist doesn't bundle test files

also gives a warnings:

warning: sdist: standard file not found: should have one of README, README.rst, README.txt

You probably need to add a manifest file.