AST Library and Cupid

[chileanvirtualobservatory]

AST Library and Cupid

[msanchezc]

Hi @maray,
I agree with you that using the whole AST library may sound overkill, but my idea of wrapping an existing application, in this case Starlink's CUPID, is to put the thinnest layer possible around it to make it work with another interface while still supporting future developments of the wrapped application. Furthermore, wrapping AST was only the first step, now I have a fully working Python wrapper of the algorithms in CUPID that doesn't require modifying a single line of C source code. Obviously, the tradeoff is having to install a subset of the Starlink suite (which is very simple to install via the make tool, but I want to look for a more Pythonic solution).

[maray]

The problem is that starlink is not a library, but a suite as you said. MERS library is a logging system (python already have one), AST includes reimplementations of dictionaries for C (made by astronomers), the code only supports the HDF5 format, which is far from the FITS standard, etc... It is sure that we want to wrap, but the guts replacement is necessary for being consistent with the Pythonic ecosystem

[maray]

In summary, @msanchezc pushes for including AST, @maray for not. I would like to know the opinion of @autopawn and the final resolution of the package maintainer @mavillan.

[msanchezc]

Hi everyone!
This is my current version of pycupid. As I said before, this is a wrapper that requires a good part of the Starlink Software Collection. As of now, compilation is required (but fear not, as a Makefile takes care of everything), and it works only on Linux (I tested it in Ubuntu 16.04). I'm researching for a way to deliver it as a pre-built package (as a wheel package, as described in PEP 513 and implemented in manylinux for the Linux case). If a pre-built package for every common platform (Windows, macOS, Linux) can be generated and uploaded to PyPI, then I don't see any major drawbacks to this approach. What do you think?

[maray]

I think this is a good idea for an independent cupid package. However, this means that if I want to install it form sources I should download the full starlink repository that weight like a Giga and compile it completely?. @autopawn also thinks that messing with the guts of cupid is a bad idea, so if this is the way that we will go (let us wait to @mavillan resolution), I think we should compile only AST, MERS and HDS support. Also, have you tested this?. The HDS, NDF and ipd looks a messy busyness, but we can discuss that in the other repo if we include the repo in the ChiVO group. Also, we can invite D. Berry and T. Jenness to solve this problem. They are thinking to divide starlink in several repositories, which will be most welcomed from our side :D

[msanchezc]

Yes, it works on Ubuntu (and it should work on every major Linux distro). I tested the four algorithms (ClumpFind, FellWalker, GaussClumps and Reinhold) using @mavillan's tests from ACALIB.
Currently, my version of pycupid has to download the full Starlink repo (yes, that Giga-something behemoth), but compilation is done in a custom way supported by Starlink to just build CUPID with all its dependencies (it's not the full install, but several dependencies are needed and, at least for me, the process takes about an hour to complete). Like I said before, I'm researching for a way to distribute it pre-built, so that the compilation step is not needed. This can be done using Python wheels to encapsulate the wrapper along with the required binaries, compiled for every major operating system supported by PIP (that's how numpy and other scientific packages are distributed in PyPI).

[maray]

Ok... so we have decided using pyCupid with the code directly compiled from starlink... closing