mrosseel/vast-automation

Using VaST and astropy to measure lightcurves of known and unknown variable stars

vast-automation

Introduction

This project takes as input:

• a number of FITS files of the same region in the sky (time series)
• the resulting files of running VaST on these files
• the observatory location (settings.txt)

and produces a website containing this information:

• phase diagram charts of VSX stars and new variable candidates (as detected by VaST)
• light curves charts
• magnitude error vs JD chart
• Ensemble comparison star lightcurves
• Global charts on aperture vs airmass and aperture vs JD
• AAVSO reports

Most output can be turned off if it's of no interest.

Preparation

Importing VSX star catalog

• First time download of AAVSO VSX catalog: use the script vsx_download.sh
• If you want to update: use the same script.
• Is an update needed? Each time you download the VSX catalog, its date is written in vsx_last_modified.txt. Compare this with the output of the script vsx_check_last_modified.sh.
• check that vsx_catalog.bin has been written successfully

Importing UCAC4 star catalog

Getting the 900 files (9Gb):

• wget ftp://cdsarc.u-strasbg.fr/0/more/UCAC4/u4b/*

Checking that all 900 files were downloaded correctly:

• md5sum -c md5sum.txt

Also download some support files:

• wget ftp://cdsarc.u-strasbg.fr/0/more/UCAC4/u4i/*

and place these two directories (u4b and u4i) in this location:

• support/ucac4/UCAC4/

Run VAST on the fits files

./vast -u -x 3 ../location/of/fits/*.fit

This uses UTC time, and ignores the 'blended' flag for stars which are close to each other. This will generate many vast files in the vast directory

Process VAST results

Needed: python 3.8+

run the command

[Install poetry][https://python-poetry.org/docs/#installation] to manage the python dependencies.

Install the dependencies and the virtual environment (do this once):

poetry install

Activate a python virtual environment (do this every time):

poetry shell

Run the actual processing software to get all options:

./vast_process.sh -h

Generate a site

If you checked out the VSX dir you can update the submodule while being in the site/vsx/themes directory:

git submodule update --init --recursive

You can also download it from the github page and put in site/vsx/themes: https://github.com/theNewDynamic/gohugo-theme-ananke

Example usage:

./vast_process.sh --vsx --candidates -d support/vast-1.0rc84 --fitsdir ./fits --apikey abcde -r /bla/my-result-dir/

Note that the fitsdir and apikey options are needed to perform automatic plate-solving via astrometry.net. If you prefer to do your own plate-solving, put the solved image as new-image.fits in the directory passed via -d

Note that the fitsdir should contain the same files as were used by VaST.

look at the results

This command line above will generate vsx information and create phase diagrams for all vast autocandidates and vsx stars. Also a few extra files are generated:

• vsx_stars.txt
• vast_list_of_all_stars_pos.txt
• vast_autocandidates_pos.txt

Local id catalog ( -l )

The file which can be passed is a CSV file. Example:

# our_name, local_id, minmax, min, max, var_type, period, period_err, epoch
MY-STAR-NAME, 101, 16.03-15.15, 16.03, 15.15, RRAB, 0.15,0.015, 2458849.59206

Minimal example:

# our_name, local_id, minmax, min, max, var_type, period, period_err, epoch
MY-STAR-NAME, 101, , , , , , ,

RADEC catalog ( -o )

This file is generated by the software, and is similar to the localid star file: instead of a local id this file has ra/dec of the star. The file which can be passed is a CSV file. Example:

# our_name, ra, dec, minmax, min, max, var_type, period, period_err, epoch
MY-STAR-NAME, 10.123, 15.321, 16.03-15.15, 16.03, 15.15, RRAB, 0.15,0.015, 2458849.59206

Minimal example:

# our_name, ra, dec, minmax, min, max, var_type, period, period_err, epoch
MY-STAR-NAME, 10.123, 15.321, , , , , , ,

Docker (no longer maintained)

The docker setup is currently mainly used to run Jupyter notebooks, but should also be a well-setup environment to run vast-automation.

Starting Docker

• have a working docker installation: https://www.docker.com/community-edition
• cd docker
• docker build . -t mrosseel/munipack-automation
• cd ..
• command: ./startJupyter.sh
• you are automatically logged into a root shell of the docker container

Docker & Jupyter lab

The docker image exposes a Jupyter lab instance on port 8888. Password is 'muni'

TODO

• for cli_inspect, add the chosen fits to the txt file
• for candidates, closest known vsx
• check out https://public.lanl.gov/palmer/fastchi.html for period determination
• check out https://github.com/toros-astro/astroalign for aligning
• stacking images to detect fainter stars+have better signal/noise ratio: https://github.com/fedhere/coaddfitim

References

• https://books.google.ae/books?id=g_K3-bQ8lTUC&pg=PA236&lpg=PA236&dq=munipack+aperture&source=bl&ots=P4BKKI25HG&sig=Lj9Kg6EZi2pwKZXK5Hk5_B4qcVg&hl=en&sa=X&ved=0ahUKEwj4qr6o4evZAhWILcAKHdtnAqYQ6AEIRDAE#v=onepage&q&f=false
• https://www.aavso.org/how-report-new-variable-star-discoveries

vast-automation's Issues

Period search margins and algorithm

The lower margin for the period search should not be set much above the Nyquist frequency of the dataset. For the IC5070 data, taking 0.5 d as the lower limit and t_base/2 as the upper limit avoids many wrong periodicities. By switching to the Generalized Lomb Scargle Periodogram, a lower limit for 0.5 can taken as a conservative lower limit for a strong periodicity with a negligible FAP. Lower peaks should be carefully checked by looking at the lightcurves. I have uploaded a python code which uses the above suggestions to compute the periodograms for thz IC 5070 data.

Magnitude range is not present in CSV output

private example:
RMH-HMB-18 has the range, 19 doesn't

Epoch is not imported from VSX data

In the site, the epochs present in VSX are not shown and are also not used in the phase diagram.

Example in private dataset: ASASSN-V J180521.55-434704.3
https://www.aavso.org/vsx/index.php?view=detail.top&oid=552295

Period_err for VSX stars is output as nan

There is no period info for VSX (I think) but either way if it's not present it should be empty output, not nan.

UY Col,UCAC4 297-008674,89.9295217,-30.6594078,9.48  V 9.63  V,9.6,9.5,DSCT,0.16955,nan,2451869.018

src/do_vast.py is missing from the code tree

After cloning the project the resulting source tree does not include the file src/do_vast.py which is needed by the main processing script ./vast_process.sh