robelgeda/dsii_dwarfs

DSII Dwarfs

dsii_dwarfs's Issues

Double check the isochrones

It's surprising that the galaxies are resolving into stars so well. Double check the magnitudes of the RGB as a function of age and metallicity in the isochrones.

Galaxy Mass Picker

Draw a galaxy at mass at random from Schechter function.

Create Over-sampled HSC PSFs

Options:

• Find where they store their PSF models and understand it.

• Measure the FHWM with Moffat profile.

Galaxy axial ratio picker

Choose the galaxy axial ratios at random from a distribution function that is representative of real galaxies.

This splits the axial-ratio picking off from #5.

Add unit tests

Get more info on HSC background noise trends

Download a bunch of HSC images (there may be an API for this), and measure the background noise levels in each wavelength, being sure to separate the wide from the deep from the ultra-deep portions of the survey. I usually use the sigma_clipped and mad_std statistics in astropy for this. Put in a table along with the info on filter and exposure time.
We are mostly interested in the co-adds.

Look at the trends in noise level vs. band for each exposure depth.

This is important if we are going to make more postage stamps. It's not particularly important if we are just going to jump to inserting the images into the real HSC images.

Make a folder for simulation Vs. machine_learning folders in dsii_dwarfs

As we grow the project we expect more code to spill into the library. As such we should move all the code in the library to a subfolder called simulations for v0.2.0

Compare WFISim with STIPS

Add download link to MIST_v1.2_vvcrit0.4 data

imf library versioning

Keep track of what commit of the imf library we are developing with.

Galaxy Surface Brightness Profile Picker

Choose a 2D model profile for the galaxy
- Typically, these galaxies have Sersic profiles with n ~ 1, half-light radii of ~ 0.5-1 kpc and axial ratios between 0.5 and 1
- We will ultimately want to draw these from distribution functions (there are some in the literature)
- This should be oversampled by at least a factor of 4 relative to the pixel sampling of the telescope/survey of choice (e.g. WFIRST, LSST or HSC).

Isochrone Picker

Given an age in metallicity chose the isochrone file name.

What is the Color of the Smooth Stellar Component

These are the low mass stars that we will not be simulating. We will need to sum up the flux in each band in the isochrone for masses below our cutoff M_indv.

imf library bug

dsii_dwarfs/tests/test_simulation.py F                                   [100%]
808
809=================================== FAILURES ===================================
810____________________ test_simulate_psf_fit_dwarf_elliptical ____________________
811
812    def test_simulate_psf_fit_dwarf_elliptical():
813        sim = SimulatePSFFitDwarfElliptical()
814>       sim.run_all_steps()
815
816dsii_dwarfs/tests/test_simulation.py:5: 
817_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
818dsii_dwarfs/simulate_psf_fit_dwarf_elliptical.py:506: in run_all_steps
819    step()
820dsii_dwarfs/simulate_psf_fit_dwarf_elliptical.py:329: in create_stochastic_portion
821    massfunc='kroupa', mmin=self.minmass_stochastic)
822/home/travis/build/robelgeda/dsii_dwarfs/src/imf/imf/imf.py:453: in make_cluster
823    mfc.mmin = mmin
824/home/travis/build/robelgeda/dsii_dwarfs/src/imf/imf/imf.py:131: in mmin
825    self.distr.m1 = value
826/home/travis/build/robelgeda/dsii_dwarfs/src/imf/imf/distributions.py:154: in m1
827    self._calcpows()
828/home/travis/build/robelgeda/dsii_dwarfs/src/imf/imf/distributions.py:171: in _calcpows
829    pows.append(PowerLaw(self.slopes[ii], self.breaks[ii], self.breaks[ii + 1]))
830_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
831
832self = <imf.distributions.PowerLaw object at 0x7f87433a8b10>, slope = -0.3
833m1 = 0.8141192614821318, m2 = 0.08
834
835    def __init__(self, slope, m1, m2):
836        """ Power law with slope slope in the interval m1,m2 """
837        self.slope = slope
838        self.m1 = float(m1)
839        self.m2 = float(m2)
840>       assert (m1 < m2)
841E       AssertionError
842
843/home/travis/build/robelgeda/dsii_dwarfs/src/imf/imf/distributions.py:90: AssertionError
844----------------------------- Captured stdout call -----------------------------
845pick_galaxy
846renormalize_isochrone
847compute_smooth_flux
848create_smooth_portion
849Peak smooth cts/s in g band: 0.0012585011987121635
850Peak smooth cts/s in r band: 0.0013534189607019703
851Peak smooth cts/s in i band: 0.0013138824882756896
852Peak smooth cts/s in z band: 0.0010984265422044166
853Peak smooth cts/s in y band: 0.0012095844859201863
854create_stochastic_portion
855=========================== short test summary info ============================
856FAILED dsii_dwarfs/tests/test_simulation.py::test_simulate_psf_fit_dwarf_elliptical
857============================== 1 failed in 12.31s ==============================
858The command "$MAIN_CMD $SETUP_CMD" exited with 1.

MNT: Stop using ci-helpers in appveyor.yml

To whom it may concern,

If you are using https://github.com/astropy/ci-helpers in your appveyor.yml , please know that the Astropy project has dropped active development/support for Appveyor CI. If it still works, good for you, because we did not remove the relevant files (yet). But if it ever stops working, we have no plans to fix anything for Appveyor CI. Please consider using native Windows support other CI, e.g., Travis CI (see https://docs.travis-ci.com/user/reference/windows/). We apologize for any inconvenience caused.

If this issue is opened in error or irrelevant to you, feel free to close. Thank you.

xref astropy/ci-helpers#464

Make Test HSC Images

Simulate and place dwarfs into actual HSC images for testing

Parameters for WFI

self.isochrone_dir
self.isofilestring
self.isochrone_column_formatter
self.inst_std # Estimate of std of WFI image background (noise)
self.max_allowed_npix = 1024 # Only used if npix is None

Output the mass cut between smooth & stochastic

It would be good to add the mass cut between the smooth & stochastic components to the metadata. Actually, also add:

• mass fraction in stochastic
• galaxy total flux, absolute, and apparent magnitude in each band
• peak number of stars per pixel inserted in the stochastic component
• stochastic flux and apparent magnitude in this peak pixel

Integrating elliptical Sersic profiles

We are currently using integrate.dblquad to integrate 2D Sersic profiles. We can improve this by finding an analytical solution to this problem. Equation (7) in Peng, C.Y. et al. looks promising.

Some dwarfs are fainter than the noise in the HSC images

Some of the distant dwarfs are very faint with their max flux pixels falling well below the STD of the images. I am using distances 1 Mpc to 15 Mpc and mf_min=1.e5, mf_max=1.e9.

Dealing with varying PSFs

In the case of HSC and LSST data, the PSF will vary significantly from image to image. But in both cases the calibration pipeline will produce a PSF for us. So we probably ought to train on sets of images that are paired with their PSFs.

Check what real dE galaxies look like at 10-15 Mpc

Maybe HSC has imaged some dE galaxies in g,r,i,z,y. A good place to look would be in the Leo I group. There is an ancient paper by Ferguson that gives a catalog. (Otherwise, try the Virgo Cluster...catalog in Binggeli+198?). If the HSC covers the field, download and look in all the bands to see if the galaxies resolve into stars in the redder bands, as they do in the simulations.

cookiecutter module name error

cookiecutter does not allow - sign in the name of a module

How do we pick where the smooth component begins

Because it's easy given the way the isochrones are stored, just saying "main sequence is smooth", "RGB is not" is easy for now. But for e.g. star-forming galaxies that's not sufficient because the brightest stars are probably BSG or RSG stars (which are MS). So we should consider how to have a slightly better definition of the "smooth" component.

Fiducial idea: define a "pixel" for a given galaxy (or an average density), work down the IMF with random samples, and when you hit a certain number (10? 100?) of stars in that one pixel, that's the cutoff. Repeat 1000x, and pick the median of that answer.

Find HSC Zero Points

Numbers that convert from Mag to counts per second in the image.

Many of the galaxies overfill the postage stamps

Double-check the sizes, but probably they are okay. If so, we should. Since we are eventually just going to be adding the to much bigger Subaru images, we might not need to get the neural-network to work with bigger postage stamps, but just move over to the one that works on scanning the big images.

Get more HSC PSF statistics

Download a bunch of HSC PSFs (I think there is an API for this), being careful to separate those for the wide, deep and ultra deep portions of the survey. Measure their Moffat parameters for each of the individual g,r,i,z,& y bands, and put them in a table. Look for trends in gamma & alpha with wavelength. Save some measure of goodness of fit as well, since it would be nice to know if the Moffat is a decent model.

Keep the PSFs around because we may down the road need to change from a Moffat to something else.

Data Packaging Ticket

The simulated cutout will have images in about 5 filter channels (each filter has a 2D image). When we deliver the final simulated product, do we want each image to be stored as 3D cubes with filter being the third dimension?

Add documentation

into page needs intro

Figure Out How Many Stars to Pick for Galaxy

Star Mass Picker

Draw a star mass from at random from an initial stellar mass function:
Draw star masses at random from an initial mass function
- There is a package for this https://github.com/keflavich/imf. One could also do this with Astropy models no doubt, but that would require more thought.
- need to decide on a lower mass limit for this so we don't spend all our time on stars that don't contribute to fluctuations. Add those in as one additional smooth component, with the right total flux relative to the stars we have added individually.
- draw enough stars that the total mass of the smooth + individual components adds up to the total mass you want for the dwarf galaxy.
- $N_{indiv}$ is the total number of individual stars

[M Priority]

Update package infrastructure

AstroPy helpers is outdated