dkirkby/bayez

Bayesian redshift estimation

bayez's Issues

MNT: Stop using ci-helpers in appveyor.yml

To whom it may concern,

If you are using https://github.com/astropy/ci-helpers in your appveyor.yml , please know that the Astropy project has dropped active development/support for Appveyor CI. If it still works, good for you, because we did not remove the relevant files (yet). But if it ever stops working, we have no plans to fix anything for Appveyor CI. Please consider using native Windows support other CI, e.g., Travis CI (see https://docs.travis-ci.com/user/reference/windows/). We apologize for any inconvenience caused.

If this issue is opened in error or irrelevant to you, feel free to close. Thank you.

xref astropy/ci-helpers#464

Update desi_bayez.py

@dkirkby @fjaviersanchez

The pull request for updating bayez to the new specsim api is almost completed I just need to finish the update on desi_bayez. I think the main thing that needs to be changed is that the downsample method should be removed from that file entirely and probably placed in another module all together. Should we create something like a helper module to migrate functions like migrate to? What do you think the best approach is for changing this file.

I'm also not sure how these changes should be tested.

Add DESI module

This issue is to add a bayez.desi module to encapsulate DESI-specific interfaces to bayez. I suggest the following functions:

• read_bricks(): Read b,r,z bricks and prepare for analysis by bayez.
• write_bricks(): Write b,r,z, bricks including truth using spectra generated by bayez simulation code.
• write_zbest(): Write bayez results to zbest file.
• desi_zfit(): Read bricks, perform redshift fits, and write zbest file.

Study efficiency - purity tradeoffs

Define a boolean truth selector:

T(v1) =  { c | zfit - ztrue | / (1 + ztrue) < v1 }

and a boolean data selector, for example:

D(v2) = { c (z68hi - z68lo) / 2 / (1 + zbest) < v2 }

Then we can count the numbers of fits in four exclusive categories, which sum to N(total):

• Positively selected in data: N(D)
  • True positives: N(D & T)
  • False positives: N(D & ~T)
• Negatively selected in data: N(~D)
  • True negatives: N(~D & ~T)
  • False negatives: N(~D & T)

Focus on two metrics:

• efficiency = N(D & T) / N(total)
• purity = N(D & T) / N(D)

This issue is to:

• Write code to calculate curves of efficiency and purity versus v2 given a definition of D(v2) and a fixed v1.
• Find the optimum definition of D(v2) and choice of v2 given a minimum allowed efficiency and purity for some v1.

The optimum might be different for different target categories and can be used to set ZWARN = ~D(v2).