phangsteam/pjpipe

Pipeline for processing JWST imaging data, tailored for nearby galaxies. Built for PHANGS

pjpipe's Issues

[FEATURE]: Add an option for cross-correlation in get_wcs_adjust_step

Throwing this out while it's fresh in my mind: adding in an option for getting shifts in get_wcs_adjust via cross-correlation could be promising (at least, more promising than trying it between different wavelengths). Given that the tiles have matched resolutions I have some hope here.

This is probably useful for the Cy2 data where we don't have the point-sourcey F1000W band (@rchwn) and also the NGC253 data near the centre where everything is blown out to the point where we don't have sources to detect (@akleroy).

Maybe something like space pylote (sp) could be rolled out here without too much stress (@ejwatkins-astro/@emsellem). It'd be good to get thoughts!

File control for MIRI Offs

Develop control method at the TOML level to control custom MIRI offs.

[FEATURE]: Destriping for MIRI data

I'm very intrigued by the success you've had destriping the NIRCam data. I have MIRI data in F560W and F770W that also has what looks like similar amounts of striping as the NIRCam data shown in your paper. I was wondering if the destriping part of the pipeline can also be applied to MIRI data in some way?

[BUG]: PJPipe incompatible with newest version of reproject

Describe the bug
The newest version of reproject, 0.14.0, is incompatible with PJPipe. Specifically, the __init__ for ReprojectArraySubset now expects imin, imax, jmin, and jmax to be provided together as a single variable ,bounds.

To Reproduce
Any attempt to run the level matching step in PJPipe.

Expected behavior
Not crashing.

[BUG]: AttributeError: lv2 'NoneType' object has no attribute 'lower'

Hi,

I'm very excited to use this pipeline for some new MIRI observations I'm working with of a nearby galaxy, it seems like such an incredible resource for the community! I've gotten through the lvl1 processing, but when I get to level 2, I get the following error:

[2024-01-30 14:13:54,639] INFO - Beginning lv2 for F560W
[2024-01-30 14:13:54,639] INFO - Building asn files
Traceback (most recent call last):
File "/Users/wcramer2/Desktop/pjpipe/run_reprocessing.py", line 28, in
pjp.do_pipeline()
File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/pipeline.py", line 513, in do_pipeline
step_result = lv2.do_step()
^^^^^^^^^^^^^
File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/lv2/lv2_step.py", line 127, in do_step
asn_files = self.create_asn_files(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/lv2/lv2_step.py", line 172, in create_asn_files
tab = get_obs_table(
^^^^^^^^^^^^^^
File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/utils/utils.py", line 480, in get_obs_table
parse_fits_to_table(
File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/utils/utils.py", line 530, in parse_fits_to_table
obs_label = im.meta.observation.observation_label.lower()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'lower'

The only key word I have set for lvl2 is: bgr_check_type = 'check_in_name'

I was wondering if what might be contributing is we don't have any "background" exposures for our data set. We have a particular image of a more blank part of the sky as part of the mosaic we were planning to use for a background image, but all images in the data set are currently labeled as "SCI" in the header. If this is causing it, any way to point to these particular frames to use for background?

[FEATURE]: Add pipeline and DR version to file headers

It's time to add pipeline version or DR version into the FITS headers of the results. Note that the “anchor” files have this versioning information:

FILENAME= 'ic5332_nircam_lv3_f360m_i2d_align.fits' / Name of the file
SDP_VER = '2023_2a '           / Data processing (DP) Software Version
PRD_VER = 'PRDOPSSOC-063'      / S&OC Project Reference Database (PRD) Version
OSS_VER = '8.7     '           / Observatory Scheduling Software (OSS) Version
GSC_VER = 'GSC2431 '           / Guide Star Catalog (GSC) Version
CAL_VER = '1.12.3.dev26+g9dfaa3724' / Calibration Software Version
CAL_VCS = '9dfaa3724'          / Calibration software version control sys number

The FILENAME should be shifted to anchor and we should add something like PIPE_VER = ‘1.0’ / PJPIPE version into the header and I certainly wouldn’t mind a PIPE_VCS github hash as well. Since there’s rapidly developing a notion of a data release versioning that is separate from the software, we might want a DR_VER keyword as well