Comments (13)
Comment by Steve Goldman on JIRA:
This issue was recently raised again in a helpdesk ticket INC0198393. It's unclear when the issue arose, or whether this has always been a problem. Most of the related files don't seem to have been updated in at least the last 5 years.
The critical code is line 131 of processInput.py where user values are merged with mdriztab default values. Since the mdriz file (which hasn't been updated recently) has a default value for final_rot of {}None{}, the user value is being replaced by {}None{}.
There is a comment in the code there stating:
# interpret MDRIZTAB, if specified, and update configObj accordingly
# This can be done here because MDRIZTAB does not include values for
# input, output, or updatewcs.```
from drizzlepac.
Maybe @mackjenn can remember how MDRIZTAB
was intended to be used. I very vaguely recall some discussions around MDRIZTAB
(> 5 years ago). It could be that the code actually works as intended (MDRIZTAB
overrides user input - it could be what INS wanted) but MDRIZTAB
should not have had custom WCS parameters: obviously these are not intended to be used in the pipeline. So, either the logic in the code is buggy or the MDRIZTAB file contains fields that it was not supposed to have.
For now, users can load MDRIZTAB
and delete parameters that they supply as input to astrodrizzle:
from drizzlepac.mdzhandler import getMdriztabParameters
mdrztab = getMdriztabParameters(['file_name1_flc.fits'])
param_to_del = [
'final_wcs',
'final_outnx',
'final_outny'
]
for k in param_to_del:
del mdrztab[k]
AstroDrizzle(
input,
mdriztab=False,
configobj=mdrztab,
final_wcs=True,
final_outnx=1111,
final_outny=2222
)
from drizzlepac.
Thanks Mihai, this is really helpful. Unfortunately, it looks like deleting the parameters results in those parameters no longer being recognized as viable user inputs. That being said, I think I can find a fix along these lines using getMdriztabParameters
. Thanks!
from drizzlepac.
A possible solution is something like the following, however, there seem to be some issues with the result. Still investigating.
from stsci.tools import teal
from drizzlepac.mdzhandler import getMdriztabParameters
configObj = teal.load("astrodrizzle")
configObj["mdrztab"] = getMdriztabParameters(["j96001cdq_flt.fits"])
input_flts = [
"j96001cfq_flt.fits",
"j96001cdq_flt.fits",
"j96001cgq_flt.fits",
"j96001ciq_flt.fits",
"j96001clq_flt.fits",
"j96001cjq_flt.fits",
"j96001cmq_flt.fits",
"j96001coq_flt.fits",
]
configObj["input"] = input_flts
configObj["output"] = "SUMMED_test"
configObj["mdrztab"]["driz_cr_corr"] = True
configObj["mdrztab"]["final_refimage"] = "hlsp_hlf_hst_wfc3-60mas_goodss_f275w_v2.0_conv_sci.fits"
configObj["mdrztab"]["final_rot"] = -35
configObj["mdrztab"]["final_outnx"] = 10000
configObj["mdrztab"]["final_outny"] = 10000
configObj["STEP 7a: CUSTOM WCS FOR FINAL OUTPUT"]["final_wcs"] = True
astrodrizzle.AstroDrizzle(
configobj=configObj,
)
from drizzlepac.
Comment by Steve Goldman on JIRA:
After discussing this with Warren, it looks like the desired outcome is still being produced. The mdriztab=True option is resulting in the best pipeline parameters being passed to the configobj. The problem is that, while the mdriztab pipeline parameters should supersede the user's parameters, there should be a warning to the user that this is happening.
from drizzlepac.
The mdriztab=True option is resulting in the best pipeline parameters being passed to the configobj.
I have to disagree with this: the best parameters are the ones that user wants. And those seem to be ignored. For the very least there has to be a workaround developed for the users.
from drizzlepac.
The mdriztab=True option is resulting in the best pipeline parameters being passed to the configobj.
I have to disagree with this: the best parameters are the ones that user wants. And those seem to be ignored. For the very least there has to be a workaround developed for the users.
I think you're right. This isn't the way it SHOULD work. I should have written above that this was the "previously" desired outcome. That being said, I'm not sure how common of an issue this is, and whether it's worth changing the way that the configuration parameters are merged. I will ping some of the other drizzling folks to see if the juice is worth the squeeze.
from drizzlepac.
Most likely users do not even realize that their options are ignored and simply move on with processing. Pipeline is probably using mdriztab
exclusively. So, allowing users to overwrite some of the parameters loaded from mdriztab
would be the right thing to do, IMO.
And if you want to keep the existing behavior, I think that the warning mentioned in a previous message should be upgraded to an exception to force users remove their custom parameters: mdriztab
must be used exclusively; no user parameters allowed. + provide an example with a workaround.
from drizzlepac.
I totally agree. Good call 👍
from drizzlepac.
Chiming in here... I agree with Mihai. The desired behavior is for the user to be able to use the MDRIZTAB default values as a starting point and then allow for any changes to those parameters from the command line.
For example, final_rot=INDEF in the MDRIZTAB, but you should be able to override this by setting final_rot=0 to drizzle north up, while retaining the other recommended values for each instrument.
astrodrizzle.Astrodrizzle('i*flt.fits', mdriztab=True, final_wcs=True, final_rot=0)
from drizzlepac.
Comment by Michele De La Pena on JIRA:
As indicated, messages need to issued so the user is informed.
I will note that the logic here is not typical though there may be other issues at play. Under most circumstances, any values supplied on the command line as a direct parameter or as part of a file which is a direct parameter to a task should supersede any configuration values read by default by the task.
At the least this needs to be well-documented. We can talk about this, but perhaps it should just be fixed according to expectation.
from drizzlepac.
Comment by Steve Goldman on JIRA:
Michele De La Pena After some discussions elsewhere including the github issue 1739, I think it was unanimous by the instrument scientists (and Mihai) that the user inputs should supersede the mdriztab defaults. I have a PR that should do the trick, and I am testing that now.
from drizzlepac.
Comment by Steve Goldman on JIRA:
closed by #1774
from drizzlepac.
Related Issues (20)
- HAP: Test the segment catalog source position update which uses the Gaussian kernel HOT 20
- Test if AstrometryDB applies WCS solutions correctly HOT 1
- issues raised by SonarQube static code analysis HOT 1
- Investigate devdep github test failures
- Create new config files and keywords for relative alignment HOT 1
- Discussion: Config file update with different values for relative and absolute alignment HOT 7
- Determine the best relative fit for pipeline and svm products
- Save "rel" header keywords for best relative fit for pipeline and SVM products
- Verify keyword mapping to values in astrometry database
- Catalogs: Sky background is incorrect when image has too many unused pixels HOT 3
- fix issues raised by sonarscan HOT 3
- Support for int64 Context Maps HOT 1
- Correct calculation of n1_exposure_time for HAP images HOT 8
- Test new "rel" fit requirements for relative pipeline products HOT 2
- Test new "rel" fit requirements for relative SVM products HOT 1
- Add "if" statement to code block unnecessary for ACS/SBC and WFC3/IR processing HOT 1
- CALACS: Update ACS/WFC truth files to accommodate the serial CTE HOT 1
- Test numpy 2.0 in STWCS
- Create cron job for STWCS
- Investigate IDCTAB=N/A
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from drizzlepac.