dsavransky/exosims

Simulator for exoplanet direct imaging space missions

exosims's Issues

Singular matrix issue with WFIRSTL2

We've run into an interesting issue. When using WFIRSTObservatoryL2.py for the Observatory, we run into a singular matrix error when an interpolation is attempted using the L2_halo_orbit_six_month_v2.p file. The error is in the image attached.

What is odd is that when tried on two other machines, (mine and Joan's) we don't see the error, and the code runs fine. It's just on the WFIRST dev site at IPAC that this issue persists. I am thinking it's cause of an older LAPACK/BLAS version. Do you see any way around this (perhaps the pickle file might be upgraded to circumvent any singular matrix calculation)?

Need good template run_one script.

Planet non-detections being lost in divide-by-zero errors

We occasionally get these warnings (not crashes) while running the EXOSIMS code, in this case for MissionSim: Seed is: 507872323. I have attached the JSON file we used at the end.

EXOSIMS/Prototypes/SurveySimulation.py:505: RuntimeWarning: invalid value encountered in divide
  s,n = self.calc_signal_noise(sInd, pInds[obs], dt, mode)

This occurred with stars of index 153 and 319, which did not have obvious errors in magnitude or anything like that.

In SurveySimulation, the photon count rates returned are all zeros, so the divide calculates 0/0 which is obviously wrong. The planets are all closer than the inner working angle, so they should get flagged as "-1" in the detection criteria, but they go missing in the DRM instead, being empty.

As a separate suggestion, if the planets are within the inner working angle, it might save some time to have a short circuit in the code so that it doesn't compute count rates/etc which could save a bit of time.

JSON:

{
  "missionLife": 2.0,
  "missionPortion": 0.5,
  "ohTime": 0.1,
  "keepStarCatalog": true,
  "minComp": 0.1,
  "pupilDiam": 4,
  "obscurFac": 0,
  "dMagLim": 26,
  "telescopeKeepout": 60,
  "attenuation": 0.65,
  "intCutoff": 100.0,
  "settlingTime": 0.0, 
  "ppFact": 0.1,
  "prange":[0.2, 0.2],
  "erange":[0, 0.01],
  "eta": 0.1,
  "arange":[0.75, 1.77],
  "Rprange":[1.0, 1.0],
  "Mprange":[1, 1],
  "scaleOrbits": true,
  "constrainOrbits": true,
  "scienceInstruments": [
    {
      "name": "imagingEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000
    },
    {
      "name": "spectroEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000,
      "Rs": 50
    }
  ],
  "starlightSuppressionSystems": [
    {
      "name": "HLC550",
      "lam": 550,
      "IWA": 0.057,
      "OWA": 2.127,
      "ohTime": 0.1,
      "throughput": 0.2,
      "BW": 0.10,
      "occ_trans": 1.0,
      "core_thruput": 0.69, 
      "core_contrast": 1.7880e-10
    },
        {
      "name": "HLC1000",
      "lam": 1000,
      "BW": 0.1,
      "IWA": 0.1031,
      "OWA": 3.868,
      "ohTime": 0.2,
      "occ_trans": 1.0,
      "core_thruput": 0.69, 
      "core_contrast": 1.7880e-10
    }
  ],
      "observingModes": [
    {
      "instName": "imagingEMCCD",
      "systName": "HLC550",
      "detection": 1,
      "SNR": 7
    },
    {
      "instName": "spectroEMCCD",
      "systName": "HLC1000",
      "SNR": 5
    }
  ],
  "modules": {
    "PlanetPopulation": "EarthTwinHabZone1",
    "StarCatalog": "EXOCAT1",
    "OpticalSystem": "KasdinBraems",
    "ZodiacalLight": "Stark",
    "BackgroundSources": "GalaxiesFaintStars",
    "PlanetPhysicalModel": " ",
    "Observatory": "WFIRSTObservatoryL2",
    "TimeKeeping": " ",
    "PostProcessing": " ",
    "Completeness": "BrownCompleteness",
    "TargetList": " ",
    "SimulatedUniverse": " ",
    "SurveySimulation": " ",
    "SurveyEnsemble": " "
  }
}

gen_outspec lost the random seed

By moving the gen_outspec to SurveySimulation, we have removed the info that's added in MissionSim, which includes the random seed. This is very bad and should be fixed - probably easiest to have MissionSim package the seed into the specs dictionary, and then have SurveySimulation explicitly add it to the dictionary.

DRM output is inconsistent

some things have units and some don't. Also, most things are lists, whereas they'd be more useful as ndarrays.

Suggestion: standardize so that everything with multiple values is an ndarray and that everything that has a unit retains a unit.

currentTime & sInds size during characterization

I am hitting an error that I cannot easily address. Well, I might, but I honestly don't understand why I am getting what I'm getting.

It appears that when running a WFIRST + KnownRV simulation, the code hits an error in WFirstobservatory.keepout while checking the size of sInds vs. currentTime, while running run_sim. The error is below with a printout of some of the relevant local variable values:

What is odd is that this error happens at different times during different runs of run_sim. At times, this happens after the first couple of planets are detected, and others, like the one instance attached, during the middle of the simulation.

I have attached an image of the full traceback below, along with the json (as a txt file) file I used.
template_WFIRST_KnownRV.txt

Thank you!

overhead time should be applied to characterizations (separately from the detection overhead)

mode['syst']['ohTime’] should also be allocated separately for characterization (using the appropriate system value)

missionLife, in ICD

the samplescripts and ICD states that the name of the variable which defines the mission duration is missionLifetime. However, the TimeKeeping module uses missionLife as the name.
I recommend the samplescripts and ICD be updated to reflect the code usage.

infinite OBduration writes to outspec as invalid value

OBduration, when left at default (Infinity) goes to outspec as Infinity, which is not a valid json value. This should be handled in the same way as OWA.

BrownCompleteness comp0=EVPOC is empty?

I attempted to run end to end with today's newest release, including the updated BrownCompleteness.py.

Here is the Traceback:

IndexError Traceback (most recent call last)
in ()
----> 1 sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/MissionSim.pyc in init(self, scriptfile, **specs)
107 self.modules = {}
108 self.modules['SimulatedUniverse'] = get_module(specs['modules']
--> 109 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
110 self.modules['Observatory'] = get_module(specs['modules']
111 ['Observatory'],'Observatory')(**specs)

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/SimulatedUniverse/KeplerLikeUniverse.py in init(self, **specs)
21 def init(self, *_specs):
22
---> 23 SimulatedUniverse.init(self, *_specs)
24
25 def gen_planetary_systems(self,**specs):

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Prototypes/SimulatedUniverse.py in init(self, eta, **specs)
87
88 # import TargetList class
---> 89 self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
90
91 # bring inherited class objects to top level of Simulated Universe

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in init(self, keepStarCatalog, minComp, **specs)
134
135 # now populate and filter the list
--> 136 self.populate_target_list(**specs)
137 self.filter_target_list(**specs)
138

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in populate_target_list(self, **specs)
189 self.nan_filter()
190 # populate completeness values
--> 191 self.comp0 = Comp.target_completeness(self)
192 # populate maximum integration time
193 self.maxintTime = OS.calc_maxintTime(self)

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Completeness/BrownCompleteness.py in target_completeness(self, targlist)
108 dMag = dMag - 2.5*np.log10(L)
109
--> 110 comp0 = EVPOC(s.to('AU').value, dMag)
111
112 return comp0

/Users/rhonda/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/function_base.pyc in call(self, _args, *_kwargs)
1698 vargs.extend([kwargs[_n] for _n in names])
1699
-> 1700 return self._vectorize_call(func=func, args=vargs)
1701
1702 def _get_ufunc_and_otypes(self, func, args):

Disturbance force unit conversion error in Starshade calculation

Trying to run a starshade simulation with the JSON script appended at the end; seed 824926308

There is a unit error in the Force calculation at line 683-687 in Observatory.py

    684         dF = ((F_O/self.scMass - F_T/self.coMass)*self.scMass).to('N')
    685         dF_axial = np.dot(dF.to('N'), u_tT)*u.N
--> 686         dF_lateral = np.sqrt(np.sum((dF - dF_axial*u_tT)**2))
    687         dF_axial = np.abs(dF_axial)

The axial force should not be multiplied by the Newton unit. This gives N^2 and causes a crash on the following line indicated by the arrow.

====

json file:

{
  "missionLife": 2.0,
  "missionPortion": 0.5,
  "ohTime": 0.1,
  "keepStarCatalog": true,
  "minComp": 0.1,
  "pupilDiam": 4,
  "obscurFac": 0,
  "dMagLim": 26,
  "telescopeKeepout": 60,
  "attenuation": 0.65,
  "intCutoff": 100.0,
  
  "settlingTime": 1.0, 
  "thrust": 450,
  "slewIsp": 4160,
  "scMass": 6000,
  "dryMass": 3400,
  "coMass":5800,
  "occulterSep":55000,
  "skIsp": 220,
  "defburnPortion":0.05,
  
  "ppFact": 0.1,
  "prange":[0.2, 0.2],
  "erange":[0, 0.01],
  "eta": 0.1,
  "arange":[0.75, 1.77],
  "Rprange":[1.0, 1.0],
  "Mprange":[1, 1],
  "scaleOrbits": true,
  "constrainOrbits": true,
  "scienceInstruments": [
    {
      "name": "imagingEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000
    },
    {
      "name": "spectroEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000,
      "Rs": 50
    }
  ],
  "starlightSuppressionSystems": [
    {
      "name": "Occulter_testcase",
      "occulter": true,

      "lam": 550,
      "IWA": 0.027,
      "OWA": 5.127,
      "ohTime": 0.0,
      "throughput": 0.2,
      "BW": 0.10,
      "occ_trans": 1.0,
      "occulterDiameter": 26,
      "NocculterDistances" :1,
      "occulterDistances": [
        {
        "occulterDistance": 55000,
        "occulterRedEdge": 690,
        "occulterBlueEdge": 450
        }
      ],
      "core_thruput": 0.69, 
      "core_contrast": 1.7880e-10
    }
 ],
      "observingModes": [
    {
      "instName": "imagingEMCCD",
      "systName": "Occulter_testcase",
      "detection": 1,
      "SNR": 7
    },
    {
      "instName": "spectroEMCCD",
      "systName": "Occulter_testcase",
      "SNR": 5
    }
  ],
  "modules": {
    "PlanetPopulation": "EarthTwinHabZone1",
    "StarCatalog": "EXOCAT1",
    "OpticalSystem": "KasdinBraems",
    "ZodiacalLight": "Stark",
    "BackgroundSources": "GalaxiesFaintStars",
    "PlanetPhysicalModel": " ",
    "Observatory": "WFIRSTObservatoryL2",
    "TimeKeeping": " ",
    "PostProcessing": " ",
    "Completeness": "BrownCompleteness",
    "TargetList": " ",
    "SimulatedUniverse": " ",
    "SurveySimulation": " ",
    "SurveyEnsemble": " "
  }
}

det_status/char_status should always be ints

For some reason, in certain cases, empty det_status/char_status arrays in DRM get defined as floats instead of ints. They should always be uniformly integers.

CpCb attenuation

in prototype OpticalSystem method CpCb, lines 432 and 433 use the telescope attenuation in the calculation of throughput:
T = syst['throughput'](lam, WA) / inst['Ns']
* self.attenuation**2 # throughput

It is my understanding from the defintion of telescope attenuation in the ICD, that the attenuation in the equation above should not be squared.

Eccentric anomaly failed to converge

Dear all,

I encountered this "failure to converge" error while trying to run :

res = sim.SurveySimulation.run_sim()

using the "template_WFIRST_EarthTwinHabZone.json" scriptfile.

It seems to have failed after the last mission times listed below:

Current mission time:  0.0 d
Current mission time:  9.60003167231 d
Current mission time:  11.6313483646 d
Current mission time:  91.0376767603 d
Current mission time:  92.0376767603 d
Current mission time:  93.0376767603 d
Current mission time:  173.99003259 d
Current mission time:  175.073673146 d
Current mission time:  262.617593242 d
Current mission time:  342.664090476 d
Current mission time:  344.424390472 d
Current mission time:  345.424390472 d
Current mission time:  346.424390472 d
Current mission time:  347.424390472 d
Current mission time:  424.387741558 d

I looked at the while loop in eccanom.py where the convergence occurs and printed out the errors the piece calculates as part of the condition for convergence. It seems the eccentric anomaly converges after only a few iterations as shown below. The entire run of 200 iterations produces 1.1368e-13 by the end of the run:

err_arr = [1.0, 48.815465289295275, 8.0299584348608732, 2.0708116579178295, 0.085908077132557992, 0.00048370889385296323, 1.4405713955056854e-08, 1.1368683772161603e-13, 1.1368683772161603e-13, ..., 
1.1368683772161603e-13]

However, it doesn't ever reduce to the required tolerance level set by

tolerance = np.finfo(float).eps*4.01

which, in my case = 8.903988657493755e-16.

Any ideas? I've listed the full traceback below.

Exception                                 Traceback (most recent call last)
<ipython-input-112-667210cb35ae> in <module>()
----> 1 res = sim.SurveySimulation.run_sim()

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/SurveySimulation.pyc in run_sim(self)
    300 
    301             # acquire a new target star index
--> 302             sInd, DRM = self.next_target(sInd, DRM)
    303 
    304             # append result values to self.DRM

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/SurveySimulation.pyc in next_target(self, sInd, DRM)
    342         while not TK.mission_is_over():
    343             # find keepout Boolean values (kogood)
--> 344             Obs.keepout(TK.currentTimeAbs, TL, OS.telescopeKeepout)
    345             # if observable targets, pick one, else allocate time and try again
    346             if np.any(Obs.kogood):

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Observatory/WFIRSTObservatory.pyc in keepout(self, time, catalog, koangle)
     92                     self.solarSystem_body_position(time, 'Venus'), # Venus
     93                     self.solarSystem_body_position(time, 'Earth'), # Earth
---> 94                     self.solarSystem_body_position(time, 'Mars'), # Mars
     95                     self.solarSystem_body_position(time, 'Jupiter'), # Jupiter
     96                     self.solarSystem_body_position(time, 'Saturn'), # Saturn

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/Observatory.pyc in solarSystem_body_position(self, time, bodyname)
    307             return self.spk_body(time,bodyname)
    308         else:
--> 309             return self.keplerplanet(time,bodyname)
    310 
    311     def spk_body(self, time, bodyname):

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/Observatory.pyc in keplerplanet(self, time, bodyname)
    408         wp = w - O
    409         # Find eccentric anomaly
--> 410         E = eccanom(M,e)[0]
    411         # Find true anomaly
    412         nu = np.arctan2(np.sin(E) * np.sqrt(1 - e**2), np.cos(E) - e)

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/util/eccanom.pyc in eccanom(M, e)
     54 
     55     if numIter == maxIter:
---> 56         raise Exception("eccanom failed to converge.")
     57 
     58     return E

Exception: eccanom failed to converge.

de432s.bsp missing from install requirements/install

The sample code

import EXOSIMS,os.path
scriptfile = os.path.join(EXOSIMS.__path__[0],'Scripts','sampleScript_coron.json')
import EXOSIMS.MissionSim
sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

Crashes with the following error:

IOError: [Errno 2] No such file or directory: '/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/Observatory/de432s.bsp'

I was able to download the file from this legitimate looking website:
https://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/planets/

And then the problem went away after I put it in the above directory /EXOSIMS/Observatory/.

EarthTwin PlanetPop error

Hello everyone,

I am trying to test out the EarthTwinHabZone.json file with the MissionSim backbone. I changed the number of monte-carlo iterations to 1 for testing purposes. I received a TypeError when running the following:

scriptfile = os.path.join(EX.__path__[0],'Scripts','template_WFIRST_EarthTwinHabZone.json')
sim = msim.MissionSim(scriptfile)

MissionSim: Seed is:  53405376
Imported SurveyEnsemble prototype module
Imported SimulatedUniverse prototype module
Imported TargetList prototype module
Imported specific StarCatalog module EXOCAT1
Imported specific OpticalSystem module WFIRSTOpticalSystem
Imported specific ZodiacalLight module Stark
Imported BackgroundSources prototype module
Imported PostProcessing prototype module
Imported specific Completeness module BrownCompleteness
Imported specific PlanetPopulation module EarthTwinHabZone2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-45cd14cd17c2> in <module>()
----> 1 get_ipython().magic(u'time sim = msim.MissionSim(scriptfile)')

/Users/rpatel/Documents/envs/py27/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
   2161         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2162         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2163         return self.run_line_magic(magic_name, magic_arg_s)
   2164 
   2165     #-------------------------------------------------------------------------

/Users/rpatel/Documents/envs/py27/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
   2082                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2083             with self.builtin_trap:
-> 2084                 result = fn(*args,**kwargs)
   2085             return result
   2086 

<decorator-gen-60> in time(self, line, cell, local_ns)

/Users/rpatel/Documents/envs/py27/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194 
    195         if callable(arg):

/Users/rpatel/Documents/envs/py27/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in time(self, line, cell, local_ns)
   1175         else:
   1176             st = clock2()
-> 1177             exec(code, glob, local_ns)
   1178             end = clock2()
   1179             out = None

<timed exec> in <module>()

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/MissionSim.py in __init__(self, scriptfile, **specs)
    107         self.modules = {}
    108         self.modules['SimulatedUniverse'] = get_module(specs['modules']\
--> 109                 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
    110         self.modules['Observatory'] = get_module(specs['modules']\
    111                 ['Observatory'],'Observatory')(**specs)

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/SimulatedUniverse.pyc in __init__(self, eta, **specs)
     87 
     88         # import TargetList class
---> 89         self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
     90 
     91         # bring inherited class objects to top level of Simulated Universe

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in __init__(self, keepStarCatalog, **specs)
    110         self.BackgroundSources = get_module(specs['modules']['BackgroundSources'],'BackgroundSources')(**specs)
    111         self.PostProcessing = get_module(specs['modules']['PostProcessing'],'PostProcessing')(**specs)
--> 112         self.Completeness = get_module(specs['modules']['Completeness'],'Completeness')(**specs)
    113 
    114         # bring inherited class objects to top level of Simulated Universe

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Completeness/BrownCompleteness.pyc in __init__(self, **specs)
     49 
     50         # bring in inherited Completeness prototype __init__ values
---> 51         Completeness.__init__(self, **specs)
     52 
     53         # set up "ensemble visit photometric and obscurational completeness"

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/Prototypes/Completeness.pyc in __init__(self, minComp, Nplanets, **specs)
     25         # import PlanetPopulation class
     26         Pop = get_module(specs['modules']['PlanetPopulation'], 'PlanetPopulation')
---> 27         self.PlanetPopulation = Pop(**specs) # planet population object class
     28         self.PlanetPhysicalModel = self.PlanetPopulation.PlanetPhysicalModel
     29 

/Users/rpatel/Dropbox/Research/WFIRST/EXOSIMS/EXOSIMS/PlanetPopulation/EarthTwinHabZone2.py in __init__(self, eta, erange, constrainOrbits, **specs)
     22         specs['scaleOrbits'] = True
     23         specs['constrainOrbits'] = constrainOrbits
---> 24         PlanetPopulation.__init__(**specs)
     25 
     26     def gen_sma(self, n):

TypeError: unbound method __init__() must be called with PlanetPopulation instance as first argument (got nothing instead)

eccanom fails to converge

[[ I just noticed the first argument of eccanom is capped at 2 pi within the caller, making this issue moot. But it can't be deleted. ]]

Steps to reproduce, using a June 20 SVN checkout --

from EXOSIMS.util import eccanom
eccanom.eccanom(8,0.9) # works (as do smaller values of M)
eccanom.eccanom(9,0.9) # fails
eccanom.eccanom(40,0.9) # fails
eccanom.eccanom(60,0.9) # works (as do larger values of M)
eccanom.eccanom(30,0.8) # fails

Typical failure message is:

Exception: eccanom failed to converge. Final error of 7.105427e-15

Outdated function calls no longer exist

Several modules now contain function calls that are now outdated and break on execution.

In Observatory.py, function distForces, line 691:

r_targ = TL.starprop_equat(sInd, currentTime)[0F] - r_obs should be changed to r_targ = TL.starprop(sInd, currentTime)[0] - r_obs

In linearJScheduler.py, line 83:

r_ts = Obs.starprop(TL, sInds, TK.currentTimeAbs) should be changed to r_ts = TL.starprop(sInds, TK.currentTimeAbs)

There may be other instances of these problem in modules that I have not run yet.

Pickled files must be written as binary

This error occurs on Windows when data files saved to disk using pickle.dump are not explicitly written as binary. I noticed the issue with L2_halo_orbit_six_month_v2.p and Fortney_etal_2007_table4.p. After checking, the current files are not binary. This error went away when I replaced these files with older binary versions.

File "C:\Users\Daniel\Documents\Cornell\Research\Python_Code_base\EXOSIMS\MissionSim.py", line 13, in __init__
    ['Observatory'],'Observatory')(**specs)

File "C:\Users\Daniel\Documents\Cornell\Research\Python_Code_base\EXOSIMS\Observatory\WFIRSTObservatoryL2.py", line 43, in __init__
    halo = pickle.load( open( orbit_datapath, "rb" ) )

ImportError: No module named multiarray

Developer Guide Needed

We need to have a developer guide as part of the documentation.

SurveySimulation causing crash after latest pull

The following code causes a crash when using the appended json test case. Seed is 659054179. I originally thought this was due to the new linearJScheduler, but in fact occurs when using the same starshade test file that I used before successfully.

import EXOSIMS,os.path
scriptfile = os.path.join(EXOSIMS.__path__[0],'scripts','starshade_testcase.json')
import EXOSIMS.MissionSim
sim = EXOSIMS.MissionSim.MissionSim(scriptfile)
res = sim.SurveySimulation.run_sim()

The error is:

WARNING: ErfaWarning: ERFA function "utctai" yielded 1 of "dubious year (Note 3)" [astropy._erfa.core]
WARNING:astropy:ErfaWarning: ERFA function "utctai" yielded 1 of "dubious year (Note 3)"
Observation #1, current mission time: 1.0 d
Detected planet(s) [7] (1/1) at target #67/856
Charact. planet(s) [7] (1/1) at target #67/856
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-667210cb35ae> in <module>()
----> 1 res = sim.SurveySimulation.run_sim()

/Users/mbottom/Desktop/Other_projects/EXOSIMS/EXOSIMS/Prototypes/SurveySimulation.pyc in run_sim(self)
    230 
    231             # Acquire the NEXT TARGET star index and create DRM
--> 232             DRM, sInd, t_det = self.next_target(sInd, detMode)
    233 
    234             if sInd is not None:

/Users/mbottom/Desktop/Other_projects/EXOSIMS/EXOSIMS/Prototypes/SurveySimulation.pyc in next_target(self, old_sInd, mode)
    437             # find values related to slew time
    438             DRM['slew_time'] = slewTime[sInd].to('day').value
--> 439             DRM['slew_angle'] = sd[sInd].to('deg').value
    440             slew_mass_used = slewTime[sInd]*Obs.defburnPortion*Obs.flowRate
    441             DRM['slew_dV'] = (slewTime[sInd]*ao*Obs.defburnPortion).to('m/s').value

AttributeError: 'numpy.float64' object has no attribute 'to'

The appended code below is the json file I was using.

{
  "missionLife": 2.0,
  "missionPortion": 0.5,
  "ohTime": 0.1,
  "keepStarCatalog": true,
  "minComp": 0.1,
  "pupilDiam": 4,
  "obscurFac": 0,
  "dMagLim": 26,
  "telescopeKeepout": 60,
  "attenuation": 0.65,
  "intCutoff": 100.0,
  
  "settlingTime": 1.0, 
  "thrust": 450,
  "slewIsp": 4160,
  "scMass": 6000,
  "dryMass": 3400,
  "coMass":5800,
  "occulterSep":55000,
  "skIsp": 220,
  "defburnPortion":0.05,
  
  "ppFact": 0.1,
  "prange":[0.2, 0.2],
  "erange":[0, 0.01],
  "eta": 0.1,
  "arange":[0.75, 1.77],
  "Rprange":[1.0, 1.0],
  "Mprange":[1, 1],
  "scaleOrbits": true,
  "constrainOrbits": true,
  "scienceInstruments": [
    {
      "name": "imagingEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000
    },
    {
      "name": "spectroEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000,
      "Rs": 50
    }
  ],
  "starlightSuppressionSystems": [
    {
      "name": "Occulter_testcase",
      "occulter": true,

      "lam": 550,
      "IWA": 0.027,
      "OWA": 5.127,
      "ohTime": 0.0,
      "throughput": 0.2,
      "BW": 0.10,
      "occ_trans": 1.0,
      "occulterDiameter": 26,
      "NocculterDistances" :1,
      "occulterDistances": [
        {
        "occulterDistance": 55000,
        "occulterRedEdge": 690,
        "occulterBlueEdge": 450
        }
      ],
      "core_thruput": 0.69, 
      "core_contrast": 1.7880e-10
    }
 ],
      "observingModes": [
    {
      "instName": "imagingEMCCD",
      "systName": "Occulter_testcase",
      "detection": 1,
      "SNR": 7
    },
    {
      "instName": "spectroEMCCD",
      "systName": "Occulter_testcase",
      "SNR": 5
    }
  ],
  "modules": {
    "PlanetPopulation": "EarthTwinHabZone1",
    "StarCatalog": "EXOCAT1",
    "OpticalSystem": "KasdinBraems",
    "ZodiacalLight": "Stark",
    "BackgroundSources": "GalaxiesFaintStars",
    "PlanetPhysicalModel": " ",
    "Observatory": "WFIRSTObservatoryL2",
    "TimeKeeping": " ",
    "PostProcessing": " ",
    "Completeness": "BrownCompleteness",
    "TargetList": " ",
    "SimulatedUniverse": " ",
    "SurveySimulation": " ",
    "SurveyEnsemble": " "
  }
}

StarCatalog should have option to fill in missing color magnitudes from mean color-color fits

This is particularly important for Kmags in EXOCAT. Can use values from Mamjek et al.:
http://www.pas.rochester.edu/~emamajek/EEM_dwarf_UBVIJHK_colors_Teff.txt

BrownCompleteness gen_updates array ValueError

I used scriptfile template_WFIRST_EarthTwinHabZone.json, but with OpticalSystem set to Nemati.
The MC completeness calculation was performed, then received an error upon Beginning completeness update. here is the Traceback:

Monte Carlo completeness calculations finished
2D completeness array stored in u'EXOSIMS/Completeness/EarthTwinHabZone2.comp'
/Users/rhonda/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/astropy/units/quantity.py:822: RuntimeWarning: invalid value encountered in true_divide
return super(Quantity, self).truediv(other)
EXOSIMS/Completeness/BrownCompleteness.py:106: RuntimeWarning: divide by zero encountered in log10
dMag = dMag - 2.5_np.log10(targlist.L)
/Users/rhonda/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/function_base.py:1769: RuntimeWarning: invalid value encountered in ? (vectorized)
outputs = ufunc(_inputs)

Beginning completeness update calculations

ValueError Traceback (most recent call last)
in ()
----> 1 sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/MissionSim.pyc in init(self, scriptfile, **specs)
107 self.modules = {}
108 self.modules['SimulatedUniverse'] = get_module(specs['modules']
--> 109 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
110 self.modules['Observatory'] = get_module(specs['modules']
111 ['Observatory'],'Observatory')(**specs)

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Prototypes/SimulatedUniverse.pyc in init(self, eta, **specs)
87
88 # import TargetList class
---> 89 self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
90
91 # bring inherited class objects to top level of Simulated Universe

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in init(self, keepStarCatalog, minComp, **specs)
139
140 # generate any completeness update data needed
--> 141 self.Completeness.gen_update(self)
142
143 # have target list, no need for catalog now

/Users/rhonda/exosims.2.0/EXOSIMS/EXOSIMS/Completeness/BrownCompleteness.pyc in gen_update(self, targlist)
147 smax = (np.tan(targlist.OpticalSystem.OWA)targlist.dist).to('AU')
148 else:
--> 149 smax = np.array([np.max(self.PlanetPopulation.arange)(1.+np.max(self.PlanetPopulation.erange))]*targlist.nStars)
150 # fill dynamic completeness values
151 for sInd in xrange(targlist.nStars):

ValueError: setting an array element with a sequence.

testdrive bug reporting

I am trying the bug reporting tool for the first time.

KnownRVPlanets semi-major axis error issue

Around line 93 of KnownRVPlanets.py, we're setting up SMA and SMA errors, like so:

        #save semi-major axes
        self.sma = data['pl_orbsmax'].data*u.AU
        mask = data['pl_orbsmax'].mask
        Ms = data['st_mass'].data[mask]*const.M_sun # units of kg
        T = data['pl_orbper'].data[mask]*u.d
        self.sma[mask] = ((const.G*Ms*T**2 / (4*np.pi**2))**(1/3.)).to('AU')
        assert np.all(~np.isnan(self.sma)), 'sma has nan value(s)'
        #sma errors
        self.smaerr = data['pl_orbsmaxerr1'].data*u.AU
        mask = data['pl_orbsmaxerr1'].mask
        Ms = data['st_mass'].data[mask]*const.M_sun # units of kg
        T = data['pl_orbpererr1'].data[mask]*u.d
        self.smaerr[mask] = ((const.G*Ms*T**2 / (4*np.pi**2))**(1/3.)).to('AU')
        self.smaerr[np.isnan(self.smaerr)] = np.nanmean(self.smaerr)

The intent of the first 7 lines is to use Kepler to fill in the SMA from T. Seems OK.

The intent of the last 7 lines is (I think) to do the same with the SMA error from T error. But, the T error is given as a delta value off a nominal value, and the Kepler formula as written does not seem to be handling this. You could compute

sma_err = Kepler(T+T_error) - Kepler(T)

but this is doing

sma_err = Kepler(T_error)

Definitions of "pl_orbsmaxerr1" are at IPAC [http://exoplanetarchive.ipac.caltech.edu/docs/API_exoplanet_columns.html]

Quickstart's sampleScript_coron.json missing

The quickstart guide in documentation/quickstart.rst calls Scripts/sampleScript_coron.json but the sample file is missing from the directory, while there are several json templates which seem to run fine. Should the sample just be replaced with one of the other templates in the guide?

scipy interpolate error in optical system prototype

Just got the most bizarre error ever. Script is a direct copy of template_WFIRST_Keplerlike_curves.json. Investigating.

In [3]: sim = EXOSIMS.MissionSim.MissionSim('wfirstscript.json')
MissionSim: Seed is:  446099828
Imported SurveyEnsemble (prototype module) from EXOSIMS.Prototypes.SurveyEnsemble
Imported KeplerLikeUniverse (specific module) from EXOSIMS.SimulatedUniverse.KeplerLikeUniverse
Imported TargetList (prototype module) from EXOSIMS.Prototypes.TargetList
Imported EXOCAT1 (specific module) from EXOSIMS.StarCatalog.EXOCAT1
Imported Nemati (specific module) from EXOSIMS.OpticalSystem.Nemati
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-2adbea44913f> in <module>()
----> 1 sim = EXOSIMS.MissionSim.MissionSim('wfirstscript.json')

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/MissionSim.pyc in __init__(self, scriptfile, **specs)
    110         self.modules = {}
    111         self.modules['SimulatedUniverse'] = get_module(specs['modules']\
--> 112                 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
    113         self.modules['Observatory'] = get_module(specs['modules']\
    114                 ['Observatory'],'Observatory')(**specs)

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/SimulatedUniverse/KeplerLikeUniverse.pyc in __init__(self, **specs)
     21     def __init__(self, **specs):
     22 
---> 23         SimulatedUniverse.__init__(self, **specs)
     24 
     25     def gen_physical_properties(self,**specs):

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/Prototypes/SimulatedUniverse.pyc in __init__(self, **specs)
     90 
     91         # import TargetList class
---> 92         self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
     93 
     94         # bring inherited class objects to top level of Simulated Universe

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in __init__(self, keepStarCatalog, minComp, **specs)
     74         # get desired module names (specific or prototype) and instantiate objects
     75         self.StarCatalog = get_module(specs['modules']['StarCatalog'],'StarCatalog')(**specs)
---> 76         self.OpticalSystem = get_module(specs['modules']['OpticalSystem'],'OpticalSystem')(**specs)
     77         self.ZodiacalLight = get_module(specs['modules']['ZodiacalLight'],'ZodiacalLight')(**specs)
     78         self.PostProcessing = get_module(specs['modules']['PostProcessing'],'PostProcessing')(**specs)

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/OpticalSystem/Nemati.pyc in __init__(self, **specs)
     21     def __init__(self, **specs):
     22 
---> 23         OpticalSystem.__init__(self, **specs)
     24 
     25     def calc_intTime(self, TL, sInds, fZ, fEZ, dMag, WA, mode):

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/Prototypes/OpticalSystem.pyc in __init__(self, obscurFac, shapeFac, pupilDiam, attenuation, intCutoff, Ndark, scienceInstruments, QE, FoV, pixelNumber, pixelSize, sread, idark, CIC, texp, ENF, Rs, starlightSuppressionSystems, lam, BW, occ_trans, core_thruput, core_contrast, core_platescale, PSF, samp, ohTime, observingModes, SNR, timeMultiplier, IWA, OWA, dMagLim, WAint, dMagint, **specs)
    284 
    285             # Get coronagraph input parameters
--> 286             syst = self.get_coro_param(syst, 'occ_trans')
    287             syst = self.get_coro_param(syst, 'core_thruput')
    288             syst = self.get_coro_param(syst, 'core_contrast', fill=1.)

/Users/ds264/Documents/gitrepos/EXOSIMS/EXOSIMS/Prototypes/OpticalSystem.pyc in get_coro_param(self, syst, param_name, fill)
    460             # parameter values outside of WA
    461             Dinterp = scipy.interpolate.interp1d(WA, D, kind='cubic',\
--> 462                     fill_value=fill, bounds_error=False)
    463             syst[param_name] = lambda l, s: np.array(Dinterp(s.to('arcsec').value),ndmin=1)
    464             # update IWA and OWA

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/interpolate/interpolate.pyc in __init__(***failed resolving arguments***)
    494 
    495             self._spline = make_interp_spline(xx, yy, k=order,
--> 496                                               check_finite=False)
    497             if rewrite_nan:
    498                 self._call = self.__class__._call_nan_spline

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/interpolate/_bsplines.pyc in make_interp_spline(x, y, k, t, bc_type, axis, check_finite)
    700     kl = ku = k
    701     ab = np.zeros((2*kl + ku + 1, nt), dtype=np.float_, order='F')
--> 702     _bspl._colloc(x, t, k, ab, offset=nleft)
    703     if nleft > 0:
    704         _bspl._handle_lhs_derivatives(t, k, x[0], ab, kl, ku, deriv_l_ords)

scipy/interpolate/_bspl.pyx in scipy.interpolate._bspl._colloc (scipy/interpolate/_bspl.c:4617)()

ValueError: Big-endian buffer not supported on little-endian compiler

Sample code crashes on Forecaster

Trying to run the sample code from the repository:

import EXOSIMS,os.path
scriptfile = os.path.join(EXOSIMS.__path__[0],'Scripts','sampleScript_coron.json')
import EXOSIMS.MissionSim
sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

Crashes with the default 'sampleScript_coron.json' with the following error.

Error: No module on paths: "EXOSIMS.PlanetPhysicalModel.Forecaster", "EXOSIMS.Prototypes.Forecaster".

ValueError Traceback (most recent call last)
in ()
2 scriptfile = os.path.join(EXOSIMS.path[0],'Scripts','sampleScript_coron.json')
3 import EXOSIMS.MissionSim
----> 4 sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/MissionSim.pyc in init(self, scriptfile, **specs)
109 self.modules = {}
110 self.modules['SimulatedUniverse'] = get_module(specs['modules']
--> 111 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
112 self.modules['Observatory'] = get_module(specs['modules']
113 ['Observatory'],'Observatory')(**specs)

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/SimulatedUniverse/KnownRVPlanetsUniverse.pyc in init(self, **specs)
18 def init(self,**specs):
19
---> 20 SimulatedUniverse.init(self, **specs)
21
22 def gen_physical_properties(self, missionStart=60634., **specs):

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/Prototypes/SimulatedUniverse.pyc in init(self, **specs)
90
91 # import TargetList class
---> 92 self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
93
94 # bring inherited class objects to top level of Simulated Universe

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/TargetList/KnownRVPlanetsTargetList.pyc in init(self, **specs)
45 'rv': 'st_radv'}
46
---> 47 TargetList.init(self, **specs)
48
49 def populate_target_list(self, **specs):

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/Prototypes/TargetList.pyc in init(self, keepStarCatalog, minComp, **specs)
78 self.ZodiacalLight = get_module(specs['modules']['ZodiacalLight'],'ZodiacalLight')(**specs)
79 self.PostProcessing = get_module(specs['modules']['PostProcessing'],'PostProcessing')(**specs)
---> 80 self.Completeness = get_module(specs['modules']['Completeness'],'Completeness')(**specs)
81
82 # bring inherited class objects to top level of Simulated Universe

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/Prototypes/Completeness.pyc in init(self, **specs)
20 def init(self, **specs):
21 # import PlanetPopulation class
---> 22 Pop = get_module(specs['modules']['PlanetPopulation'],'PlanetPopulation')(**specs)
23 self.PlanetPopulation = Pop # planet population object class
24 self.PlanetPhysicalModel = Pop.PlanetPhysicalModel

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/PlanetPopulation/KnownRVPlanets.pyc in init(self, smaknee, esigma, rvplanetfilepath, **specs)
57 specs['smaknee'] = float(smaknee)
58 specs['esigma'] = float(esigma)
---> 59 KeplerLike1.init(self, **specs)
60
61 #default file is ipac_2016-05-15

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/PlanetPopulation/KeplerLike1.pyc in init(self, smaknee, esigma, **specs)
46 specs['prange'] = [0.083,0.882]
47 specs['Rprange'] = [1,22.6]
---> 48 PlanetPopulation.init(self, **specs)
49
50 assert (smaknee >= self.arange[0].to('AU').value) and \

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/Prototypes/PlanetPopulation.pyc in init(self, arange, erange, Irange, Orange, wrange, prange, Rprange, Mprange, scaleOrbits, constrainOrbits, eta, **specs)
95 # import PlanetPhysicalModel
96 self.PlanetPhysicalModel = get_module(specs['modules']['PlanetPhysicalModel'],
---> 97 'PlanetPhysicalModel')(**specs)
98
99 def checkranges(self, var, name):

/Users/mbottom/Desktop/Other_projects/EXOSIMS/code/EXOSIMS/util/get_module.pyc in get_module(name, folder)
232 full_module = get_module_in_package(name, folder)
233 if not full_module:
--> 234 raise ValueError('Could not import module "%s" (path issue?).' % name)
235 source = full_module.name
236 note = 'prototype' if 'EXOSIMS.Prototypes' in full_module.name else 'specific'

ValueError: Could not import module "Forecaster" (path issue?).

Inspection of the Forecaster.py file revealed that it requires the following extra files:

forecaster_fitting_parameters.h5
forecaster_func.py

which I downloaded from the jjcheng github account and renamed as shown, then added to the directory PlanetPhysicalModel. The code still crashed with the same error.

This error can be avoided by replacing "Forecaster" with " " in the sampleScript_coron.json file, but then it does not use forecaster anymore.

eta

In the prototype SimulatedUniverse, line 127 uses eta to select the fraction of stars that will have planets:

Map planets to target stars

probs = np.random.uniform(size=TL.nStars)
self.plan2star = np.where(probs > self.eta)[0]

when I set eta=1, there are no planets.
When i set eta=0, all the stars have planets.

The traditional definition of eta is the fraction of stars with planets, so I would expect the reverse behavior for planet selection given eta. I recommend the following correction to line 127 to flip the inequality:
self.plan2star = np.where(probs < self.eta)[0]

Aggressive dMagInts lead to hugely wasteful sims.

When I was setting up runs this weekend, I found that if dMagint (or dMaglim) were set to higher than 23 (say 23.5), the simulation would run for a very long time, but would not end up observing any targets at all. I'm guess this is because the time calculated ended up being higher than the cutoff for all targets. In any case, this is something we should investigate, and, if necessary, provided some useful user warnings for. It seems like if you set up an impossible set of conditions for any observations to happen, the code should be smart enough to catch that and tell you. (Lowest priority).

Clerical error in EXOSIMS/EXOSIMS/BackgroundSources/stellar_cnts.txt

The last entry on row 2 is 5.572 but the corresponding value of Table 19.11 in Allan's Astrophysical Quantities 4th edition is 5.722

PlanetPhysicalModel looks for non-existent file

I followed the quickstart instructions at https://github.com/dsavransky/EXOSIMS/blob/master/documentation/quickstart.rst, using the example script 'sampleScript_coron.json'

When I instantiate the MissionSim via
sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

The PlanetPhysicalModel module has an error trying to locate a non-existent file, planetphysicalmodel/fitting_parameters.h5.

I pased my error trace below.

/Users/nzimmerm/WFIRST_tech/EXOSIMS/EXOSIMS/Prototypes/PlanetPopulation.py in init(self, arange, erange, Irange, Orange, wrange, prange, Rprange, Mprange, scaleOrbits, constrainOrbits, eta, **specs)
99 # import PlanetPhysicalModel
100 self.PlanetPhysicalModel = get_module(specs['modules']['PlanetPhysicalModel'],
--> 101 'PlanetPhysicalModel')(**specs)
102
103 def checkranges(self, var, name):

/Users/nzimmerm/WFIRST_tech/EXOSIMS/EXOSIMS/PlanetPhysicalModel/Forecaster.py in init(self, n_pop, **specs)
30 filename = 'fitting_parameters.h5'
31 parampath = os.path.join(classpath, filename)
---> 32 h5 = h5py.File(parampath, 'r')
33 self.all_hyper = h5['hyper_posterior'][:]
34 h5.close()

/Users/nzimmerm/anaconda3/envs/py27/lib/python2.7/site-packages/h5py/_hl/files.pyc in init(self, name, mode, driver, libver, userblock_size, swmr, **kwds)
270
271 fapl = make_fapl(driver, libver, **kwds)
--> 272 fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
273
274 if swmr_support:

/Users/nzimmerm/anaconda3/envs/py27/lib/python2.7/site-packages/h5py/_hl/files.pyc in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
90 if swmr and swmr_support:
91 flags |= h5f.ACC_SWMR_READ
---> 92 fid = h5f.open(name, flags, fapl=fapl)
93 elif mode == 'r+':
94 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/Users/ilan/minonda/conda-bld/work/h5py/_objects.c:2696)()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper (/Users/ilan/minonda/conda-bld/work/h5py/_objects.c:2654)()

h5py/h5f.pyx in h5py.h5f.open (/Users/ilan/minonda/conda-bld/work/h5py/h5f.c:1942)()

IOError: Unable to open file (Unable to open file: name = '/users/nzimmerm/wfirst_tech/exosims/exosims/planetphysicalmodel/fitting_parameters.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

linearJScheduler slew distance by star index is incorrect

In the linearJScheduler module, if the OpticalSystem used has an occulter, the weighting by slew distance to neighboring target stars is incorrect. Cutting out all other weighting factors, each next target star should be the one closest to the current star; however, the traversal instead looks like so (where the stars in red are those that are visited over a 1-year period):

From narrowing down the source of the error, I believe there is an indexing problem where the correct star weights are being attributed to the incorrect star index.

The source of the problem should be in one of these two code blocks:

if OS.haveOcculter:
    combs = np.array([np.array(x) for x in itertools.combinations(range(nStars),2)])
    r_ts = Obs.starprop(TL, sInds, TK.currentTimeAbs)
    u_ts = (r_ts.value.T/np.linalg.norm(r_ts,axis=1)).T # good to here
    angdists = np.arccos(np.clip(np.dot(u_ts,u_ts.T)[combs[:,0],combs[:,1]],-1,1))
    A[np.tril(np.ones((nStars,nStars),dtype=bool),-1)] = angdists
    A = self.coeffs[0]*(A+A.T)/np.pi

lc = nStars-1
step1 = np.tile(A[sInds==old_sInd,:],(lc,1)).flatten('F')
step2 = A[np.array(np.ones((nStars,nStars)) - np.eye(nStars),dtype=bool)]
tmp = np.argmin(step1+step2)
sInd = sInds[int(np.ceil(tmp/float(lc)))]

Note:
I decided to rewrite some of the old code to create a depth=1 search to narrow down the source of the error. I replaced the above code blocks with the following:

slews = np.zeros(nStars)
r_curr = Obs.starprop(TL, old_sInd, TK.currentTimeAbs)
u_curr = (r_curr.value/np.linalg.norm(r_curr,axis=1))

r_seq = Obs.starprop(TL, sInds, TK.currentTimeAbs)
u_seq = (r_seq.value.T/np.linalg.norm(r_seq,axis=1)).T
angdists = np.arccos(np.dot(u_seq,u_curr.T))
slews[np.ones((nStars),dtype=bool)] = angdists
slews = slews/np.pi
slews = np.where(slews==0,np.Inf,slews)

sInd = sInds[np.argmin(slews)]

This produced a result that seemed to make more sense, and led me to believe that the problem is caused by an error when the star weights are used to choose the next target by sInd.The results of the above code can be seen below:

Replace sma lambda function with method in KeplerLike1

Methods should never be overwritten with attributes to prevent downstream inheritance breaking.

Default WAint shouldn't be the IWA

The default WAint is set to the IWA, which is way too aggressive. It should do what the maxtime calculation does, which sets it to halfway between the IWA/OWA, or 2*IWA if the OWA is infinite.

L2 Orbit: earth excursion too great

A video of the keepout zones for the star shade shows that the largest separation between the earth and the sun is ~80 degrees. This is too large. Doing the geometry by hand, using the semimajor axis for the WFIRST L2 orbit file in EXOSIMS/Observatory, the largest separation should be <29 degrees.

Potentially, the issue could be the position of the L2 node. In the frame below of the video, the distance between the earth and observatory is 0.061 AU; the L2 point is ~0.01 AU from the earth.

I also checked how much the moon separates from the earth, which should be < 12 degrees, and this looks correct in the video, to the coarse resolution provided by the 10 degree ticks on the axes.

DRM output cleanup

Currently DRM saves the full inst & syst dicts on every iteration, which is wasteful. We should just save the inst/syst names to DRM - the dictionaries can either be saved once separately, or recreated via the outspec.

Encountered prematurely ended simulation error

When running a simulation using the script file below, I encountered the following output:

OB1: survey beginning.
  Observation #1, target #287/948 with 0 planet(s), mission time: 0.0 d
OB2: previous block was 1.0 d long, advancing 0.0 d.
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/units/quantity.py:924: RuntimeWarning: invalid value encountered in multiply
  return super(Quantity, self).__mul__(other)
Mission complete: no more time available
Simulation finishing OK. Results stored in SurveySimulation.DRM

I reverted my local EXOSIMS repo to commit 6beee01, as running this simulation had worked previously, and noted that the problem disappeared and the RuntimeWarning above did not appear. I suspect that one of the more recent commits is the source of the error.

{
  "missionLife": 1.0,
  "missionPortion": 1.0,
  "keepStarCatalog": true,
  "minComp": 0.1,
  "pupilDiam": 4,
  "obscurFac": 0,
  "dMagLim": 26,
  "telescopeKeepout": 45,
  "attenuation": 0.65,
  "intCutoff": 50.0,
 
  "settlingTime": 1.0, 
  "thrust": 450,
  "slewIsp": 4160,
  "scMass": 6000,
  "dryMass": 3400,
  "coMass":5800,
  "occulterSep":55000,
  "skIsp": 220,
  "defburnPortion":0.05,
  
  "ppFact": 0.1,
  "prange":[0.2, 0.2],
  "erange":[0, 0.01],
  "eta": 1.0,
  "arange":[0.75, 1.77],
  "Rprange":[1.0, 1.0],
  "Mprange":[1, 1],
  "scaleOrbits": true,
  "constrainOrbits": true,
  "scienceInstruments": [
    {
      "name": "imagingEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000
    },
    {
      "name": "spectroEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000,
      "Rs": 50
    }
  ],
  "starlightSuppressionSystems": [
    {
      "name": "Occulter_testcase",
      "occulter": true,
      "lam": 550,
      "IWA": 0.027,
      "OWA": 5.127,
      "ohTime": 0.0,
      "throughput": 0.2,
      "BW": 0.10,
      "occ_trans": 1.0,
      "occulterDiameter": 26,
      "NocculterDistances" :1,
      "occulterDistances": [
        {
        "occulterDistance": 55000,
        "occulterRedEdge": 690,
        "occulterBlueEdge": 450
        }
      ],
      "core_thruput": 0.69, 
      "core_contrast": 1e-10
    }
 ],
      "observingModes": [
    {
      "instName": "imagingEMCCD",
      "systName": "Occulter_testcase",
      "detection": 1,
      "SNR": 7
    },
    {
      "instName": "spectroEMCCD",
      "systName": "Occulter_testcase",
      "SNR": 5
    }
  ],
  "modules": {
    "PlanetPopulation": "EarthTwinHabZone1",
    "StarCatalog": "EXOCAT1",
    "OpticalSystem": "KasdinBraems",
    "ZodiacalLight": "Stark",
    "BackgroundSources": "GalaxiesFaintStars",
    "PlanetPhysicalModel": " ",
    "Observatory": "WFIRSTObservatoryL2",
    "TimeKeeping": " ",
    "PostProcessing": " ",
    "Completeness": "GarrettCompleteness",
    "TargetList": " ",
    "SimulatedUniverse": " ",
    "SurveySimulation": "linearJScheduler",
    "SurveyEnsemble": " "
  }
}

calc_intTime SNR

Nemati.py line 67 in method calc_intTime : SNR = PP.SNchar
is identical to Nemati.py line118 in method calc_charTime.
The occurence on line 67 should be changed to the SNR for detection.

starprop is overwritten by lambda function in TargetList prototype

Line 126 - in general we should try to avoid overwriting methods with attributes.

BrownCompleteness can't reload cached file

I performed a newclone of EXOSIMS yesterday. It crashes when using BrownCompleteness.
I intend to use BrownCompleteness until unit testing is performed on GarrettCompleteness.

Here is the screen output and error message:

MissionSim: Seed is: 686326949
Imported SurveyEnsemble (prototype module) from EXOSIMS.Prototypes.SurveyEnsemble
Imported SimulatedUniverse (prototype module) from EXOSIMS.Prototypes.SimulatedUniverse
Imported TargetList (prototype module) from EXOSIMS.Prototypes.TargetList
Imported EXOCAT1 (specific module) from EXOSIMS.StarCatalog.EXOCAT1
Imported KasdinBraems (specific module) from EXOSIMS.OpticalSystem.KasdinBraems
Imported Stark (specific module) from EXOSIMS.ZodiacalLight.Stark
Imported PostProcessing (prototype module) from EXOSIMS.Prototypes.PostProcessing
Imported GalaxiesFaintStars (specific module) from EXOSIMS.BackgroundSources.GalaxiesFaintStars
Imported BrownCompleteness (specific module) from EXOSIMS.Completeness.BrownCompleteness
Imported EarthTwinHabZone1 (specific module) from EXOSIMS.PlanetPopulation.EarthTwinHabZone1
Imported PlanetPhysicalModel (prototype module) from EXOSIMS.Prototypes.PlanetPhysicalModel
Loading cached completeness file from "/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Completeness/EarthTwinHabZone10cdf3ce3f4d97068fea6468ef03add60.comp".

ValueError Traceback (most recent call last)
in ()
----> 1 sim = EXOSIMS.MissionSim.MissionSim(scriptfile)

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/MissionSim.pyc in init(self, scriptfile, **specs)
110 self.modules = {}
111 self.modules['SimulatedUniverse'] = get_module(specs['modules']
--> 112 ['SimulatedUniverse'],'SimulatedUniverse')(**specs)
113 self.modules['Observatory'] = get_module(specs['modules']
114 ['Observatory'],'Observatory')(**specs)

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Prototypes/SimulatedUniverse.pyc in init(self, **specs)
90
91 # import TargetList class
---> 92 self.TargetList = get_module(specs['modules']['TargetList'],'TargetList')(**specs)
93
94 # bring inherited class objects to top level of Simulated Universe

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in init(self, keepStarCatalog, minComp, **specs)
93
94 # now populate and filter the list
---> 95 self.populate_target_list(**specs)
96 self.filter_target_list(**specs)
97

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Prototypes/TargetList.pyc in populate_target_list(self, **specs)
148 self.nan_filter()
149 # populate completeness values
--> 150 self.comp0 = Comp.target_completeness(self)
151 # populate minimum integration time values, for minimum dMag in detection mode
152 mode = filter(lambda mode: mode['detectionMode'] == True, OS.observingModes)[0]

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Completeness/BrownCompleteness.pyc in target_completeness(self, TL)
100 # path to 2D completeness pdf array for interpolation
101 Cpath = os.path.join(self.classpath, self.filename+'.comp')
--> 102 Cpdf, xedges2, yedges2 = self.genC(Cpath, nplan, xedges, yedges, steps)
103
104 EVPOCpdf = interpolate.RectBivariateSpline(xedges, yedges, Cpdf.T)

/proj/exep/rhonda/exosimsgit2017/EXOSIMS/EXOSIMS/Completeness/BrownCompleteness.pyc in genC(self, Cpath, nplan, xedges, yedges, steps)
286 if os.path.exists(Cpath):
287 print 'Loading cached completeness file from "%s".' % Cpath
--> 288 H = pickle.load(open(Cpath, 'rb'))
289 print 'Completeness loaded from cache.'
290 #h, xedges, yedges = self.hist(nplan, xedges, yedges)

ValueError: buffer size does not match array size

Need to reproduce builddocs.sh functionality for Windows

It should be straight-forward to replicated builddocs.sh as a .bat file to do the same steps on Windows. I don't have a properly configured windows box handy for testing, though, so leaving this as an open feature request.

Need to add json script values to hash for cached completeness files

Future implementations of PlanetPopulation may have additional important attributes which should be captured in the hash for the cached completeness files.

Silent Error in SurveySimulation: observation_detection

Around line 632 in SurveySimulation, there is a line of code that fails silently when it is called:

WA = np.random.uniform(mode['IWA'].to('mas'), np.minimum(mode['OWA'], \
                    np.arctan(max(PPop.arange)/TL.dist[sInd])).to('mas'))

np.random.uniform() cannot handle astropy quantities, and will return a -1 and an error when it is attempted. See the test case below as an example:

x = 1*u.m
y = 5*u.m
np.random.uniform(x,y)

-1.0

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/units/quantity.py", line 1073, in __float__
    raise TypeError('Only dimensionless scalar quantities can be '
TypeError: Only dimensionless scalar quantities can be converted to Python scalars

While an error is raised, the simulation will continue to run using WA = -1, causing further problems down the line.

One simple solution is the following:

WA = np.random.uniform(mode['IWA'].to('mas').value, np.minimum(mode['OWA'], \
                    np.arctan(max(PPop.arange)/TL.dist[sInd])).to('mas').value)

Which remedies the problem without any further changes necessary.

dMagInt and WAint full arrays should go into outspec

When I fixed the outspec bug in SurveySimulation introduced by adding the dMagInt and WAint vars, I basically made it so that those are never added to the outspec. This should be changed by adding some better logic: if all elements of these are the same, then it should just add the first value for each, if they're different, then it should add the full array.

genOutSpec not recognizing git repository

Getting the "Not a valid Github or SVN revision." error when running from current git master branch.

Update e2etests with all relevant combinations of modules + add characterization test

in Prototype OpticalSystem calc_maxintTime doesn't check for OWA < 2*IWA

Currently WA is set to 2.self.IWA, but this can produce a silent (non-terminating) error when OWA < 2IWA. Should be an explicit check for this condition, in which case WA should be set to (IWA+OWA)/2.

Got error messages while running EXOSIMS under Python 2.7.12

Got following error messages (ValueError: setting an array element with a sequence in /EXOSIMS/EXOSIMS/Prototypes/Observatory.py) while running EXOSIMS under Python 2.7.12.

However, same script ran fine under Python 2.7.11.

Crashes with starshade simulation

I'm trying to run a starshade simulation case but running into a fatal error in the function starprop in Observatory.py. The json file I am using is appended below, and the mission seed is 881634040

When trying to execute the following code:

In [1]: import EXOSIMS,os.path
   ...: scriptfile = os.path.join(EXOSIMS.__path__[0],'scripts','starshade_testcase.json')
   ...: import EXOSIMS.MissionSim
   ...: sim = EXOSIMS.MissionSim.MissionSim(scriptfile)
In [2]: res = sim.SurveySimulation.run_sim()

The crash occurs on return r_star.to('km').reshape(currentTime.size,3) in Observatory.py.

There is an array mismatch error after the fifth iteration _* ValueError: total size of new array must be unchanged_**. In particular, we have the following output for the first iterations:

r_star size 2568
currentTime size 856
r_star size 1314
currentTime size 438
Current mission time: 1.0 d
r_star size 3
currentTime size 1
r_star size 3
currentTime size 1
r_star size 3
currentTime size 1
> r_star size 2568
currentTime size 1

The bold text shows the error. The r_star should be 3 times as large as the currentTime size for the vectors to match, but instead it is 2568 times as large.

json file used:

{
  "missionLife": 2.0,
  "missionPortion": 0.5,
  "ohTime": 0.1,
  "keepStarCatalog": true,
  "minComp": 0.1,
  "pupilDiam": 4,
  "obscurFac": 0,
  "dMagLim": 26,
  "telescopeKeepout": 60,
  "attenuation": 0.65,
  "intCutoff": 100.0,
  
  "settlingTime": 1.0, 
  "thrust": 450,
  "slewIsp": 4160,
  "scMass": 6000,
  "dryMass": 3400,
  "coMass":5800,
  "occulterSep":55000,
  "skIsp": 220,
  "defburnPortion":0.05,
  
  "ppFact": 0.1,
  "prange":[0.2, 0.2],
  "erange":[0, 0.01],
  "eta": 0.1,
  "arange":[0.75, 1.77],
  "Rprange":[1.0, 1.0],
  "Mprange":[1, 1],
  "scaleOrbits": true,
  "constrainOrbits": true,
  "scienceInstruments": [
    {
      "name": "imagingEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000
    },
    {
      "name": "spectroEMCCD",
      "focal": 120,
      "pitch": 5e-6,
      "QE": 0.999,
      "idark": 0.0,
      "CIC": 0,
      "sread": 0.0,
      "ENF": 1,
      "texp": 1000,
      "Rs": 50
    }
  ],
  "starlightSuppressionSystems": [
    {
      "name": "Occulter_testcase",
      "occulter": true,

      "lam": 550,
      "IWA": 0.027,
      "OWA": 5.127,
      "ohTime": 0.0,
      "throughput": 0.2,
      "BW": 0.10,
      "occ_trans": 1.0,
      "occulterDiameter": 26,
      "NocculterDistances" :1,
      "occulterDistances": [
        {
        "occulterDistance": 55000,
        "occulterRedEdge": 690,
        "occulterBlueEdge": 450
        }
      ],
      "core_thruput": 0.69, 
      "core_contrast": 1.7880e-10
    }
 ],
      "observingModes": [
    {
      "instName": "imagingEMCCD",
      "systName": "Occulter_testcase",
      "detection": 1,
      "SNR": 7
    },
    {
      "instName": "spectroEMCCD",
      "systName": "Occulter_testcase",
      "SNR": 5
    }
  ],
  "modules": {
    "PlanetPopulation": "EarthTwinHabZone1",
    "StarCatalog": "EXOCAT1",
    "OpticalSystem": "KasdinBraems",
    "ZodiacalLight": "Stark",
    "BackgroundSources": "GalaxiesFaintStars",
    "PlanetPhysicalModel": " ",
    "Observatory": "WFIRSTObservatoryL2",
    "TimeKeeping": " ",
    "PostProcessing": " ",
    "Completeness": "BrownCompleteness",
    "TargetList": " ",
    "SimulatedUniverse": " ",
    "SurveySimulation": " ",
    "SurveyEnsemble": " "
  }
}