tardis-sn/tardis

TARDIS - Temperature And Radiative Diffusion In Supernovae

tardis's Issues

Installation of TARDIS: No errors but not successful

I am still attempting to get Tardis to install. At the moment I run the following:
pip install tardis-sn --user --pre
I then received a rundown of the redundancies, which are all met, and then the following
cleaning up
It does not fail, just acts as if it was testing requirements.

The full readout:
Requirement already satisfied (use --upgrade to upgrade): tardis-sn in /home/username/.local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): astropy>=0.3 in /usr/local/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/local/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): scipy in /usr/local/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): h5py in /usr/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): pandas>=0.12 in /usr/local/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): pyyaml in /usr/local/lib/python2.7/dist-packages (from tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /usr/local/lib/python2.7/dist-packages (from pandas>=0.12->tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): pytz>=2011k in /usr/local/lib/python2.7/dist-packages (from pandas>=0.12->tardis-sn)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from python-dateutil->pandas>=0.12->tardis-sn)
Cleaning up...

sometimes photons are redder than the next line

why? debug

Tests for line list position

Inspired by #119
we should write a test (that needs some atomic data) to verify that the binary search correctly places packets in the line list, or outside the line list (on the red extreme)

@wkerzendorf perhaps we can devise this together, once the NLTE population test is in place and understood in #112

pandas warning on config ?

I pulled from Master today. With the current version I got the following warning:

radial1d = model.Radial1DModel(tardis_config)
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py:2536: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame
inplace=inplace)

it's only a warning (not an error) but it didn't happen before. Sounds like something we should clean up. Can someone tell me if something in this config file is not really valid now (@wkerzendorf @mklauser)?

The config file I used was:

#Currently only simple1d is allowed
tardis_config_version: v1.0
supernova:
    luminosity_requested: 9.44 log_lsun
    time_explosion: 13 day
#    distance : 24.2 Mpc

atom_data: kurucz_atom_pure.h5


model:

    structure:
        type: specific

        velocity:
            start : 11000 km/s
            stop : 20000 km/s
            num: 20

        density:
            type : uniform
            value: 1.e-14 g/cm3

    abundances:
        type: uniform
        O: 0.19
        Mg: 0.03
        Si: 0.52
        S: 0.19
        Ar: 0.04
        Ca: 0.03

plasma:
    disable_electron_scattering: no
    ionization: nebular
    excitation: dilute-lte
    radiative_rates_type: dilute-blackbody
    line_interaction_type: scatter

montecarlo:
    seed: 23111963
    no_of_packets : 1.0e+5
    iterations: 10

    black_body_sampling:
        start: 1 angstrom
        stop: 1000000 angstrom
        num: 1.e+6
    last_no_of_packets: 1.e+5
    no_of_virtual_packets: 10

    convergence_criteria:
        type: specific
        damping_constant: 1.0
        threshold: 0.05
        fraction: 0.8
        hold: 3
        t_inner:
            damping_constant: 1.0

spectrum:
    start : 500 angstrom
    stop : 20000 angstrom
    num: 10000

seed is not read from yaml

and make seed shorter for 32 bit systems

Restructure atomic database references

to rid lines with abundance 0.

cuts in excitation potential, ions, ....

important that the references to different levels are right. (more clever indexing scheme)

Make able to read input models in simple format

Currently models are input using artis files (model.txt and abundances.txt). We want to retain the ability to do this but add another option that reads simpler file formats (density and elemental composition as functions of velocity).

chianti has line with nu=0.0 (hydrogen)

reflecting packets

Note that (somewhat arbitrarily) as a quick and dirty approach "reflected" packets currently get a new direction that is chosen from an isotropic distribution in the "outgoing" hemisphere. This is wrong and if we ever want to use that option for anything real we should think about it. E.g. at least use some sort of foreshortening (as for initialization of packets) or better still think about what it really means!

check used zetas

currently from knox et al.

Problem with setuptools_bootstrap module on install

On trying to do the pip install, I get the following error message:

ImportError: No module named setuptools_bootstrap

I installed astropy as recommended

pip install astropy --user

where I was also told to upgrade my setuptools with

easyinstall -U setuptools

But the pip install still gave the above error. I was able to install tardis fine if I commented out the line in setup.py that imported the setuptools_bootstrap module and built myself, which is a little confusing.

This may be a problem with some other installation (e.g. astropy) or a fault on my part, but I thought I'd flag it anyway as other users may come across it if they have a simialr python set up.

Versions:
Python: 2.7.3
Numpy: numpy-1.8.0
OS X 10.8.2

difference between code and description

excitation: nebular (I don't think it's an option, but I don't know which are. It's not clear)
radiative_rates_type: detailed (log file in attachment, also, the comment says that lte, nebular and detailed are supported but the error message says only dilute-blackbody and detailed)

https://www.dropbox.com/s/cvjclktxdyhffg5/sofiatti-Mar11-2014-tardis_rad_detailed.log

stimulated emission in population solver

@wkerzendorf
Do you remember if there's a reason that we handle simulated emission in the population solver via:

(this from plasma_array.py, line 557ish)

            r_ul_matrix = np.zeros((number_of_levels, number_of_levels, len(self.t_rads)), dtype=np.float64)
            r_ul_matrix_reshaped = r_ul_matrix.reshape((number_of_levels**2, len(self.t_rads)))
            r_ul_matrix_reshaped[r_ul_index] = A_uls[np.newaxis].T
            r_ul_matrix_reshaped[r_ul_index] *= beta_sobolevs[lines_index]

            stimulated_emission_matrix = np.zeros_like(r_ul_matrix)
            stimulated_emission_matrix.reshape((number_of_levels**2, len(self.t_rads)))[r_lu_index] = self.stimulated_emission_factor[lines_index]


            r_lu_matrix = np.zeros_like(r_ul_matrix)
            r_lu_matrix_reshaped = r_lu_matrix.reshape((number_of_levels**2, len(self.t_rads)))
            r_lu_matrix_reshaped[r_lu_index] = B_lus[np.newaxis].T * j_blues[lines_index] * beta_sobolevs[lines_index]
            r_lu_matrix *= stimulated_emission_matrix

Why didn't we just do:

            r_ul_matrix = np.zeros((number_of_levels, number_of_levels, len(self.t_rads)), dtype=np.float64)
            r_ul_matrix_reshaped = r_ul_matrix.reshape((number_of_levels**2, len(self.t_rads)))
            r_ul_matrix_reshaped[r_ul_index] = A_uls[np.newaxis].T + B_uls[np.newaxis].T * j_blues[lines_index] 
            r_ul_matrix_reshaped[r_ul_index] *= beta_sobolevs[lines_index]

            r_lu_matrix = np.zeros_like(r_ul_matrix)
            r_lu_matrix_reshaped = r_lu_matrix.reshape((number_of_levels**2, len(self.t_rads)))
            r_lu_matrix_reshaped[r_lu_index] = B_lus[np.newaxis].T * j_blues[lines_index] * beta_sobolevs[lines_index]

Perhaps I'm being thick (is there an indexing issue here?) - but if this second one would work, perhaps we should consider switching to that? It's easier to read code and involves less dancing around.

If it's not stupid then I can make a version/pull request to deal with this.

handling of missing zeta data

When the atomic data base does not contain zeta-values (for nebular ionization mode) there is currently a kluge to put 1.0's into the array (which is based on catching nan's).

This should be done better, ideally not relying on nan catching.

Exponential profile - problem with units

The exponential density profile doesn't work - problem with units on the rho_0 value.

Also, a question of names: shouldn't we call this a power-law density profile?

We could also have an exponential one (i.e. rho propto exp(-v / v_0) ).

Putting these in is easy and would be useful - simple profiles that work for exponential and power-law cases are things that may be useful for many people to play with.

Add warning when w > 1

When the dilution factor is greater than 1 put out a warning to the logger. This is in the plasma_array.py file
This issue replace #65

Masking lines still makes them not 'invisible'

When Masking lines by setting tau_sobolev=0 it still sometimes happens that d_line is the smallest one and it jumps into the line evaluation procedure.

This should not happen and must have something todo with electron scattering problems.

Ion fraction exploration

Compare CHIANTI vs Kurucz in Nebular settings

Should be done before submission.

bugfix for the current_nu < nu_line

Needs bugfix for the current version of the code #30 not applicable anymore.

Change in the dimensionality of the arrays in plasma_array.py

In plasma_array.py the dimensionality of the arrays changes to n-1 if we have only one shell. A test case is coming soon (to be written).

Problems with convergence of NLTE level populations

oscillating around the real value - straight lines for levels 50 and 90 of species 20,1 (Ca II)

Move tests for `tardis.util` from io/tests/test_config_reader to tests/test_util.py

The tests are located in the wrong place.
Also add tests for 'si2' and other species without space.

There are functions in test_config_reader (like test_species_string_to_species() that test functions that are located in util. These functions should be moved to test_util.py.

collisional data missing

This needs to be a meaningful error message when collision data is missing:

AttributeError: 'NLTEData' object has no attribute 'C_ul_interpolator'

Suggest that it should return a warning (or simply refuse to run) if asked for this and there is no data. I think its ok to just refuse to run if NLTE is asked for and there is no collisional data (although it's also ok to continue provided that the user is thrown a big warning message)

Installation problem with TARDIS on Ubuntu 12.04

I am using Ubuntu 12.04. When trying to install TARDIS with pip install tardis-sn --user --pre, the installation breaks with a "#error Do not use this file, it is the result of a failed Cython compilation". I am attaching the error traceback along here.

correct treatment of frames in electron scattering

angle aberation

interpolate outside of 2000 and 50000 setting zeta to 1.

change absorbed nus in `montecarlo_nus` to be positive irregardless of absorption.

correct treatment of frames in estimators

check speed of partition function calculation

Currently 337 ms, speedup with Cython? Numpy?

be able to save states of the code

use memory view in Storagemodel

cdef float_type_t[:,:,:] arbitrarily_np_array_view works in class definition

Atomic database seems to contain float level indices

That is strange, but seems to work in production code. I recall that somehow that was okay - maybe even desired. This should be fixed/explored.

Adding warning to MD5 mismatch

Hi @mklauser, you said TARDIS complained when you use the wrong database (well an MD5 mismatch). Can you show me where this is happening? Maybe just a copy of the Traceback would be great!

zone boundaries with read in models

Problems can arise if one tries to set the inner/outer boundary value for the velocity (in the model part of the yaml file) to be very close (exactly?) at the boundary of a zone as defined by a read in ascii file (containing the density profile). Problem, I think is that one can end up with zones that have no volume. Should fix this with a check that says we can only subdivide zones from the input file if the velocity boundary is sufficiently different (above some threshold difference, or something).

sanity checks on radiation field model

Cases are emerging where our ionization approximation (nebular) appears to go "out the window". My suspicion is that these can be attributed to the radiation field model becoming a very poor approximation (high optical depths combined with our current lack of thermalisation parameters). Ultimately better physics will help. But, in the short term, I would like us to think about putting in sanity checks on the radiation field model itself: some check for how good an approximation equation (1) of Kerzendorf & Sim 2014 actually is in each grid zone. If it's clearly a terrible model, we should warn/abort think about doing something better. Obvious (simple) suggestion would be to test on dispersion (in frequency space) but we can think more on this.

This is something we should do well rather than soon!

Adding distance, redshift, reddening to final spectrum calculation

Dealing with incomplete atomic data

Issue is how to deal with input data that doesn't contain all ions. RIght now, TARDIS wants complete information - that means ionization potentials (and statistical weights to calculate partition functions).

Ideally, it should be able to handle (consecutive) ions (e.g. Fe I, Fe II and Fe III) and just spit out a warning that it doesn't have data for higher ions.

If provided with non-consecutive ions (Fe I and Fe III but no Fe II) it should complain and exit, I think.

sum(input packet energy) more than output + absorbed

Maybe some doppler factor injection - maybe fixed already.

simple sets of stratified abundances

It would be good to have a version of the abundance section that can deal with simple sets of multi zone abundances - say a "type:stratified" or something. Basically one would set up a bunch of abundance zones, each of which is specified by a set of numbers like:

         start: 10000 km/s
         stop: 11000 km/s
         O: 0.19
         Mg: 0.03
         Si: 0.52
         S: 0.19
         Ar: 0.04
         Ca: 0.03

…so that the abundances are populated with those vales in that velocity range. The user can then set up a model with a handful of zones (perhaps up to a max of 10 or so) so that they can play with non-uniform abundances along one of the simple density profiles (i.e. a level of sophistication between our current "uniform" setting and the gory details of giving a fully tabulated set via an input file).

Can this be incorporated as part of the next version of the config file?

document use of ionization `nebular`

and warn if excitation =lte

w never greater than 1 in level populations?

is that right?

this line level_populations[~self.atom_data.levels.metastable] *= np.min([self.ws, np.ones_like(self.ws)],axis=0) in plasma_array. worth checking

Inspect convergence criteria for speedup

Wavelength vacuum vs air

Make sure there's either only vacuum or only air

Logging

Added first logging hooks. There's two seperate logging streams. The normal one logs everything outside the TARDIS montecarlo simulation. The second one is a packet logger, which needs to be switched on at compile time. It creates very large logfiles and is useful for debugging the packet propagation.

There's still quite a few functions that don't have loggers attached to.

Close this issue once the logging has been satisfactorily implemented

Colorize logger

Currently our log messages are uncolorized which makes it a bit difficult to spot warnings and errors during runtime. There are some implementations floating around on the web which show the general gist.

Contribution of lines to wavelength bins

Convert spectrum to physical units

No module named plasma / BasePlasmaArray AttributeError

I'm pretty sure this is just a documentation issue, but I thought I would flag anyway.

The documentation on readthedocs says to do the following to create a simple plasma:

>>> from tardis import atomic, plasma
>>> atom_data = atomic.AtomData.from_hdf5()
>>> my_plasma = plasma.BasePlasma({'Fe':0.5, 'Ni':0.5}, 10000, 1e-13, atom_data)
>>> print my_plasma.abundances
atomic_number abundance_fraction number_density
------------- ------------------ --------------
           28                0.5  513016973.936
           26                0.5  539183641.472

However, there I cannot import plasma (it does not exist):

>>> from tardis import plasma
ImportError                               Traceback (most recent call last)
<ipython-input-18-0dbefda4ca15> in <module>()
----> 1 from tardis import plasma

ImportError: cannot import name plasma

There is however, a module called plasma_array, containing the BasePlasmaArray class. I tried an analogous command:

>>>from tardis import plasma_array, atomic
>>> atom_data = atomic.AtomData.from_hdf5()
>>> my_plasma = plasma_array.BasePlasmaArray({'Fe':0.5, 'Ni':0.5}, 10000, 1e-13, atom_data)
AttributeError                            Traceback (most recent call last)
<ipython-input-24-a42101f19f8c> in <module>()
----> 1 my_plasma = plasma_array.BasePlasmaArray({'He':1.0}, 10000, 1e-13, atom_data)

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tardis/plasma_array.pyc in __init__(self, number_densities, atom_data, time_explosion, delta_treatment, nlte_config, ionization_mode, excitation_mode)
    141         self.nlte_config = nlte_config
    142         self.delta_treatment = delta_treatment
--> 143         self.electron_densities = self.number_densities.sum(axis=0)
    144 
    145         self.level_populations = pd.DataFrame(index=self.atom_data.levels.index, columns=number_densities.columns,

AttributeError: 'dict' object has no attribute 'sum'

This may be a separate issue, or perhaps the docs just need updating and I am running the wrong commands. Probably wants fixing regardless.

Also, please tell me if you'd rather I flagged issues like this somewhere else e.g. a user's group.