darkquestcosmology/dark_emulator_public

A cosmological tool for emulation of various basic statistics of dark matter haloes based on Dark Quest project (https://darkquestcosmology.github.io/).

dark_emulator_public's Issues

clustering predictions beyond redshift 1.5

Hello,

I am using this fantastic tool you have developed. Would it be possible to extended the predictions of clustering to redshift 2 or 3 ?

Indeed ongoing spectroscopic surveys are gathering large samples there and having this tool to interpret these would be great.

Best wishes

cannot import name 'fftlog' when importing dark_emulator

Hi,

I am trying to run the dark emulator using a pip installation:

pip install dark_emulator

was successful.
Then running

from dark_emulator import darkemu

in ipython, leads to this error:
ImportError: cannot import name 'fftlog' from partially initialized module 'dark_emulator' (most likely due to a circular import) (/Users/ferte/anaconda3/lib/python3.9/site-packages/dark_emulator/init.py)

Any inputs on how to solve this?

Thank you,
Best,
Agnès

Extracting P_gg from emulator

I have written some code to extract P_gg directly from the emulator (rather than w_p or deltaSigma). In that code I have called self._compute_p_2hcc etc. but I am getting the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-f74c1312f5c2> in <module>
     21 Pk_dq_mg = DarkQuest.get_p_gm(hod, ks, z)
     22 if DQ_gg:
---> 23     Pk_dq_gg = DarkQuest.get_p_gg(hod, ks, z)
     24 else:
     25     Pk_dq_gg = np.copy(Pk_dq_mg)

~/Code/python_library/mead_DarkQuest.py in get_p_gg(self, k, redshift)
    724     self._compute_p_1hcs(redshift)
    725     self._compute_p_1hss(redshift)
--> 726     self._compute_p_2hcc(redshift)
    727     self._compute_p_2hcs(redshift)
    728     self._compute_p_2hss(redshift)

/usr/local/lib/python3.9/site-packages/dark_emulator/model_hod/hod_interface.py in _compute_p_2hcc(self, redshift)
    860             self._compute_ng()
    861         if self.p_hh_computed == False:
--> 862             self._compute_p_hh(redshift)
    863 
    864         poff = self.gparams["poff"]

/usr/local/lib/python3.9/site-packages/dark_emulator/model_hod/hod_interface.py in _compute_p_hh(self, redshift)
    521 
    522     def _compute_p_hh(self, redshift):
--> 523         self._compute_p_hh_spl(redshift)
    524 
    525         logdens_de = self.g1.logdens_list

/usr/local/lib/python3.9/site-packages/dark_emulator/model_hod/hod_interface.py in _compute_p_hh_spl(self, redshift)
    439                 logdens2.append([logdens_m[j], logdens_p[j],
    440                                  logdens_m[j], logdens_p[j]])
--> 441         xi_dir_all = self._get_xiauto_direct_noint(np.concatenate(
    442             logdens1), np.concatenate(logdens2), redshift)
    443 

/usr/local/lib/python3.9/site-packages/dark_emulator/model_hod/hod_interface.py in _get_xiauto_direct_noint(self, logdens1, logdens2, redshift)
    212         logdens2_mod = np.copy(logdens2)
    213         logdens2_mod[sel2] = -5.75
--> 214         xi_dir = self.xi_auto.getNoInterpol(
    215             redshift, logdens1_mod, logdens2_mod).transpose()
    216         if sel1.sum() != 0:

/usr/local/lib/python3.9/site-packages/dark_emulator/darkemu/auto.py in getNoInterpol(self, redshift, logdens1, logdens2)
     89         elif sindex >= 20:
     90             s0 = 20
---> 91             xia0 = np.array([rbs(-self.logdens_list, -self.logdens_list,
     92                                  self.xih_mat[:, :, s0, i])(-logdens1, -logdens2) for i in range(21)])
     93             return xia0

/usr/local/lib/python3.9/site-packages/dark_emulator/darkemu/auto.py in <listcomp>(.0)
     89         elif sindex >= 20:
     90             s0 = 20
---> 91             xia0 = np.array([rbs(-self.logdens_list, -self.logdens_list,
     92                                  self.xih_mat[:, :, s0, i])(-logdens1, -logdens2) for i in range(21)])
     93             return xia0

/usr/local/lib/python3.9/site-packages/scipy/interpolate/fitpack2.py in __call__(self, x, y, dx, dy, grid)
    905 
    906             if (x.size >= 2) and (not np.all(np.diff(x) >= 0.0)):
--> 907                 raise ValueError("x must be strictly increasing when `grid` is True")
    908             if (y.size >= 2) and (not np.all(np.diff(y) >= 0.0)):
    909                 raise ValueError("y must be strictly increasing when `grid` is True")

ValueError: x must be strictly increasing when `grid` is True

Any idea on what is wrong? The ks array that I am passing to get_p_gg is strictly increasing.

Add notes for developers

We need to add notes for developers to describe the procedures of development.

dlogMh -> self.dlogMh

On line 279 of hod_interface.py:

dark_emulator_public/dark_emulator/model_hod/hod_interface.py

Line 279 in af71ff8

self.dndM * self.Ncen * self.Mh, dx=dlogMh)

Should it read dx=self.dlogMh instead of dx=dlogMh? I'm getting an undefined variable warning flagging up on my IDE.

replace pyfftlog by fftlog written by Xiao Fang for efficient computation of cosmology signals

FFTLog implementation by Xiao Fang includes a function to reduce the effect of ringing at high Fourier modes, called as c_window.

Changing cosmology in tutorial-hod.ipynb

If I run the unmodified tutorial.ipynb notebook it works fine and runs through to the end, producing all the plots. However, if I change any of the cosmological parameters in the first line of In[5]:

cparam = np.array([0.02225,0.1198,0.6744,3.094,0.9645,-1.])

I get the following error:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-5-9026577a174b> in <module>
      1 cparam = np.array([0.02225,0.1198,0.6744,3.094,0.9645,-1.])
----> 2 hod.set_cosmology(cparam)
      3 
      4 gparam = {"logMmin":13.13, "sigma_sq":0.22, "logM1": 14.21, "alpha": 1.13, "kappa": 1.25, # HOD parameters
      5            "poff": 0.2, "Roff": 0.1, # off-centering parameters p_off is the fraction of off-centered galaxies. Roff is the typical off-centered scale with respect to R200m.

/usr/local/lib/python3.9/site-packages/dark_emulator/model_hod/hod_interface.py in set_cosmology(self, cparams)
    148             if self.do_linear_correction:
    149                 self.massfunc.set_cosmology(self.cosmo_orig)
--> 150             self.rho_m = (1. - cparams[0][2])*rho_cr
    151             self.R200 = (3*self.Mh/(4.*np.pi*self.rho_m)/200.)**(1./3.)
    152             self.R200_mat = np.tile(self.R200, (len(self.fftlog_2h.k), 1)).transpose()

IndexError: invalid index to scalar variable.

Am I doing something wrong? If so, how do I change the cosmology within the hod part of the emulator?

Bias utility functions

API for linear bias as a function of halo mass or mass threshold would be useful. I will work on it.

pyfftlog_interface does not exist

I was going to put together a PR to address issues #22 and #25 (which are duplicates), but I was unable to install from source because submodule pyfftlog_interface does not seem to have been included with the repository.

Version 1.1.1 Importing Error

I am seeing this error whenever I try to import version 1.1.1. I installed 1.0.23 in my environment without a problem. I think it could be some unremoved requirement fftlog?

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import dark_emulator

File [/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/__init__.py:4](https://vscode-remote+ssh-002dremote-002bnode056.vscode-resource.vscode-cdn.net/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/__init__.py:4)
      2 __version__ = '1.1.1'
      3 __url__ = 'https://dark-emulator.readthedocs.io/'
----> 4 from . import darkemu
      5 from . import model_hod

File [/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/darkemu/__init__.py:1](https://vscode-remote+ssh-002dremote-002bnode056.vscode-resource.vscode-cdn.net/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/darkemu/__init__.py:1)
----> 1 from .de_interface import base_class

File [/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/darkemu/de_interface.py:10](https://vscode-remote+ssh-002dremote-002bnode056.vscode-resource.vscode-cdn.net/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/darkemu/de_interface.py:10)
      8 from .auto import auto_gp
      9 from .hmf import hmf_gp
---> 10 from .. import fftlog
     11 import numpy as np
     12 from scipy.interpolate import InterpolatedUnivariateSpline as iuspline

ImportError: cannot import name 'fftlog' from partially initialized module 'dark_emulator' (most likely due to a circular import) (/data/groups/jeltema/zhou/miniconda3/envs/darkemu3/lib/python3.11/site-packages/dark_emulator/__init__.py)

Fig bugs in API in model_hod.get_wp

I forgot to reflect the API change from self.xi_gg to self.get_xi_gg.

Output cosmology-dependent fitting parameters A and a of the HMF

Adding a function to output cosmology-dependent fitting parameters A(z) and a(z) of the halo mass function. This makes it easier to use the Dark Emulator HMF in external codes.

Replace calls to print to calls to logging

It'd be great if all the print statements about initializing modules are converted into calls to logging (especially for when running MMC chains!). See for instance https://github.com/DarkQuestCosmology/dark_emulator_public/blob/main/dark_emulator/darkemu/sigmaM.py#L12

A small bug when the galaxy profile is changed from "NFW" to "emulator" (or vice versa)

There is a flag p_hm_satdist_computed in the HOD module. This is to avoid to compute the radial profile of satellites around halos at various masses, when it is already computed. Usually, one needs to recalculate it only when the cosmological parameters or the redshift are updated, and not when only the halo-galaxy connection parameters are modified. However, when sat_dist_type (which is in the gparam dictionary) is updated, this must be recalculated. Otherwise, when one wants to compare the galaxy correlation function with the "NFW" and "emulator" profile, the code gives an identical result.

(This is following an investigation of the issue reported by C. Cuesta-Lazaro. Thanks!)

Turn off Colossus cache

We should turn off Colossus cache since it slows down model calculation when running chains. It can be done when instantiating cosmology class by putting persistence = ''.

Adding auto correlation function as a function of theta to HOD module

There's a request to add auto correlation function as a function of theta (Eq. (49) in this paper). I'll implement it.