halomod/thehalomod-spa

Single-page app for TheHaloMod

TheHaloMod-SPA

A single-page application to act as a GUI for HaloMod. A spiritual successor to TheHaloMod.

Table of Contents

• Contributing
  • How to run it locally
  • Architecture
  • Github Actions

Contributing

How to run it locally

1. Clone the repo locally.
2. Run cd TheHaloMod-SPA. In other words, move into that directory in your shell of choice.
3. Get the server up and running. Instructions are in the server readme.
4. Startup the client. Instructions are in the client readme.

Architecture

The overall architecture of TheHaloMod-SPA is broken up into two parts: server and client. From a high-level perspective, the server handles the API portion of the website, and the client holds the front-end. This way, the front-end can benefit from modern JavaScript UI libraries and the backend can run all the Python code of the various libraries that this project is based on.

Each part, the server and the client, have their own documentation on their architecutre in their readmes. This includes info on the package managers they use, general thought and design considerations, as well as other aspects.

• server architecture
• client architecture

GitHub Actions

The only parts that are not included in the server and client at this moment are the GitHub workflows. Those are included in root as required by GitHub. Github workflows are an aspect of GitHub Actions and form the continuous integration part of this project. Some of the things that the Github workflows are setup to do are:

• Run tests on both the server and client when new pushes are made to any branch in the project
• Run linting on both the server and client when new pushes are made to any branch in the project
• Deploy the client when a new push is made to the main branch

Note that in order to execute .sh files inside of a workflow (the .yaml files in .github/workflows), they need to be ran with sh shell_file_name.sh INSTEAD of something like . ./shell_file_name.sh. If they are ran with . ./shell_file_name.sh they will not apply to the environment correctly.

Click here to see more information about how github actions are used

thehalomod-spa's Issues

Tooltips for parameters

I know that tooltips are already an issue, but I wanted to say explicitly that tooltips for the input parameters in the form itself would be awesome. These could literally be gotten directly from the hmf/halomod library docstrings. I guess this would be another deploy-time script to pull the docstrings into a json file somewhere.

feature request: uploadable transfer function

There used to be a feature where a user could upload a file to be read as the transfer function (so instead of using CAMB/EH/ etc.). It would be nice to get this back in.

Some Plot Choice Quantity Names

Some of the names in the plot choice drop-downs aren't great (this is my fault). IN particular:

• Halo/Tracer Concentration Mass Relation --> Halo/Tracer Concentration
• Remove the BoxSize (or "how_many") entry
• Wavenumber --> "Fourier Scale, k (linear)"
• Fourier Scale --> "Fourier Scale, k (halo model)"
• Scale --> "Comoving Scale, r"
• Mass Function --> "Halo Mass, m"

Furthermore, the last four above should be the first four in the X-axis drop-down. Might even make sense to put a little divider between them and the rest in the dropdown (if possible).

Confirmation dialog ENTER

Pressing ENTER in the model creation dialog should confirm the creation. At the moment, it does nothing.

Also, you have to manually click on the little dialog to get it to be in focus.

Non-unique model names

If I make a model with name "yerp" and then go and create a new one with name "yerp", it seems to overwrite the first one. It should block on finding the same model name and ask you to rename it before saving.

Feat: ability to report a bug in a specific model

In the current version at https://thehalomod.app, you can report a specific model as buggy (this is useful, because sometimes the model will compute just fine, but a user will notice that it "looks wrong"). This sends the full parameter list of the particular model to me, so I can debug more easily. I'd say this should appear as an icon in the Models box.

I have so far done this through sentry, but I don't think this is the best way (I actually lose some of the info, because it overflows the maximum message size for sentry). Not sure what a better way would be... maybe an email?

Effective Spectral Index not showing anything

For some reason the plot shows nothing. This is probably a bug in hmf, but I'll need to confirm.

image in the browser tab should the THM

At the moment it seems to show with the Vue logo.

Workflow Docs

There are docs for the server and client, but not for the deployment/workflows (other than very basic stuff on the main readme).

It would be useful for me to have an outline of how deployment works, where secrets are kept, what they're for etc.

Initial Plot Choice

At the moment, the initial plot choice is the "Halo Concentration Relation", which is usually extremely boring to look at. This should be changed to the Tracer Power Spectrum. Furthermore, the plot choice changes back to the Halo Concentration Relation every time a new model is created/edited. It should remember where it was at.

X-axis choice "mass function" should be "Mass"

Ability to plot r^2 xi(r)

The correlation function(s) are often multiplied by r^2 when they are plotted, which reveals more details. I could add r^2 xi(r) to halomod as a new quantity, OR we could just add the ability to do such transforms in the plot. The first is much less clunky for THM, but is kind of not in the scope for halomod itself. So.... not sure.

Design Idea: Collapsible Model Parameter Forms

Some of the models have a LOT of parameters, including some of the default models (eg. Tinker for the HMF). This makes the vertical spacing quite long, and makes it easy to miss some of the inputs. One way around this might be to have the model parameter sections, that appear on the right of the model choices, as collapsible. Not sure if this is easy or hard. If easy, I'd say it's worth it.

At the moment, at the very least, the mass range and mass resolution are very easy to miss (being after all the Tinker parameters), and should be fixed. One way would be just to bring them to the top of the HMF section (so, above the choice for Tinker). Another would be to bring all the "resolution" kinds of parameters -- so lnk-range, mass-range, resolution, scale range, etc., into one section called "Ranges", which should be towards the top.

It's not clear how to edit the model names

Indeed, I thought you couldn't, and was going to make an issue about it, but then accidentally clicked on one of the names in the Model box. Now that I know it's there, it makes sense, but it's not visually obvious that you can do it.

I guess this could be fixed by #100 as well.

1-halo central-satellite PS seems wrong

This is most likely a bug in halomod. Will have to confirm. Simply for the default model.

Little Usability Improvements

It might be nice to have some little usability improvements. Priority and the contents of this US can be discussed with Steven. @steven-murray if you wanted to check some of these out and put in your thoughts if you have some time that would be helpful 🙂 .

• The model name isn’t centered to the icons on the right
• It might be nice to have a ? icon next to things like “Models“, “Download“, and “Plot“ which has a little popup that explains what is is for.
• It also might be nice to have the title at the top left link back to the home page.
• The about page could include a link to the Github page for the project.
• “Submit issue“ could be added to the top bar kind of like with the original website.
• If the plot remembered what was last selected, it might be more intuitive for the user when they return to the main page.
• If the plot defaulted to an axis combo that wasn’t a straight line 😛
  

Is KEYMAP required any more?

TheHaloMod-SPA/server/halomod_app/utils.py

Line 172 in 5e4e508

KEYMAP = {

This is all happening on frontend now, right?

Version of hmf/halomod shown on top

Currently the version of hmf/halomod show in the top bar. I like this.

Feature: make parameters that are different from default a different color

There are a lot of parameters, and it can be difficult to remember what you changed. It might be nice if the fields that had different values from the default initial values would be highlighted with some different color (just their underline).

Idea: initial spashscreen/tutorial

You know how sometimes on a new app, when you first go there, it will come up with floating boxes that tell you how to use the app in a stepwise manner? Not sure how hard this is, but it would be cool, and kinda replace the need for a dedicated Help page.

Parameter Names

Some comments about parameter names:

• Under Halo Model: "Scale (log10)" should be "Real-Space Scale Range (log10 Mpc/h)"
• Under Halo Model: "Wavenumner (log10)" should be "Fourier Scale Range (log10 h/Mpc)"
• Number of r bins should be "Number of Real-Space Scales"
• Parameters like M_min and M_1 in the HOD section should have subscripts for the "min" and "1" if possible.
• greek names like "alpha" "beta" and "gamma" can be their unicode greek letters, if possible (α, β, etc)

Client Readme Comments

• I can't seem to install serve properly for local production runs. I've tried npm i serve and yarn global add serve. Both times, when running serve dist -p 5111, I get the error "command not found: serve". Am I doing something wrong?

Just in general, I expect that people working on this (essentially myself) will be far less familiar with JS/vue/node etc. than python. The docs here are great high-level stuff, and the function-level documentation in the files is great. I just wonder if there's something we can do between these levels. Perhaps what I'm looking for is just a tutorial on how to use vue, which I'm sure exists in abundance ;-)

feature request: ability to upload the downloaded params to instantiate a model

Would be kinda cool to be able to upload the downloaded model param file (the .toml one) to create a new model. This would also mean that people could take control of their own model persistence.

analytics

I have Google Analytics set up in the current version. This is really helpful because it means I can add numbers to proposals etc. about how many people are using it. I get the feeling however that it doesn't really capture everyone. Happy to go with something like umami if we can set that up.

Link to github in the top bar

Just like in the current version, ability to report an issue on github straight from the page. I think this is neat.

Default entry for tracer concentration/profile

Default entry for tracer concentration/profile should be "Same as halo concentration/profile", rather than "Tracer Profile"

Sometimes broken models just display the previous unbroken one

So, say I have a model with sigma_8 = 0.8 (which is default). Now, update it to sigma_8=8000 (which almost certainly breaks something). Now, copy it and set the new one to sigma_8=0.8. Look at dn/dm, which should look different between them. But it looks the same, which I think shows that the crazy model is broken and showing its original unbroken values, without displaying any error.

Errors in the form don't stop the save

Putting bad values in the form shows up with an error underneath the field, but you can still click "Save". Once it returns to the main screen, that model just doesn't show up.

Make Icon Clickable

I'd prefer the Icon+"TheHaloMod" in the top-left to be clickable, and take you home. Then we can remove the Home button from top right.

Downloadable Citation List

In a similar vein to #131, it would be awesome if, after the full model has been described, the user could download a list of all the citations that went into the full model. This would be another item in the download dropdown.

feature request: default hires transfers for the default cosmologies

There are a few options for default cosmologies (Planck18, PLanck15, etc.). Since a lot of people will be choosing these defaults, it makes sense to have pre-computed hi-res transfer functions from CAMB sitting in files to be read (running CAMB is the slowest part of the calculation).

Provide links to references for each model

It would be awesome if each chosen component model (Tinker 2008, SMT, etc.) would be able to link to a particular paper that introduced that model. Not sure where exactly to put the link --- probably just as an icon next to the model name, once it is chosen? I intend to include these references in the actual hmf/halomod code. So again, this could be a job for a deploy-script.

Colors of plot lines

The big issue here is that the color palette for the plot lines is too visually compact. There are too many greens that are close together in color-space, and I can't easily distinguish models. This should be fixed.

Of secondary importance: it might be nice if the user could choose the color of the lines. Perhaps by clicking on the legend somewhere, or perhaps in the Model box.

Plot Formatting: top and right spines

Usually scientific plots have spines on the top and right. I'm not sure what I think about this -- would love to see both.

Changing the plot choice resets state of visible models

If I set bunch of models to not show on the main plot, and then change the axis plot choice, all the models show again. This is annoying.

Server Readme Comments

Some comments I thought about when reading the server readme:

• In the installation section, it might be useful to point out whether the user must create/enter a virtual environment before running ./run.sh --install. It's very common for Python types to manage their envs through conda or poetry, and to enter it manually before installing anything.
• It occurred to me -- you're using the requirements.txt to pin dependencies. Are you using a specific system to update those dependencies? It's quite difficult to get consistent pinned deps with just pip.
• When running redis-server I get 174707:M 19 Mar 2021 11:09:16.143 # Could not create server TCP listening socket *:6379: bind: Address already in use. I figure this was fine, probably set it up to run automatically previously. But, then when I ran . ./run.sh --test I got:

================================================================== short test summary info ===================================================================
FAILED tests/test_halomod.py::test_get_plot_data - TypeError: argument of type 'NoneType' is not iterable
FAILED tests/test_halomod.py::test_plot - redis.exceptions.ResponseError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able t...
FAILED tests/test_halomod.py::test_create - redis.exceptions.ResponseError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able...

This might be something I'm doing wrong, but if so, the docs should be updated. Otherwise, there's a bug in the code :-)

• When starting the server, I get the warning "Some Halo-Exclusion models have significant speedup when using Numba", indicating that numba is not in the requirements. This should be added -- it really does make some things faster!

Breaking CAMB shuts down the whole server

CAMB is a library that computes the transfer function (by default). It is written in FORTRAN, and if it really breaks, Python doesn't seem to be able to handle it as an exception -- the whole server just crashes. This is obviously not desirable. One way to mitigate this is to get the right parameter constraints in for cosmology, on the forms (these don't seem to be in yet). I'm not sure how else to protect against this... it might just be that we need a heartbeat program on the server to restart it if its down.

Idea: possibility of plotting two different quantities at once

Sometimes it is useful to be able to plot two things at once, eg. the tracer power spectrum, 2-halo tracer PS, and 1-halo tracer PS. If possible, these should be plotted with the same color (for the given model), but different linestyle (dashed, dot-dashed etc.) ON THE SAME PLOT.

Color Scheme / Theme

I like the layout already. However, the colors could use some tweaking.

I'd like the top panel to be dark -- some grey-black (around #555555. The rest of the colors could be something like a Monokai pallete. I like the pink used in the range sliders.

I think the current blue is a bit bleh, so I'd change that.

Also, not sure, but it might be nice to set the background of the sidebar in the form to another color than white?

Also, the top navbar, "The Halo Mod" should be "TheHaloMod".

"takahashi" --> "Use Takahashi?"

This makes it look cleaner (this is in the Transfer form)

log buttons

The log buttons have two deficiencies:

1. They should remember their state (so if you go to another plot choice, then return to the original, it should remember the state it was in).
2. They should have defaults for each plot choice. This information should already be in the KEYMAP (or whatever that is now).

The first point is not necessary if it seems too clunky, but the second is important.

lnk_min/max precision

The lnk_min/max currently look like this by default:

We don't need allll the decimal places, just a couple.

Where you name the model

At the moment, when you create a new model, it will only ask you to name the model after you click "Create". I intuitively thought I should be able to modify the model name in the form itself here:

where it says "New Model". Not sure if this is difficult architecture-wise (you'd still need the popup in case the user doesn't set the title there...). If not, I'd suggest adding it.

Tinker function parameter labels formatting

All the A_200 etc should have the "200" as a subscript, if possible.

Plot Formatting: make lines go above grid

At the moment, the grey grid lines go "over" the plot lines, which to me looks weird. I think the plot lines should be on top.

Add "Acknowledging" page

There's a page to this effect on the current website. This should be added as a button in top-right.

Cosmology: only Planck15 has H0

H0 is a parameter for every cosmology, but at the moment only Planck15 seems to have it listed?

Also, the redshift is really not part of the cosmology per se (it makes sense to put in the Cosmology form, but it isn't decided by the WMAP7 or Planck15 etc). I think visually it should be on a row of its own at the top of the Cosmology form.

Order of models in legend doesn't always matcht the model box

Eg.

Really not sure why?

Big Red Error doesn't show which model failed

It would be good if the error showed the model name of the one that failed. Shouldn't be too hard.

Loading indicator for long-running computations

Some quantities take a while to compute (especially 2-halo power spectra). When you click on those in the plot choice menu, often the plot takes a while to update, and it kinda feels broken because there's no spinny loader. On the other hand, I don't want a loader to appear for shorter calculations. Not sure if this is possible.