sciml/diffeqbenchmarkserver.jl

Server behind the benchmarking bot @DiffEqBot for benchmarking scientific machine learning (SciML)

diffeqbenchmarkserver.jl's Issues

Setup other repositories to use DiffEqBot

Crossref: SciML/OrdinaryDiffEq.jl#769 (comment)
StochasticDiffEq, DelayDiffEq, and DiffEqSensitivity setup with to be runnable. DiffEqBase should get triggerable scripts for downstream steps.

Automated Deployment

We should have some way to deploy the bot to heroku whenever we make a commit on the master of this repository. The config file should remain as it is, rest of the code should migrate over there.

Use a new environment for every job

Ref: #2
Every job should make a new environment so that the dependencies don't get messed up (especially if we run simultaneous jobs).

Better scheduling algorithm

This bot may be used for testing purposes later, so we might need a new scheduling algorithm for the bot.

1. We can run simultaneous tests.
2. But the benchmark job should always run on the idle runner.
3. Support multiple runners.