sciml/highdimpde.jl

A Julia package for Deep Backwards Stochastic Differential Equation (Deep BSDE) and Feynman-Kac methods to solve high-dimensional PDEs without the curse of dimensionality

highdimpde.jl's Issues

CI for testing GPU functionality

The GPU functionality of this package is currently not tested, and from a quick local test seems to have regressed.
This package should be set up with buildkite.

Add license

This repo was made without a license, though because some of the code comes from NeuralPDE.jl it should probably get an MIT license.

Integrate DeepBSDE.jl

For now, the file DeepBSDE.jl is a copy from NeuralPDE.jl and is not included in HighDimPDE.jl. It needs to be included, and modified to accept PIDEProblem and return PIDESolution, as well as accompanied with tests.

NeuralPDE.jl migrations

SciML/NeuralPDE.jl#514

Problem with gradient in nonlinear function `f`

For now gradients ∇v_y,∇v_z cannot be used in the nonlinear function

f(y,z,v_y,v_z,∇v_y,∇v_z, p, t)

Uncommenting those two line https://github.com/vboussange/HighDimPDE.jl/blob/21a25b7f13f15332a1b5872961315b7bcedc5a5b/src/DeepSplitting.jl#L92-L93

should make this work. Nonetheless, possibly because of issue FluxML/Flux.jl#1464 , this fails : the gradient throws a one dimensional array ∇vi(y1)[1] = [0f0]. Here is an example to play with (given that one has uncommented the above lines).

@testset "DeepSplitting algorithm - gradient squared" begin
    batch_size = 2000
    train_steps = 1000
    K = 1
    tspan = (0f0, 5f-1)
    dt = 5f-2  # time step

    μ(x, p, t) = 0f0 # advection coefficients
    σ(x, p, t) = 1f-1 #1f-1 # diffusion coefficients
    
    for d in [1,2,5]
        u1s = []
        for _ in 1:2
            u_domain = (fill(-5f-1, d), fill(5f-1, d))

            hls = d + 50 #hidden layer size

            nn = Flux.Chain(Dense(d,hls,tanh),
                            Dense(hls,hls,tanh),
                            Dense(hls,1)) # Neural network used by the scheme

            opt = ADAM(1e-2) #optimiser
            alg = DeepSplitting(nn, K=K, opt = opt, mc_sample = UniformSampling(u_domain[1],u_domain[2]) )

            x = fill(0f0,d)  # initial point
            g(X) = exp.(-0.25f0 * sum(X.^2,dims=1))   # initial condition
            a(u) = u - u^3
            f(y, z, v_y, v_z, ∇v_y, ∇v_z, p, t) = begin @show ∇v_y; sum(∇v_y.^2,dims=1) end

            # defining the problem
            prob = PIDEProblem(g, f, μ, σ, tspan, x = x, neumann = u_domain)
            # solving
            @time xs,ts,sol = solve(prob, 
                            alg, 
                            dt, 
                            # verbose = true, 
                            # abstol=1e-5,
                            use_cuda = false,
                            maxiters = train_steps,
                            batch_size=batch_size)
            push!(u1s, sol[end])
            println("d = $d, u1 = $(sol[end])")

        end
        e_l2 = mean(rel_error_l2.(u1s[1], u1s[2]))
        println("rel_error_l2 = ", e_l2, "\n")
        @test e_l2 < 0.1
    end
end

Method definition overwritten

I get the following output running the tests in this package locally:

WARNING: Method definition (::Type{HighDimPDE.PIDESolution{X0, Ts, L, Us, NNs, Ls} where Ls where NNs where Us where L where Ts where X0})(Any, Any, Any, Any, Any, Any) in module HighDimPDE at /home/runner/work/HighDimPDE.jl/HighDimPDE.jl/src/HighDimPDE.jl:115 overwritten at /home/runner/work/HighDimPDE.jl/HighDimPDE.jl/src/HighDimPDE.jl:122.
ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
WARNING: Method definition (::Type{HighDimPDE.PIDESolution{X0, Ts, L, Us, NNs, Ls} where Ls where NNs where Us where L where Ts where X0})(Any, Any, Any, Any, Any, Any) in module HighDimPDE at /home/runner/work/HighDimPDE.jl/HighDimPDE.jl/src/HighDimPDE.jl:115 overwritten at /home/runner/work/HighDimPDE.jl/HighDimPDE.jl/src/HighDimPDE.jl:122.
ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.

I think this makes running the tests much slower than before.

Problem with Zygote v0.6

We got the error

ERROR: LoadError: Can't differentiate foreigncall expression

with environment

(stochanal) pkg> st
      Status `~/projects/stochanal/Project.toml`
  [052768ef] CUDA v3.2.1
  [587475ba] Flux v0.12.4
  [57c578d5] HighDimPDE v0.1.0 `~/projects/HighDimPDE.jl`
  [295af30f] Revise v3.1.16
  [9a3f8284] Random

This comes from mc_sample that we suspect should be redefined within the HighDimPDE module

Migrate NNPDEHan from NeuralPDE.jl

While NNPDENS with Euler-Maruyama is similar, it is a different algorithm as Han does the expansion to layers-per-step instead of repeated layers. I think keeping this difference for future discovery of its effect on stability of the training processes is an interesting piece.

Integrate `NeuralPDE.NNStopping` and `NeuralPDE.NNKolmogorov` in HighDimPDE.jl

#75

`p::Dict` for parameters

Not sure if we should consider having a Dictionary of parameter, and how to implement it.

Generalize Deep Splitting Algorithm

https://github.com/vboussange/HighDimPDE.jl/blob/main/src/DeepSplitting.jl#L110

This is hardcoded to Euler-Maruyama. This could potentially be improved by making this use StochasticDiffEq to generalize the time stepping algorithms which could be used. It would be interesting research wise to just see whether a higher order method with a larger dt ends up being more efficient here.

Register

@JuliaRegistrator register()

Use LoopVectorization.@turbo on unrolled loops

https://github.com/vboussange/HighDimPDE.jl/blob/main/src/reflect.jl#L12-L36

This should be using LoopVectorization.@turbo, or LoopVectorization.@tturbo.

CI: Format Check Fails

Describe the bug 🐞

The format check fails because it tries to format the docs/ directory. We should exclude it and limit to src and test directories only.

Expected behavior

Limit format check to src and test directories only.

Add to SciML.ai website homepage

It's worth mentioning this package there, and in the documentation section.

Integrate `NNStopping` and `NNKolmogorov ` in HighDimPDE.jl

NeuralPDE.KolmogorovPDEProblem, documented in

• https://neuralpde.sciml.ai/dev/solvers/kolmogorovbackwards_solver/
• https://neuralpde.sciml.ai/dev/examples/kolmogorovbackwards/

are HighDimPDE.PIDEProblem where the nonlinear term is 0.
Associated code and documentation should be migrated to HighDimPDE.jl, which is a more appropriate package to host those types of problem.

Similarly, the optimal stopping problem solver NeuralPDE.NNStopping, documented in

• https://neuralpde.sciml.ai/dev/solvers/optimal_stopping/
• https://neuralpde.sciml.ai/dev/examples/optimal_stopping_american/

should be transferred to HighDimPDE.jl.

How many paths are used with the DeepSplitting algorithm?

Hi,
Is there a parameter to control how many paths are simulated within the Deep Splitting algorithm?
Furthermore, is there a parameter that controls how many points (x_0) are generated when solving the problem on a hypercube?

NLFunction?

https://github.com/vboussange/HighDimPDE.jl/blob/main/src/HighDimPDE.jl#L15-L18

Why does this exist? This doesn't seem like it would do anything useful.