lululxvi/deeponet

Learning nonlinear operators via DeepONet

DeepONet: Learning nonlinear operators

The source code for the paper L. Lu, P. Jin, G. Pang, Z. Zhang, & G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3, 218-229, 2021.

System requirements

Most code is written in Python 3, and depends on the deep learning package DeepXDE. Some code is written in Matlab (version R2019a).

Installation guide

1. Install Python 3
2. Install DeepXDE v0.11.2 (https://github.com/lululxvi/deepxde). If you use DeepXDE>0.11.2, you need to rename OpNN to DeepONet and OpDataSet to Triple with other modifications. For DeepONet code using a more recent version of DeepXDE, please see https://github.com/lu-group/deeponet-fno.
3. Optional: For CNN, install Matlab and TensorFlow 1; for Seq2Seq, install PyTorch

The installation may take between 10 minutes and one hour.

Demo

Case Antiderivative

1. Open deeponet_pde.py, and choose the parameters/setup in the functions main() and ode_system() based on the comments;
2. Run deeponet_pde.py, which will first generate the two datasets (training and test) and then train a DeepONet. The training and test MSE errors will be displayed in the screen.

A standard output is

Building operator neural network...
'build' took 0.104784 s

Generating operator data...
'gen_operator_data' took 20.495655 s

Generating operator data...
'gen_operator_data' took 168.944620 s

Compiling model...
'compile' took 0.265885 s

Initializing variables...
Training model...

Step      Train loss    Test loss     Test metric
0         [1.09e+00]    [1.11e+00]    [1.06e+00]
1000      [2.57e-04]    [2.87e-04]    [2.76e-04]
2000      [8.37e-05]    [9.99e-05]    [9.62e-05]
...
50000     [9.98e-07]    [1.39e-06]    [1.09e-06]

Best model at step 46000:
  train loss: 6.30e-07
  test loss: 9.79e-07
  test metric: [7.01e-07]

'train' took 324.343075 s

Saving loss history to loss.dat ...
Saving training data to train.dat ...
Saving test data to test.dat ...
Restoring model from model/model.ckpt-46000 ...

Predicting...
'predict' took 0.056257 s

Predicting...
'predict' took 0.012670 s

Test MSE: 9.269857471315847e-07
Test MSE w/o outliers: 6.972881784590493e-07

You can get the training and test errors in the end of the output.

The run time could be between several minutes to several hours depending on the parameters you choose, e.g., the dataset size and the number of iterations for training.

Case Stochastic ODE/PDE

1. Open sde.py, and choose the parameters/setup in the functions main();
2. Run sde.py, which will generate traning and test datasets;
3. Open deeponet_dataset.py, and choose the parameters/setup in the functions main();
4. Run deeponet_dataset.py to train a DeepONet. The training and test MSE errors will be displayed in the screen.

Case 1D Caputo fractional derivative

1. Go to the folder fractional;
2. Run Caputo1D.m to generate training and test datasets. One can specify the orthongonal polynomial to be Legendre polynomial or poly-fractonomial in Orthogonal_polynomials.m. Expected run time: 20 mins.
3. Run datasets.py to pack and compress the genrated datasets. Expected outputs: compressed .npz files. Expected run time: 5 mins.
4. Run DeepONet_float32_batch.py to train and test DeepONets. Expected outputs: a figure of training and test losses. Expected run time: 1 hour.

Case 2D fractional Laplacian

Learning a 2D fractional Laplacian using DeepONets

1. Run Fractional_Lap_2D.m to generate training and test datasets. Expected outputs: text files that store the training and test data. Expected run time: 40 mins.
2. Run datasets.py to pack and compress the genrated datasets. Expected outputs: compressed .npz files. Expected run time: 15 mins.
3. Run DeepONet_float32_batch.py to train and test DeepONets. Expected run time: 3 hours.

Learning a 2D fractional Laplacian using CNNs

1. Suppose that the text files containing all training and test sets have been generated in the previous step.
2. Run CNN_operator_alpha.py to train and test CNNs. Expected outputs: a figure of training and test losses. Expected run time: 30 mins.

Seq2Seq

1. Open seq2seq_main.py, choose the problem in the function main(), and change the parameters/setup in the corresponding function (antiderivative()/pendulum()) if needed.
2. Run seq2seq_main.py, which will first generate the dataset and then train the Seq2Seq model on the dataset. The training and test MSE errors will be displayed in the screen. Moreover, the loss history, generated data and trained best model will be saved in the direction ('./outputs/').

A standard output is

Training...
0             Train loss: 0.21926558017730713         Test loss: 0.22550159692764282
1000       Train loss: 0.0022761737927794456     Test loss: 0.0024939212016761303
2000       Train loss: 0.0004760705924127251     Test loss: 0.0005566366016864777
...
49000     Train loss: 1.2885914202342974e-06    Test loss: 1.999963387788739e-06
50000     Train loss: 1.1382834372852813e-06    Test loss: 1.8525416862757993e-06
Done!
'run' took 747.5421471595764 s
Best model at iteration 50000:
Train loss: 1.1382834372852813e-06 Test loss: 1.8525416862757993e-06

You can get the training and test errors in the end of the output.

The run time could be between several minutes to several hours depending on the parameters you choose, e.g., the dataset size and the number of iterations for training.

Instructions for use

The instructions for running each case are as follows.

• Legendre transform: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
• Antiderivative: In Demo.
• Fractional (1D): In Demo.
• Fractional (2D): In Demo.
• Nonlinear ODE: The same as Antiderivative in Demo. You need to modify the functions main() and ode_system() in deeponet_pde.py.
• Gravity pendulum: The same as Antiderivative in Demo. You need to modify the functions main() and ode_system() in deeponet_pde.py.
• Diffusion-reaction: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
• Advection: The same as Antiderivative in Demo. You need to modify the functions main() in deeponet_pde.py, run() in deeponet_pde.py, CVCSystem() in system.py, and solve_CVC() in CVC_solver.py to run each case.
• Advection-diffusion: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
• Stochastic ODE/PDE: In Demo.

Cite this work

If you use this code for academic research, you are encouraged to cite the following paper:

@article{lu2021learning,
  title   = {Learning nonlinear operators via {DeepONet} based on the universal approximation theorem of operators},
  author  = {Lu, Lu and Jin, Pengzhan and Pang, Guofei and Zhang, Zhongqiang and Karniadakis, George Em},
  journal = {Nature Machine Intelligence},
  volume  = {3},
  number  = {3},
  pages   = {218--229},
  year    = {2021}
}

Questions

To get help on how to use the data or code, simply open an issue in the GitHub "Issues" section.

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

deeponet's Issues

ImportError: cannot import name 'GRF' from 'spaces' (D:\miniconda3\lib\site-packages\spaces\__init__.py)

I have some problem while I run the code deeponet_ode.py:

---

ImportError Traceback (most recent call last)
C:\Users\ALEXYA~1\AppData\Local\Temp/ipykernel_12048/4004186280.py in
9
10 import deepxde as dde
---> 11 from spaces import FinitePowerSeries, FiniteChebyshev, GRF
12 # from spaces import GRF
13 from system import LTSystem, ODESystem, DRSystem, CVCSystem, ADVDSystem

ImportError: cannot import name 'FinitePowerSeries' from 'spaces' (D:\miniconda3\lib\site-packages\spaces_init_.py)

I have run pip install spaces in the virtual environment, then I got this error. I check the file 'spaces_init_.py', there is nothing in the file.
Is there anything I miss?

---

I found that is because the code run in the wrong folder. The module 'spaces' is actually involved in it.
Sorry for the dumb question.

Follow up question for #32

Hi,
As suggested in the reply for #32, I modified the function safe_test

y_pred.append(model.predict(dde.nn.tensorflow_compat_v1.nn.NN.apply_feature_transform(self,X_add)))

but it gives an error saying

TypeError: apply_feature_transform() missing 1 required positional argument: 'transform'

First of all, I am not sure if this is how I am supposed to change the code based on the answer for #32.

If I put apply_feature_transform(self,X_add)),

and it gives

NameError: name 'self' is not defined

I see in nn.py, self is defined. What should I do in this case?

Please refer the full code below. Thanks.

def safe_test(model, data, X_test, y_test, fname=None):
       
    def is_nonempty(X):
        return len(X[0]) > 0 if isinstance(X, (list, tuple)) else len(X) > 0
    
    y_pred = []
    X = X_test
    while is_nonempty(X):
        X_add, X = trim_to_65535(X)
        
        #Original Code
        #y_pred.append(model.predict(data.transform_inputs(X_add))) 
        
        #Modified Code
        y_pred.append(model.predict(dde.nn.tensorflow_compat_v1.nn.NN.apply_feature_transform(self,X_add)))
        
                
    y_pred = np.vstack(y_pred)
    error = np.mean((y_test - y_pred) ** 2)
    print("Test MSE: {}".format(error))
    error = mean_squared_error_outlier(y_test, y_pred)
    print("Test MSE w/o outliers: {}\n".format(error))

    if fname is not None:
        np.savetxt(fname, np.hstack((X_test[1], y_test, y_pred)))

AttributeError: module 'config' has no attribute 'processes'

Hi there,

When I run deeponet_pde.py, I get an error message that

AttributeError: module 'config' has no attribute 'processes'

It is because in spaces.py, eval_u function has this line

p = ProcessPool(nodes=config.processes)

In this case, how should I modify the code?

Thanks in advance!

About the difference between the term "generalization error" and "test error" in the deeponet paper

Dear Dr Lu,
Hello, I want to ask a question about the difference between the term "generalization error" and "test error" in the deeponet paper.
Here, you wrote 'the error of a neural network for previously unseen data', so I understood it as same as 'test error',

also, in some of the web results 1 2 seem also approve that: generalization error and test error are the same thing.

However, in fig. 3 abcd you gave subfigures with the test error and generalization error independently, that's why?

About one-standard-deviation

Hi, Professor Lu. I want to ask two questions about the error bars and shaded region in the DeepONet paper.

• Did you use "uncorrected sample standard deviation" (divided by N) or "corrected sample standard deviation" (divided by N-1) to plot shaded region and error bar in the paper?
• Which software did you use for graphing the error bars and shaded region?
• Thank you!

AttributeError: module 'tensorflow' has no attribute 'trainable_variables'

Traceback (most recent call last):
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 285, in
main()
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 281, in main
run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 173, in run
print("# Parameters:", np.sum([np.prod(v.get_shape().as_list()) for v in tf.trainable_variables()]))
AttributeError: module 'tensorflow' has no attribute 'trainable_variables'

how to solve this error?

Selection of function u

Hello, author, why do you sample in Gaussian function space and Chebyshev function space

two-dimensional PDE problem of deeponet

Thank you for your previous answer. The problem of model recovery has been solved. But I have another question.Can we use deeponet to solve the two-dimensional PDE problem? If so, can I input a list of X and Y in the Trunk net?

How is time encoded in the pendulum example?

Hi,

I would like to use DeepOnet to analyze a dataset that consists of measurements done in several locations simultaneously. Thus, the shape of the input array is [time_points, spatial_coordinate].

I thought I would look at the pendulum example to understand how I can use DeepOnet for my application, but I am confused by the dimensions in the generated data. I am using the default parameters to generate the data:

# data
T = 3
s0 = [0, 0]
sensor_in = 100
sensor_out = 100
length_scale = 0.2
train_num = 1000
test_num = 10000
# seq2seq
cell = 'GRU' # 'RNN', 'LSTM' or 'GRU'
hidden_size = 5
# training
lr = 0.001
iterations = 100000
print_every = 1000

The generated arrays have shapes:

X_train: (1000, 100, 1)
y_train: (1000, 100, 1)

I assumed this array would be 1000 time points for 100 locations, but a quick inspection of X shows that none of the dimensions seems to indicate time as it is discontinuous. How is time encoded in this experiment? Is it assumed to be uniformly sampled?

Also, if you have any suggestions on how I could use DeepOnet to analyze the data I described above, I would highly appreciate it.

Thanks!

How advanced DeepOnet can be compatible with simple Fully-connected networks

Hello Dr. Lu,
Thank you very much for the DeepXDE you developed. I would like to ask you a question about DeepOnets.
It is reasonable that advanced DeepOnet can be compatible with simple FNN.
For example, in a simple house price prediction problem, we have 2 inputs, like the size and age of the house. Then, we want to predict the price of the house.
There seems to be no trunk net counterpart in this example. So, how do we implement a simple FNN with DeepOnet?
I do know that I can use FNN directly. I'm still trying to figure out how to be compatible with a simple network using DeepOnet architecture.
Could you please answer my confusion? I am looking forward to your reply!

ValueError: The passed save_path is not a valid checkpoint: model/model.ckpt-43000

How to fix this problem?

raceback (most recent call last):
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 285, in
main()
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 281, in main
run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
File "D:/Desktop/Codes/deeponet-master/deeponet-master/src/deeponet_pde.py", line 176, in run
model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
File "D:\Users\FIVE\miniconda3\lib\site-packages\deepxde\model.py", line 666, in restore
self.saver.restore(self.sess, save_path)
File "D:\Users\FIVE\miniconda3\lib\site-packages\tensorflow\python\training\saver.py", line 1290, in restore
raise ValueError("The passed save_path is not a valid checkpoint: " +
ValueError: The passed save_path is not a valid checkpoint: model/model.ckpt-43000

AttributeError: module 'tensorflow' has no attribute 'gloabl_variables_initializer'

Hello！
When I run DeepONet_float32_batch.py, I get an error message that:
Traceback (most recent call last):
File "E:\AIss\DeepOnet\deeponet-master\deeponet-master\fractional\DeepONet_float32_batch.py", line 208, in
sess.run(tf.gloabl_variables_initializer())
File "D:\software\anaconda\envs\tensorflow1.x\lib\site-packages\tensorflow_core\python\util\module_wrapper.py", line 193, in getattr
attr = getattr(self._tfmw_wrapped_module, name)
AttributeError: module 'tensorflow' has no attribute 'gloabl_variables_initializer'

I tried to modify this line of code:sess.run(tf.compat.v1.gloabl_variables_initializer()), but failed.

In this case, how should I modify the code?
Thanks in advance!

If the input of the trunk net could be set to be different

Hello Dr. Lu,

I am Bo Chen from SJTU. We had some communication before in online
conference, and I have read your code of the article"A comprehensive and fair
comparison of two neural operators (with practical extensions) based on FAIR
data". It is really wonderful work and I have learnt a lot from it.

But it can be found that in the cases you offered, the input of the trunk
net is always the same distribution for every input of the branch net so that
the data can be set by "dde.data.TripleCartesianProd()" with a grid of only one
distribution.

However in my application, the input of the trunk net must be different
distribution because of the data limitation of labels. This situation is just
mentioned in your article as case I or case II ,which means different u have the
different locations of ξ.

I have made same attempts but it seems that the shape of data_trunknet
is fixed and hard to be changed. And I also see a type of error
"ValueError("DeepONet does not support setting trunk net input.")" in the
code even though I did not meet this kind of error in my attempts.

So I wonder if there is a method of that could feed the input of the trunk
net with different distribution to DEEPONET. If possible, could you give me
some guidance or a simple demo is better.

Thanks very much!

Looking forword to your reply!

q

How to improve the result

Hi, I am working on deeponet_pde.py code to learn the operator from u -> I for ODE dI/dt = u * gamma * (1 - I) * I - gamma * I.
In deeponet_pde.py, I basically only modified the ode_system(T) but the result is really bad even I increase the number of training and test set. At this point, I think maybe the ode I am working with is not proper ode for the code. I am not sure what to modify.

Here is my code below. Any suggestion will be appreciated!

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import itertools

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

import deepxde as dde
from spaces import FinitePowerSeries, FiniteChebyshev, GRF
from system import LTSystem, ODESystem, DRSystem, CVCSystem, ADVDSystem
from utils import merge_values, trim_to_65535, mean_squared_error_outlier, safe_test

import scipy
from scipy.integrate import solve_ivp

# Define the length of period
T = 29

def test_u_ode(nn, system, T, m, model, data, u, fname, num=100):
    """Test ODE"""
    
    sensors = np.linspace(0, T, num=m)[:, None]
    sensor_values = u(sensors)
    
    x = np.linspace(0, T, num=num)[:, None]
    X_test = [np.tile(sensor_values.T, (num, 1)), x]
    y_test = system.eval_s_func(u, x)
    
    if nn != "opnn":
        X_test = merge_values(X_test)
        
    y_pred = model.predict(data.transform_inputs(X_test))
    np.savetxt(fname, np.hstack((x, y_test, y_pred)))
    print("L2relative error:", dde.metrics.l2_relative_error(y_test, y_pred))
    
    return X_test[1], y_pred


def ode_system(T):

    def g(s, u, x):
                  
        gamma = 0.1
        
        return u * gamma * (1-s) * s - gamma * s
        
    s0 = [0.1] 
    
    return ODESystem(g, s0, T)



def solve_ode(u, t_range):
   """
   Solve the ordinary differential equation (ODE) dI/dt = u * gamma * (1 - I) * I - gamma * I
   over the range of `t` values specified by `t_range`. The solution is used
   to compare with the prediction of DeepONet.
   """
   # Define the initial condition of I
   gamma = 0.1
   
   I_0 = 0.1
   
   def ode_func(t, I):
       return u * gamma * (1 - I) * I - gamma * I

   sol = solve_ivp(ode_func, t_range, [I_0], dense_output=True)

   t = sol.t
   I = sol.y[0]
   
   return t, I


t_range = [0, T]  # Start and end time
    
   
def run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test):
    # space_test = GRF(1, length_scale=0.1, N=1000, interp="cubic")

    X_train, y_train = system.gen_operator_data(space, m, num_train)

    
    X_test, y_test = system.gen_operator_data(space, m, num_test)
    if nn != "opnn":
        X_train = merge_values(X_train)
        X_test = merge_values(X_test)


    X_test_trim = trim_to_65535(X_test)[0]
    y_test_trim = trim_to_65535(y_test)[0]
    
    if nn == "opnn":
        data = dde.data.OpDataSet(
            X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
        )
    else:
        data = dde.data.DataSet(
            X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
        )

    model = dde.Model(data, net)
    model.compile("adam", lr=lr, metrics=[mean_squared_error_outlier])
    checker = dde.callbacks.ModelCheckpoint(
        "model/model.ckpt", save_better_only=True, period=10
    )
    losshistory, train_state = model.train(epochs=epochs, callbacks=[checker])
    print("# Parameters:", np.sum([np.prod(v.get_shape().as_list()) for v in tf.compat.v1.trainable_variables()]))
    dde.saveplot(losshistory, train_state, issave=True, isplot=True)

    model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
    safe_test(model, data, X_test, y_test)
    
    
    # Function to test the trained model
    
    u1 = 2.0
    u2 = 1.0
    
    t1,I1 = solve_ode(u1, t_range)
    t2,I2 = solve_ode(u2, t_range)
    
    tests = [
        (lambda x: u1*np.ones_like(x), "x1.dat", t1, I1),
        (lambda x: u2*np.ones_like(x), "x2.dat", t2, I2),
    ]
    
    for u, fname, x, sol_s in tests:

        if problem == "ode":
            
            # Predicted solution of trained model
            x_pred, y_pred = test_u_ode(nn, system, T, m, model, data, u, fname)
            
            plt.figure()
            plt.title('%s' %fname)
            plt.plot(x_pred,y_pred,label = 'Predicted')
            plt.plot(x,sol_s, label = 'True')
            plt.xlabel('x')
            plt.ylabel('s(x)')
            plt.legend()
            
            

def main():

    problem = "ode"
    

    if problem == "ode":
        system = ode_system(T)
    

    # Function space

    space = GRF(T, length_scale=0.1, N=3000, interp="cubic")


    # Hyperparameters
    
    # Number of sensors
    m = 15

    # Number of training set
    num_train = 500
    
    # Number of test set
    num_test = 500
    
    # Learning rate 
    lr = 0.01
    
    # Number of epochs
    epochs = 100

    # Network
    nn = "opnn"
    activation = "relu"
    initializer = "Glorot normal"  # "He normal" or "Glorot normal"
    dim_x = 1 
    if nn == "opnn":
        net = dde.maps.OpNN(
            [m, 40, 40],
            [dim_x, 40, 40],
            activation,
            initializer,
            use_bias=True,
            stacked=False,
        )
    elif nn == "fnn":
        net = dde.maps.FNN([m + dim_x] + [100] * 2 + [1], activation, initializer)
    elif nn == "resnet":
        net = dde.maps.ResNet(m + dim_x, 1, 128, 2, activation, initializer)

    run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)

    

if __name__ == "__main__":
    main()

And this is the prediction.

High test loss for nonlinear ODE case

I modified both main() and ode_system() as stated in the instructions, but I get a very high test loss. Do I have to modify other functions? I also increased the number of epochs to 10^5 as pointed in the supplementary information of the cited paper (a very interesting work I have to say).
Thank you in advance.

Additional information:
Modified lines (all in src/deeponet_pde.py):
line 104: uncommented after having commented line 102
line 231: be sure to have problem="ode"
line 260: increased epochs to 100000

Example of output:

Step      Train loss    Test loss     Test metric   
0         [3.68e-01]    [1.74e+25]    [3.55e-01]    
1000      [3.08e-03]    [1.74e+25]    [2.05e-03]    
2000      [8.55e-04]    [1.74e+25]    [8.00e-04]    
3000      [3.57e-04]    [1.74e+25]    [5.24e-04]    
4000      [2.06e-04]    [1.74e+25]    [3.58e-04]    
5000      [1.65e-04]    [1.74e+25]    [3.22e-04]    
6000      [1.43e-04]    [1.74e+25]    [2.83e-04]    
7000      [1.28e-04]    [1.74e+25]    [2.47e-04]    
8000      [1.01e-04]    [1.74e+25]    [2.26e-04]    
9000      [8.53e-05]    [1.74e+25]    [2.00e-04]    
10000     [7.95e-05]    [1.74e+25]    [1.87e-04]    
11000     [7.90e-05]    [1.74e+25]    [1.80e-04]    
12000     [9.09e-05]    [1.74e+25]    [1.88e-04]    
13000     [9.10e-05]    [1.74e+25]    [1.69e-04]    
14000     [5.61e-05]    [1.74e+25]    [1.45e-04]    
15000     [5.24e-05]    [1.74e+25]    [1.40e-04]    
16000     [5.50e-05]    [1.74e+25]    [1.44e-04]    
17000     [5.00e-05]    [1.74e+25]    [1.33e-04]    
18000     [5.21e-05]    [1.74e+25]    [1.39e-04]    
19000     [4.54e-05]    [1.74e+25]    [1.31e-04]    
20000     [1.13e-04]    [1.74e+25]    [1.78e-04]    
21000     [4.39e-05]    [1.74e+25]    [1.31e-04]    
22000     [5.68e-05]    [1.74e+25]    [1.45e-04]    
23000     [4.37e-05]    [1.74e+25]    [1.31e-04]    
24000     [4.18e-05]    [1.74e+25]    [1.28e-04]    
25000     [4.00e-05]    [1.74e+25]    [1.28e-04]    
26000     [6.98e-05]    [1.74e+25]    [1.55e-04]    
27000     [7.45e-05]    [1.74e+25]    [1.58e-04]    
28000     [4.10e-05]    [1.74e+25]    [1.22e-04]    
29000     [3.62e-05]    [1.74e+25]    [1.20e-04]    
30000     [5.16e-05]    [1.74e+25]    [1.36e-04]    
31000     [3.60e-05]    [1.74e+25]    [1.19e-04]    
32000     [3.82e-05]    [1.74e+25]    [1.20e-04]    
33000     [3.33e-05]    [1.74e+25]    [1.14e-04]    
34000     [3.36e-05]    [1.74e+25]    [1.14e-04]    
35000     [3.24e-05]    [1.74e+25]    [1.11e-04]    
36000     [3.15e-05]    [1.74e+25]    [1.12e-04]    
37000     [3.14e-05]    [1.74e+25]    [1.08e-04]    
38000     [3.78e-05]    [1.74e+25]    [1.18e-04]    
39000     [3.40e-05]    [1.74e+25]    [1.09e-04]    
40000     [4.34e-05]    [1.74e+25]    [1.14e-04]    
41000     [3.58e-05]    [1.74e+25]    [1.09e-04]    
42000     [2.89e-05]    [1.74e+25]    [1.04e-04]    
43000     [2.99e-05]    [1.74e+25]    [1.09e-04]    
44000     [4.13e-05]    [1.74e+25]    [1.10e-04]    
45000     [2.72e-05]    [1.74e+25]    [1.03e-04]    
46000     [2.71e-05]    [1.74e+25]    [1.03e-04]    
47000     [2.88e-05]    [1.74e+25]    [1.02e-04]    
48000     [5.22e-05]    [1.74e+25]    [1.33e-04]    
49000     [2.55e-05]    [1.74e+25]    [9.98e-05]    
50000     [2.92e-05]    [1.74e+25]    [1.01e-04]    
51000     [2.63e-05]    [1.74e+25]    [9.88e-05]    
52000     [2.39e-05]    [1.74e+25]    [9.90e-05]    
53000     [2.76e-05]    [1.74e+25]    [1.02e-04]    
54000     [3.15e-05]    [1.74e+25]    [9.99e-05]    
55000     [3.11e-05]    [1.74e+25]    [1.11e-04]    
56000     [2.47e-05]    [1.74e+25]    [9.72e-05]    
57000     [3.14e-05]    [1.74e+25]    [1.08e-04]    
58000     [3.34e-05]    [1.74e+25]    [1.01e-04]    
59000     [2.54e-05]    [1.74e+25]    [9.68e-05]    
60000     [3.11e-05]    [1.74e+25]    [9.88e-05]    
61000     [4.05e-05]    [1.74e+25]    [1.20e-04]    
62000     [4.36e-05]    [1.74e+25]    [1.22e-04]    
63000     [2.21e-05]    [1.74e+25]    [9.88e-05]    
64000     [2.31e-05]    [1.74e+25]    [9.72e-05]    
65000     [3.04e-05]    [1.74e+25]    [9.75e-05]    
66000     [2.57e-05]    [1.74e+25]    [9.81e-05]    
67000     [2.77e-05]    [1.74e+25]    [9.60e-05]    
68000     [2.22e-05]    [1.74e+25]    [9.49e-05]    
69000     [2.21e-05]    [1.74e+25]    [9.86e-05]    
70000     [2.57e-05]    [1.74e+25]    [9.89e-05]    
71000     [2.21e-05]    [1.74e+25]    [9.48e-05]    
72000     [2.11e-05]    [1.74e+25]    [9.38e-05]    
73000     [2.57e-05]    [1.74e+25]    [1.02e-04]    
74000     [3.00e-05]    [1.74e+25]    [1.09e-04]    
75000     [2.15e-05]    [1.74e+25]    [9.34e-05]    
76000     [2.06e-05]    [1.74e+25]    [9.62e-05]    
77000     [3.68e-05]    [1.74e+25]    [1.16e-04]    
78000     [1.95e-05]    [1.74e+25]    [9.37e-05]    
79000     [2.38e-05]    [1.74e+25]    [9.98e-05]    
80000     [6.12e-05]    [1.74e+25]    [1.42e-04]    
81000     [2.97e-05]    [1.74e+25]    [1.08e-04]    
82000     [2.12e-05]    [1.74e+25]    [9.13e-05]    
83000     [2.11e-05]    [1.74e+25]    [9.13e-05]    
84000     [1.92e-05]    [1.74e+25]    [9.08e-05]    
85000     [4.75e-05]    [1.74e+25]    [1.28e-04]    
86000     [6.35e-05]    [1.74e+25]    [1.44e-04]    
87000     [1.88e-05]    [1.74e+25]    [9.50e-05]    
88000     [1.85e-05]    [1.74e+25]    [9.09e-05]    
89000     [1.74e-05]    [1.74e+25]    [9.12e-05]    
90000     [1.75e-05]    [1.74e+25]    [9.28e-05]    
91000     [3.79e-05]    [1.74e+25]    [1.05e-04]    
92000     [1.98e-05]    [1.74e+25]    [9.06e-05]    
93000     [2.04e-05]    [1.74e+25]    [9.09e-05]    
94000     [2.14e-05]    [1.74e+25]    [9.92e-05]    
95000     [1.70e-05]    [1.74e+25]    [9.22e-05]    
96000     [3.40e-05]    [1.74e+25]    [9.47e-05]    
97000     [1.80e-05]    [1.74e+25]    [9.49e-05]    
98000     [1.65e-05]    [1.74e+25]    [9.20e-05]    
99000     [1.67e-05]    [1.74e+25]    [8.97e-05]    
100000    [1.65e-05]    [1.74e+25]    [9.20e-05]    

Best model at step 98000:
  train loss: 1.65e-05
  test loss: 1.74e+25
  test metric: [9.20e-05]

Output as a Mutidimensional array

Dear Prof.Lu, l see that the example codes you put here are codes that deal with scalar outputs, by saying 'scalar' l mean at each position or time instance where the output function is evaluated, the function is a scalar function. What if my output function is a vector function? Meaning that at each position(or moment), my output is like a two dimensiona vector?

And l want to know that, if l am going to use the Physics Informed version of DeepONet, like discribed in the Science Advance paper, how am l gonna use deepxde library to achieve that? Looking forward to your reply and thanks!

ValueError: Cannot feed value of shape (100000, 225) for Tensor Placeholder:0, which has shape (?, 15)

thx for your excellent work!
when running "Learning a 2D fractional Laplacian using DeepONets" part, I encounter the following mistake:
Traceback (most recent call last):
File "C:\Users\lenovo\Desktop\deeponet\fractional\DeepONet_float32_batch.py", line 303, in
sess.run(train_op_Adam, feed_train_batch)
File "C:\Users\lenovo.conda\envs\sdsd_torch\lib\site-packages\tensorflow\python\client\session.py", line 970, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "C:\Users\lenovo.conda\envs\sdsd_torch\lib\site-packages\tensorflow\python\client\session.py", line 1167, in _run
raise ValueError(
ValueError: Cannot feed value of shape (100000, 225) for Tensor Placeholder:0, which has shape (?, 15)

DeepONet Triple Data Shape | if branch & trunk inputs are just time a Parametric Space

I have a similar question to #9.
Imagine you have a transient simulation of a 3D domain and you are interested in an averaged quantity, e.g., Maximum Velocity V(t) . Then you would repeat this for various parameters (a parametric space), like Inlet Pressure (many Pi) and Inlet Temperature (many Ti). To translate this to the DeepONet terminology, I guess:

G(u)(y) = V(t)(Pi,Ti)
That is to say,
u is t :: Input to Branch is time
(y) is (Pi, Ti) :: Input to Trunk is pair of (Pi, Ti)

Provided that the t array is always the same for all training, test, and future prediction values, I mean you are always interested in certain time values like [t1=0.1, t2=0.2, t3=0.3, ...], i.e., the objective is to predict V(t) always at the same times for any given (Pi,Ti),
then I wonder which of the following forms is correct for the DeepONet Triple Data:

X_train=
[
	[
		[ V(t1)(P1,T1), V(t2)(P1,T1), V(t3)(P1,T1)... ] ,
		[ V(t1)(P2,T2), V(t2)(P2,T2), V(t3)(P2,T2)... ] ,
		...
	],
	[
		[ t1, t2, t3... ] ,
		[ t1, t2, t3... ] ,
		...
	]
]

y_train=
[
	[P1 , T1] ,
	[P2 , T2] ,
	...
]

OR

X_train=
[
	[
		[ V(t1)(P1,T1) ] ,
		[ V(t2)(P1,T1) ] ,
		[ V(t3)(P1,T1) ] ,
		...
		[ V(t1)(P2,T2) ] ,
		[ V(t2)(P2,T2) ] ,
		[ V(t3)(P2,T2) ] ,
		...
	],
	[
		[t1] ,
		[t2] ,
		[t3] ,
		...
		[t1] ,
		[t2] ,
		[t3] ,
		...
	]
]

y_train=
[
	[P1 , T1] ,
	[P1 , T1] ,
	[P1 , T1] ,
	...
	[P2 , T2] ,
	[P2 , T2] ,
	[P2 , T2] ,
	...
]

Basically, train & predict over the entire [t] array together? or train & predict per each t value?
or both of them are totally wrong?

Summary:

• Time: [t1, t2, t3,..., tn]
• Parameters: [ [P1,T1] , [T2, P2] , ... , [Tm,Pm] ]

I guess in some form:

• branch: Time would be the input to the branch net.
• trunk: Parameters would be the input to the branch net.

Am I correct?

Thanks!

Originally posted by @cfd-ai in #9 (comment)

KeyError: 'Glorot normalstacked'

Hi,sir.
It confused me a lot that if I change 'stacked' to True, error happens:

Traceback (most recent call last):
File "C:/Users/lenovo/PycharmProjects/pythondeep/deeponet_pde.py", line 286, in
main()
File "C:/Users/lenovo/PycharmProjects/pythondeep/deeponet_pde.py", line 269, in main
net = dde.maps.DeepONet(
File "C:\Users\lenovo\anaconda3\lib\site-packages\deepxde\nn\tensorflow_compat_v1\deeponet.py", line 60, in init
self.kernel_initializer_stacked = initializers.get(
File "C:\Users\lenovo\anaconda3\lib\site-packages\deepxde\nn\initializers.py", line 185, in get
return INITIALIZER_DICT[identifier]
KeyError: 'Glorot normalstacked'

Process finished with exit code 1

Could you please give me a help! thanks alot

input data format for OpNN

Hi Lu,
Thanks for your work. I am trying to make some data for OpNN training. just for clarifying, should the input data be [initial condition, position] and the label be [true value w.r.t. input]?

Dimension issue in generating operator "system.py".

Hi,
First of all, thank you very much for your excellent work.

I tried to test the dimension-match in system.py for the case of simple linear ODE with the following parameters:

• The space that I choose is Finite Power Series with N = 100 and M = 1
• m: the number of sensors (So I get the sensors to be an array of m-by-1) Choose m = 50
• m_u: the number of u's ( the sample u is an array of m_u-by-m) Choose m_u = 60
• m_y: the number of y's ( an array of m_y-by-1) Choose m_y = 40

However, I get the dimension for s which value of G(u)(y) is an array of 40-by-1.
Should this value have to be an array of (m_u * m_y)-by-1 which is in this case 2400-by-1?

This is my modified part:

def gen_operator_data(self,space,m, m_u, m_y):
        print("GENERATING THE DATA...", flush = True)           
        features= space.random(m_u) # build the features (m_u, N) check
        sensors = np.linspace(0, self.T, m)[:, None] # build the sensors (m,1) check 
        u = space.eval_u(features_train, sensors) # get the sample u's (m_u, m) check
        x  = self.T * np.random.rand(m_y)[:,None] # get the sample y's (m_y, 1) check
        s = self.eval_op_space(space, features, x) # get the values s'  
        Xdata = [u, x] 
        Ydata = s
        return Xdata, Ydata

I look forward to hearing from you soon.

Best Regards.

Is the DeepONet predict solution more efficient than the original method?

I generage a dataset from original method( finite difference methods for 2-D wave problem)，train a DeepONet and predict solutions.
But, it seems not to speed the predict efficient compare to original method.
I notice many article about DeepONet has the 1000x speed up than original method.
What can i do to improve it?

Thanks a lot for your help.

AttributeError: 'Triple' object has no attribute 'transform_inputs'

Any idea how to resolve the following error?
This is for the default problem = "ode" test case.
Backend: tensorflow.compat.v1
TensorFlow version: 2.6.2
Thanks.

Restoring model from model/model.ckpt-37000.ckpt ...

Traceback (most recent call last):
File "deeponet_pde.py", line 285, in
main()
File "deeponet_pde.py", line 281, in main
run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
File "deeponet_pde.py", line 177, in run
safe_test(model, data, X_test, y_test)
File "/[HOME-PATH]/tests/deeponet/src/utils.py", line 56, in safe_test
y_pred.append(model.predict(data.transform_inputs(X_add)))
AttributeError: 'Triple' object has no attribute 'transform_inputs'

AttributeError: 'Triple' object has no attribute 'transform_inputs'

Hi, I have the same issue as #25 and my code is already modified for the new version. What can I use instead of 'transform_inputs'?

Tests not running properly

Hi @lululxvi ,
Thanks for putting together deepxde. I was trying to get started with ODESystem but noticed a couple of things that are off. First off, here is my version info:

In [1]: import tensorflow, deepxde

In [2]: tensorflow.__version__
Out[2]: '2.6.0'

In [3]: deepxde.__version__
Out[3]: '0.13.5'

Now if I try to run the gravity pendulum ODE, I get a weird type mismatch error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-359904db8a4e> in <module>()
     47 
     48 if __name__ == "__main__":
---> 49     main()

14 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    992           except Exception as e:  # pylint:disable=broad-except
    993             if hasattr(e, "ag_error_metadata"):
--> 994               raise e.ag_error_metadata.to_exception(e)
    995             else:
    996               raise

TypeError: in user code:

    /usr/local/lib/python3.7/dist-packages/deepxde/model.py:169 outputs_losses  *
        losses = self.data.losses(targets, outputs_, loss_fn, self)
    /usr/local/lib/python3.7/dist-packages/deepxde/data/triple.py:32 losses  *
        return [loss(targets, outputs)]
    /usr/local/lib/python3.7/dist-packages/deepxde/losses.py:27 mean_squared_error  *
        return bkd.reduce_mean(bkd.square(y_true - y_pred))
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:1383 binary_op_wrapper
        raise e
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:1367 binary_op_wrapper
        return func(x, y, name=name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206 wrapper
        return target(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:548 subtract
        return gen_math_ops.sub(x, y, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py:10654 sub
        "Sub", x=x, y=y, name=name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py:558 _apply_op_helper
        inferred_from[input_arg.type_attr]))

    TypeError: Input 'y' of 'Sub' Op has type float32 that does not match type float64 of argument 'x'.

Here is the piece of code that I am trying to run (I cloned and manually appended the path to the git repo to directly import some of the submodules)

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import itertools

import numpy as np
import sys, os

sys.path.append("deeponet/src/")

import deepxde as dde
from spaces import FinitePowerSeries, FiniteChebyshev, GRF
from system import LTSystem, ODESystem, DRSystem, CVCSystem, ADVDSystem
from utils import merge_values, trim_to_65535, mean_squared_error_outlier, safe_test

def test_u_ode(nn, system, T, m, model, data, u, fname, num=100):
    """Test ODE"""
    sensors = np.linspace(0, T, num=m)[:, None]
    sensor_values = u(sensors)
    x = np.linspace(0, T, num=num)[:, None]
    X_test = [np.tile(sensor_values.T, (num, 1)), x]
    y_test = system.eval_s_func(u, x)
    if nn != "opnn":
        X_test = merge_values(X_test)
    y_pred = model.predict(data.transform_inputs(X_test))
    np.savetxt(fname, np.hstack((x, y_test, y_pred)))
    print("L2relative error:", dde.metrics.l2_relative_error(y_test, y_pred))

def ode_system(T):
    """ODE"""

    def g(s, u, x):
        # Gravity pendulum
        k = 1
        return [s[1], -k * np.sin(s[0]) + u]

    s0 = [0, 0]  # Gravity pendulum
    return ODESystem(g, s0, T)

def run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test):
    # space_test = GRF(1, length_scale=0.1, N=1000, interp="cubic")

    X_train, y_train = system.gen_operator_data(space, m, num_train)
    X_test, y_test = system.gen_operator_data(space, m, num_test)
    if nn != "opnn":
        X_train = merge_values(X_train)
        X_test = merge_values(X_test)

    # np.savez_compressed("train.npz", X_train0=X_train[0], X_train1=X_train[1], y_train=y_train)
    # np.savez_compressed("test.npz", X_test0=X_test[0], X_test1=X_test[1], y_test=y_test)
    # return

    # d = np.load("train.npz")
    # X_train, y_train = (d["X_train0"], d["X_train1"]), d["y_train"]
    # d = np.load("test.npz")
    # X_test, y_test = (d["X_test0"], d["X_test1"]), d["y_test"]

    X_test_trim = trim_to_65535(X_test)[0]
    y_test_trim = trim_to_65535(y_test)[0]
    if nn == "opnn":
        data = dde.data.Triple(
            X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
        )
    else:
        data = dde.data.DataSet(
            X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
        )

    model = dde.Model(data, net)
    model.compile("adam", lr=lr, metrics=[mean_squared_error_outlier])
    checker = dde.callbacks.ModelCheckpoint(
        "model/model.ckpt", save_better_only=True, period=1000
    )
    losshistory, train_state = model.train(epochs=epochs, callbacks=[checker])
    print(
        "# Parameters:",
        np.sum([np.prod(v.get_shape().as_list()) for v in tf.trainable_variables()]),
    )
    dde.saveplot(losshistory, train_state, issave=True, isplot=True)

    model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
    safe_test(model, data, X_test, y_test)

    tests = [
        (lambda x: x, "x.dat"),
        (lambda x: np.sin(np.pi * x), "sinx.dat"),
        (lambda x: np.sin(2 * np.pi * x), "sin2x.dat"),
        (lambda x: x * np.sin(2 * np.pi * x), "xsin2x.dat"),
    ]
    for u, fname in tests:
        if problem == "ode":
            test_u_ode(nn, system, T, m, model, data, u, fname)

def main():
    # Problems:
    # - "lt": Legendre transform
    # - "ode": Antiderivative, Nonlinear ODE, Gravity pendulum
    # - "dr": Diffusion-reaction
    # - "cvc": Advection
    # - "advd": Advection-diffusion
    problem = "ode"
    T = 1
    system = ode_system(T)

    # Function space
    # space = FinitePowerSeries(N=100, M=1)
    # space = FiniteChebyshev(N=20, M=1)
    # space = GRF(2, length_scale=0.2, N=2000, interp="cubic")  # "lt"
    space = GRF(1, length_scale=0.2, N=1000, interp="cubic")
    # space = GRF(T, length_scale=0.2, N=1000 * T, interp="cubic")

    # Hyperparameters
    m = 100
    num_train = 10000
    num_test = 100000
    lr = 0.001
    epochs = 50000

    # Network
    nn = "opnn"
    activation = "relu"
    initializer = "Glorot normal"  # "He normal" or "Glorot normal"
    dim_x = 1 if problem in ["ode", "lt"] else 2
    if nn == "opnn":
        net = dde.maps.DeepONetCartesianProd(
            [m, 40, 40],
            [dim_x, 40, 40],
            activation,
            initializer,
            # use_bias=True,
            # stacked=False,
        )
    elif nn == "fnn":
        net = dde.maps.FNN([m + dim_x] + [100] * 2 + [1], activation, initializer)
    elif nn == "resnet":
        net = dde.maps.ResNet(m + dim_x, 1, 128, 2, activation, initializer)

    run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)

if __name__ == "__main__":
    main()

Taking a look briefly at the exports in deepxde.nn.tensorflow I think that the Installation Guide should point to DeepONetCartesianProd and not DeepONet, right? Or am I missing anything? Now even if I make that change, the code is not runnable as is. Do you have any inputs?

I will keep exploring the library, but it'd be great if you could clarify a bit on how to make the above snippet runnable.

Thanks!

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 38: invalid continuation byte

Hello, professor. I have some porblem.What's the way you coded?
I have the following error while running the code.

Traceback (most recent call last):
File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 285, in
main()
File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 281, in main
run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 176, in run
model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\deepxde\model.py", line 377, in restore
self.saver.restore(self.sess, save_path)
File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\training\saver.py", line 1289, in restore
if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix):
File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\training\checkpoint_management.py", line 383, in checkpoint_exists_internal
if file_io.get_matching_files(pathname):
File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 350, in get_matching_files
return get_matching_files_v2(filename)
File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 371, in get_matching_files_v2
compat.as_bytes(pattern))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 38: invalid continuation byte

Process finished with exit code 1

About activation function in deeponet

Can I add a new activation function to change the activation function in the trunk net?

Thank you very much if you can answer the questions！

two-dimensional PDE problem

Thank you for your previous answer. The problem of model recovery has been solved. But I have another question.Can we use deeoonet to solve the two-dimensional PDE problem? If so, can I input a list of X and Y in the Trunk net?

Setting of branch network input u in PDE

Hello, teacher.The input of branch network in the code is a one-dimensional u. if the source term U of PDE is a two-dimensional function, how to set u in the code.It's equivalent to u being a function of t and x.

Number of parameters in `seq2seq` scales with the length**2 of the sequence

deeponet/seq2seq/learner/nn/seq2seq.py

Lines 58 to 59 in 44e7b5a

	def __init_att_weights(self):
	return torch.nn.Parameter(torch.zeros([self.len_out, self.len_in]))

This does not seem right, considering DeepONet does not have such kind of dependence.

deeponet/seq2seq/learner/nn/deeponet.py

Lines 39 to 41 in 44e7b5a

	modules['Branch'] = FNN(self.branch_dim, self.width, self.branch_depth, self.width,
	self.activation, self.initializer)
	modules['TrLinM1'] = nn.Linear(self.trunk_dim, self.width)

Using operator learning if dimension of input function is more than 1

The examples given for DeepOnet use data sets with only one input function. What should I modify to use more than one input function?

DeepONetCartesianProd vs DeepONet

Hi, Sir
I was confused on what is the difference between DeepONet and DeepONetCartesian Product, could you please explain briefly that:-
Q1. How they both are different from each other?
Q2. What would change while using the two?

Could a function of Lp space(p=2) be branch input？

Dear Professor.Lulu:

As title.
I notice you employ KL expansion in Learning stochastic operators, and use N eigenfunctions scaled by the eigenvalues as the input of branch net.

My problem is about Sturm-Liouville problem, the solution is also consist of eigenfunctions and eigenvalues.

Is that mean i should train model for the problem with different number of eigenvalues, and the input of branch net is N orthogonal functions?
If there is no random process, the input of the branch net is function u of variable x, the function is sample from Lp space(p=2) .It's OK use the function u(x) instead of Fourier-Bessel expansion?

The data shape of the triplet if the number of input functions of branch nets is more than one?

Hello:
First of all, thank you very much for your excellent work!
I have met a little problem.
How to reshape the dataset (or change the code) if the number of input functions of branch nets is more than one?
Just like G_phi in this paper("DeepM&Mnet: Inferring the electroconvection multiphysics fields..."), (c+, c-) -->phi.
Or like G_rhoNO in this paper("DeepM&Mnet for hypersonics..."), (rho_N2, rho_O2) --> rho_NO.
Specifically, what about the data shape of the triplet?
Thanks in advance.

How to use the trained model to predict without training?

Hello Dr. Lu,
Thank you very much for putting forward this DeepONet for our study.I'm very interested in DeepONet .
But now I have a problem.I spent a lot of time generating datasets and training on reaction-diffusion equations. I save the training data set and the best model after training. But how can I use this optimal model to predict the solution of the reaction-diffusion equation without training? How can I modify the code in deeponet_pde.py?
Thank you very much for your time.

More than one input function maps to more than one output function by DeepONet

Hello Dr. Lu,
It is very nice to get your reply.
Following your answer, I think the data of the branch nets may be like this:
[ u1(x1), ..., u1(xm) ] for one input function u1;
[ u1(x1), ..., u1(xm), u2(x1), ..., u2(xm) ] for two input function u1 & u2.
Or like this:
[ u1(x1), u2(x1),..., u1(xm), u2(xm) ] ?
I think the former seems to be the case. Do I have the correct understanding?
I am looking forward to your reply.
Thanks in advance.

Originally posted by @WangYicunZJU in #9 (comment)

AttributeError: 'Triple' object has no attribute 'transform_inputs'

Hi,

I have opened this issue wrongly! Please delete it!

Questions about the difference PINN and DeepXDE

Dear Prof. Lu, Thanks for your time for looking at this question.

1. Could you tell me is the DeepXDE is one type of PINN? But I didn't find any code about using control equation to describe the loss in DeepXDE.
2. Currently, I am solving a equations contained possion and transport equations. I want to obtain the dynamic process of the electron density change. Should I use the (x,y,t) and (E,ne,np,nn) as the input and output, respectively. Can the DeepXDE predict the extra time results without trained time data as input.

How DeepONet provides multiple outputs

Professor Lu:
For a complex output problem, I divided into two networks with the same structure and hyperparameters for training.
The previous issue has been resolved.
So, i need put model1 and model2 in two files.
I use step method to reduce the dimension of Trunk net, the input of each step is the output of the two networks of the previous step. So there is a bug. I think it is necessary to train a network with two outputs at the same time.
May i ask some relevant examples so i can change my code?

AttributeError: 'Triple' object has no attribute 'transform_inputs'

model.restore("model/model.ckpt-" + str(train_state.best_step) + ".ckpt", verbose=1)
#model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
safe_test(model, data, X_test, y_test)

Saving loss history to D:\LULU\deeponet-master1\deeponet-master\src\loss.dat ...
Error: The network has multiple inputs, and saving such result han't been implemented.
Error: The network has multiple inputs, and plotting such result han't been implemented.
Restoring model from model/model.ckpt-5000.ckpt ...

Traceback (most recent call last):
File "D:/LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 285, in
main()
File "D:\LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 281, in main
run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
File "D:\LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 177, in run
safe_test(model, data, X_test, y_test)
File "D:\LULU\deeponet-master1\deeponet-master\src\utils.py", line 56, in safe_test
y_pred.append(model.predict(data.transform_inputs(X_add)))
AttributeError: 'Triple' object has no attribute 'transform_inputs'

进程已结束,退出代码1

How to solve 3d ode equations

Hi,
First of all, thank you very much for your excellent work.

In case Gravity pendulum, how do I print the predicted value of the variable s2?

I also try to solve a 3D ode equation set, like ds1/dt=g1(x),ds2/dt=g2(x),ds3/dt=g3(x).It works very well, but only outputs the s1 in test ode function.

Looking forward to your reply.

Thank you very much.