bessagroup/f3dasm

Framework for Data-Driven Design & Analysis of Structures & Materials (F3DASM)

BessaGroup

Website for the research group based on the JekyllSerif Theme.

Installing Ruby & Jekyll

If this is your first time using Jekyll, please follow the Jekyll docs and make sure your local environment (including Ruby) is setup correctly.

How to Collaborate

1. Fork the repository to your account
2. Create a new branch
3. Do some contribution/improvement
4. Test changes locally
5. Create a pull request to original repository

To run the repo locally, modify the baseurl: in the _cofig.yml. Change the value to ''

Then, navigate to the theme directory and run:

bundle install

To start the Jekyll local development server.

bundle exec jekyll serve

To build the theme.

bundle exec jekyll build

f3dasm's Issues

why is this check necessary?

    ## TODO: why is this check necessary?

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/base/utils.py#L24

Feedback of F3dasm

1. Overall I like the framework of F3dasm and I am trying to use f3dasm now.
2. The problem when I am using is that I can't have any keywords hints on my Desktop, which makes my feeling on using f3dasm is not good. Because every step I want to use, I need to look at the source code.
3. The documentation is also good, but it seems you didn't update it with the lasted version. For example, for make constinous design varibales, the keyword in the documenation is 'dimension', however, the correct keyword in the source code is 'dimensionalitty'.
4. For my side, I really want to save my responses of output for every point after their calculation because it may have some problems getting the responses from FEM. If we don't save it, then we have to restart the repsonse calculation from the very beginning once there are errors occurred. However, it seems in this version, for the DoE, it only save all response to f3dam.data after getting all responses of DoE. It will be very risk for FeM involved responses calculation.

Overall, I think comments 2 and 3 are more urgent for the user, because it makes is not so friendly to user.

Improve package structure

Implement a new and more modular structure for the package based on the discussion and the group's agreements on 25-October

Agree on data structures for Gawel's problem case

The case Gawel's is currently working on requires defining problems and geometries in ways different than the cases for suppercompressibles and hyperelastic.
We should discuss and agree on how to handle such cases and what changes to DoeVars.py are required.

Requirements file is missing

A requirements file is missing for this implementation. Must be added before the new release

Sampler random seed

rsampler = f3dasm.sampling.RandomUniformSampling(design, seed = 5)
samples_random = rsampler.get_samples(2)

gives the result

If I run samples_random = rsampler.get_samples(2) again, it gives a different set of samples.

So wouldn't it make more sense to add the seed option to the get_samples() function instead ?

Fix conversion to torch tensor

When passing non-numeric values to the DATA class, the method torch_sensor() produces the following error:

return torch.tensor(self.values).float()

TypeError: can't convert np.ndarray of type numpy.object_. 
The only supported types are: float64, float32, float16, complex64, 
complex128, int64, int32, int16, int8, uint8, and bool.

We need a fix for this. For now, we remove the line of code that triggers the error.

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L129

Implement more general data representation for DoE Variables

• representation shall use a dictionary as a base data structure
• users shall be free to define the names and values of the dictionary
• the representation shall implement a tree-like structure in which parameters that required sampling are at a single level at the top of the tree
• shall implement methods to normalize and serialize dictionary into a pandas dataframe

Documentation center edits

Personally, I delve into a documentation center when learning a new software. The current documentation center appears to have all the nuts and bolts (I like this), but the intuitive navigation of the code is not there for me. I want to suggest the following for improved readability:

• Provide more explanatory detail. Currently, when I open the documentation center, there is a large list of functions which are new and not readily understandable. For example, I see the "Function interface" tab, select it and am given one line "implement your own functions." This is not enough for me to understand what the point of this class is for.
• Include more examples. Examples show me as a user where these classes, objects, functions etc. can be used. The current "optimizing a benchmark function" is helpful, but is just one. Build more examples to show the variety of tasks which may be completed with this code base.

this function is incompatible with a 1D design

        # TODO: this function is incompatible with a 1D design, 

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/functions/pybenchfunction.py#L207

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L305

Suggestion to reduce complex structure in execution parts of the code

I feel that some parts of the execution code are structured in a complex way, and the intuition might be lost to the reader. Referring to an example of multiple realizations of SGA from the tutorial practical_session_solutions.ipynb:

# Define the blocks:
dimensionality = 20
iterations = 600
realizations = 10
hyperparameters= {} # If none are selected, the default ones are used
domain = np.tile([-1., 1.], (dimensionality,1))
design = f3dasm.make_nd_continuous_design(bounds=domain, dimensionality=dimensionality)
data = f3dasm.Data(design)

# We can put them in a dictionary if we want
implementation = {
'realizations': realizations,
'optimizer': f3dasm.optimization.CMAES(data=data, hyperparameters=hyperparameters), 
'function': f3dasm.functions.Ackley(dimensionality=dimensionality, noise=False, scale_bounds=domain),
'sampler': f3dasm.sampling.LatinHypercubeSampling(design, seed=seed),
'iterations': iterations,
}

results = f3dasm.run_multiple_realizations(**implementation)

It would be helpful if there aren't many complex arguments of domain, design, and data (which take one another as argument for instantiation) used again in dictionary implementation.

: Fix this ordering issue

https://github.com/bessagroup/F3DASM/blob/a9d097bd41ac47d1f8882c50ad31aae959b203c3/f3dasm/base/samplingmethod.py#L65

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L1103

correct formula should be np.sum((X - 1) ** 2) - np.sum(X[1:] * X[:-1]). Remove unused d and i variables.

    def evaluate(self, X): #TODO: correct formula should be np.sum((X - 1) ** 2) - np.sum(X[1:] * X[:-1]). Remove unused d and i variables.

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L2720

Implement Fenics example

Extend code in the simtulator to implement Fenics example using Taylan's code as starting point

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L165

add possibility to insert ONLY input numpy array (leave output array as NaNs)

    # TODO: add possibility to insert ONLY input numpy array (leave output array as NaNs)

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/base/data.py#L63

Same name for parameters when creating design space

x0 = f3dasm.ContinuousParameter(name='x0', lower_bound=-1.0, upper_bound=1.0)
x1= f3dasm.ContinuousParameter(name='x0', lower_bound=-1.0, upper_bound=1.0)

Accessing samples by

rsampler = f3dasm.sampling.RandomUniformSampling(design, seed)
samples_random = rsampler.get_samples(30)

causes the following error:
"KeyError: "Only a column name can be used for the key in a dtype mappings argument. '('input', 'x0')' not found in columns."

It would be nice if the error is more helpful!

optimization

Hi,

Which optimization algorithm do you use?

I wonder how you conduct optimization algorithm in this project.

Could you explain it?

Best!

Installation of f3dasm

I have faced errors during the installation of f3dasm, which made me dig into the installation a bit more in detail. The following suggestions wont necessarily fix the issue I am facing, but would definitely be a clean-up, and also make installation more straightforward for others.

Comments on current instructions: For Linux

Reg: Preferred way

I see that there are many (slightly more than usual) packages in the f3dasm_environment.yml file, and probably these are packages that got installed as additional packages. If this is the case:

1. I recommend that the yaml file is exported as:
   conda env export -n f3dasm_env --from-history -f f3dasm_environment.yml

2. Of course, this wouldn't list the pip installations. For that, do a regular export as:
   conda env export -n f3dasm_env -f f3dasm_environment_pip.yml

3. Manually update the f3dasm_environment.yml to have the pip installation list from f3dasm_environment_pip.yml

Reg: If things fail

1. Minor documentation error: Before pip install -e . the environment has to be activated. So,

conda activate f3dasm_env
pip install -e .

2. Importing f3dasm fails because setup.py doesn't install tensorflow as a dependency. I also see in that in setup.py there are some packages which the preferred way are installed through via conda . It would be a good idea to separate out the pip installations and conda installations.

3. Furthermore, it might be worth defining the version of the packages in the setup.py, unless it is left out on purpose.

Reg: Installing all packages manually

1. Minor documentation error: Before pip install -e . the environment has to be activated. So,

conda activate f3dasm_env
pip install -e .

Suggested instruction: For Linux

If the preferred way and if things fail are updated as suggested above, the Installing all packages manually would become redundant. For if things fail, I suggest that conda installations happens first, followed by installation through setup.py, which only installs packages that are additionally required (i.e. redundancies removed from setup.py).

Update Readme file

Write a new readme file for the new release

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L94

UQlab,python files ,matlab files

Dear Dr Bessa:

Excuse me,STEP2a_ML_Classification.py and STEP2b_ML_Regression.py are python files,but others are matlab files,then how to use them?
Does UQlab need to be installed?

Best regards!

this function is incompatible with a 1D design

        # TODO: this function is incompatible with a 1D design, 

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/functions/pybenchfunction.py#L634

F3DASM tutorial feedback

Here are my comments regarding the first F3DASM tutorial session hosted by Martin.

1. The interactive aspect of the session was executed well, in that it highlighted the aspect in which F3DASM extracts its power: a lot of experimental scripting can be rewritten in a modular fashion.
2. While optimization is a central concept in machine learning, I would like to see followup tutorials focusing on the regression part of data-driven design. Since I'm developing the GPR code for F3DASM right now, I could provide some help with this.
3. One of the important goals of F3DASM is for group members to - in some degree - contribute to and / or apply to problems in their own projects. This has understandably not yet been explored in the first session. To this end, a future tutorial session should encourage participants to think about this and formulate their relationship with regards to the F3DASM tool set.

Description for add_output()

https://github.com/bessagroup/F3DASM/blob/82e42fdf00b5d3d5a58c33219cc8b75f45ef3c26/src/f3dasm/base/data.py#L51-L52

The name of the function might be misleading . I expected for the function to keep adding outputs (y1, y2 ..etc.) but instead it assigns values to the already created output space.

Declare sampling method in DoE parameters

Implement declaration of sampling method as part of each DoE parameter:
Example:

vars ={ salib_solob({'f11': [1,2] ..... , n=10}),
materil: {}
}

The above syntax is not possible, because, in Python, a dictionary must define key:value pairs. This means that the above example has to be translated in something like: vars = {"param1": salib_sobol( { 'f11':[1,2]...., n=10} ), material: {} }. In other words, we will have to define a key for every parameter that aims to use a sampling method. This will make the whole dictionary difficult to pars and distinguish which keys are names for the columns of the data frame, and which keys aren't.
Below, I propose some options to deal with this issue considering the less impact on the current code base. Any suggestions or ideas on a better way to deal with this?

Freeze version when tutorials are released

I feel that it would be a good practice to freeze the version at the stage of a tutorial so that the jupyter notebooks can be reproduced in the exact same way as it was done during the tutorial. I can see from release notes that this was the intention, i.e. v1.0.0 was supposed to be frozen for the tutorial. However some commits have slipped through this. For example, I have figured out that commit bfd8395, made to v1.0.0 has refactored scaling of functions. This influences the results of the tutorial, due to which there a mismatch between the tutorial and the notebook practical_session_solutions.ipynb.

Suggestion:
If the version v1.0.0 was frozen at that stage of the tutorial, and all future changes were made to version v.1.0.1 for example, this issue could have been avoided.

Consolidate Docs folder

Organize and consolidate information, examples and tutorials in the docs folder.

• Create a branch to publish documentation to RTD.org
• Create a template for documentation using RTD.
• Publish documentation to f3dasm read the docs website.

Example issue

This import statement is commented

https://github.com/bessagroup/F3DASM/blob/6d5d00be8e77388c303244df2815b14fd4fe43b0/f3dasm/run_optimization.py#L7

change to True

    continuous = False #TODO: change to True

https://github.com/bessagroup/F3DASM/blob/18591237720e370bbb295de823888bb3f7dd8649/src/f3dasm/functions/pybenchfunction.py#L1031

this is a test issue from selection of code

this is just a test

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/base/function.py#L208-L211

What to do with UML folder?

Decide what to do with the content of the uml folder. Are the files relevant for future development?

Integrate commits by Taylan

Integrate commit by Taylan by creating a pull request to a patch branch and emerging it to the new-structure branch

python 3 libraries compatible with abaqus python 2

Excuse me,do you encounter the problem of compatiblity?

You seem to use abaqus python script and python 3 libraries.

Now abaqus uses python 2,but most libraries are python 3,how to integrate abaqus python script with python 3 libraries? They are often uncompatible.

Best!

evaluation is not defined at (x, y) = (0, 0)

        # TODO: evaluation is not defined at (x, y) = (0, 0)

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/functions/pybenchfunction.py#L1301

Details for the installation github page

It would be nice to know what all packages will be installed (when installing F3DASM) and how much space it would take up.

New DoE and data implementation

Implement a new version of the DoE component based on the agreements on the 25-Oct.

How to represent materials in a generic and flexible way?

Materials can be represented in several ways. Which representation makes more sense for the cases in F3DASM?
The representation should be:

• genric, useful for any kind of material
• flexible, number and names will vary

Here is one possibility :

mat1 = {'elements': [ {'name': 'STEEL', 'params': {'param1': 1, 'param2': 2}},
                    {'name': 'CARBON', 'params': {'param1': 3, 'param2': 4, 'param3': 'value3'} }
                    ]
                }

Do we need to change the license for the new release?

TU Delft has a new software policy for open-source software, and it has a specific way to write the license file. Do we adopt that? or do we keep the current license?

Squeezed F values

I noticed some undesired behaviour regarding the Sobol sampling. Basically, if we look at the Example 1 in the pull request mentioned above, 'Fs' is sampled with Sobol in 3 dimensions (F11, F12, F22), but in the final doe, these dimensions become squeezed to a single one - F1s, (I believe that happens in 'create_combinations' in doevars.py line 102), and as a result the number of doe points goes from desired 5 to 5 * #dim = 15 in this case, and the overall # of doe is 270, but should be 90. In other words, I think these F11, F12, F22 should appear each in a separate column instead of squeezing them together. Maybe some multi-indexing from pandas could be a solution.

Implement Abaqus example

Extend code in the simulator to implement examples using Abaqus. Use Luis code as a starting point.

change noise calculation to work with autograd.numpy @mpvanderschelling

        # TODO: change noise calculation to work with autograd.numpy @mpvanderschelling

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/base/function.py#L337

example issue 2

https://github.com/bessagroup/F3DASM/blob/dc5680dc1d534ab6212088c0297b08769094a86b/f3dasm/examples/multi-fidelity/simple_sogpr_example.py#L15

Best input parameters ?

https://github.com/bessagroup/F3DASM/blob/82e42fdf00b5d3d5a58c33219cc8b75f45ef3c26/src/f3dasm/base/data.py#L123-L135

It is not clear if the function calculates the point closest to the global minima or the one that is closest in value to the global minima.

Do we need to create a container for this release?

_from_input_to_scaled issue

Input scaling doesn't work with dimensionality 3 or higher.

Coding session feedback

Thank you for the session. It was interactive, helpful and insightful. Here are some thoughts of things I would do again, and items I would change:

• Having us fill in the code on the spot is nice. It got me to think in the moment on code syntax and intention of the functions. Further, being asked to complete it showed me the points I was missing.
• On fixing the points I did not understand raised the issue of where to go in the documentation center. Before delving into what each class is intended for, the overall setup of the code needs to be understood. I was lacking this understanding. At the introduction, please provide more background on the session intentions. Not all of us have optimization backgrounds.
• Spending time to explain the setup of the documentation center would go a long way. Providing us the know how to navigate the documentation center on our own would facilitate us to readily use the code.
• Benchmark functions are handy, but ultimately in material or geometry design, I intend to use physical based data. Including a physical example for the example data would improve session impact. For example, using data from a colleague, or creating your own with a simple physics model (pendulum, visco-elastic material model, etc.)