herb-ai/herblearn.jl

Machine Learning module of Herb

HerbLearn.jl

Machine Learning module of Herb

herblearn.jl's Issues

Suppress function overwrite warnings by using anonymous functions

Usage of PyCallUtils adds valuable functionality for formulating machine learning functionality to Herb.jl. However, the overwrite outputs a number of warnings during execution. To avoid this, we should use anonymous functions for the overwrite instead.

Explore Flux for neural networks

Use Conda.jl for environment handling

We should do the following to install and handle the Python environment from Julia (and not just assume the correct installation):

An alternative is to configure PyCall to use a Julia-specific Python distribution via the Conda.jl package (which installs a private Anaconda Python distribution), which has the advantage that packages can be installed and kept up-to-date via Julia. As explained in the PyCall documentation, set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then, to install the torch module, you can use pyimport_conda("torch", PKG), where PKG is the Anaconda package that contains the module torch, or alternatively you can use the Conda package directly (via using Conda followed by Conda.add etcetera).

Add condition on maximal output length for generating data

Generated programs may do very odd things with the inputs, especially when loops are available.

Hence, we may want to add a maximum length on the output sequence length.

Integrate StarSpace LLM embeddings for inputs/outputs

Add support for graph learning over AST

Use torch_geometric to learn representations for programs based on their abstract syntax tree (AST). While the encoder is fairly straightforward, the formulation of the decoder is a little more intricate. A possible formulation would be as a heuristic to guiding search itself. In both cases, the loss function is still under-defined.

Separate encoding and decoding of programs from derivation prediction

• This implementation should handle encoding and decoding as modular actions on programs
• the prediction of the next derivation based on the current program should be modular/separate from encoding/decoding
• the loss should allow for end-to-end training of all modules

Add progress monitoring for ML pretraining

Add support for minimum depth of programs during generation

IOP examples should have a minimum depth in order to avoid overly simple and trivial programs.

Save model to disk

Saved model should be saved to disk after pretraining. Yet, it is not quite clear where and how to do so.