domml/datasail

DataSAIL is a tool to split datasets while reducing information leakage.

DataSAIL: Data Splitting Against Information Leaking

DataSAIL is a tool that splits data while minimizing the information leakage. This tool formulates the splitting of a dataset as constrained minimization problem and computes the assignment of data points to splits while minimizing the objective function that accounts for information leakage.

Internally, DataSAIL uses disciplined quasi-convex programming and binary quadratic programs to formulate the optimization task. To solve this DataSAIL relies on SCIP, one of the fastest non-commercial solvers for this type of problems and MOSEK, a commercial solver that distributes free licenses for academic use.

Apart from the here presented short overview, you can find a more detailed explanation of the tool on ReadTheDocs.

Installation

DataSAIL is installable from conda using mamba. using

conda create -n sail -c conda-forge -c kalininalab -c bioconda MPP
conda activate sail
pip install grakel

to install it into a new empty environment or

conda install -c conda-forge -c kalininalab -c bioconda MPP
pip install grakel

to install DataSAIL in an already existing environment. Due to dependencies of the clustering algorithms, the latter might lead to package conflicts with the already installed packages and requirements.

DataSAIL is available from Python 3.8 and newer.

Usage

DataSAIL is installed as a commandline tool. So, in the conda environment DataSAIL has been installed to, you can run

sail --e-type P --e-data <path_to_fasta> --e-sim mmseqs --output <path_to_output_path> --technique C1e

to split a set of proteins that have been clustered using mmseqs. For a full list of arguments run sail -h and checkout ReadTheDocs.

When to use DataSAIL and when not to use

One can distinguish two main ways to train a machine learning model on biological data.

• Either the model shall be applied to data that is substantially different from the data to train on. In this case it is important to have test cases that model this real world application scenario properly by being as dissimilar as possible to the training data.
• Or the training dataset already covers the full space of possible samples shown to the model.

DataSAIL is created to compute complex splits of the data by separating data based on similarities. This creates complex data-splits for the first scenario. Therefore, use DataSAIL when your model is applied to data that is different from your training data but not if the data in application is more or less the same as in the training.