An R package for reproducible research. Analyze your experimental data and generate HTML and PDF reports.
In this repository I host and maintain my personal website.
It was designed long ago and hosted elsewhere, but I just wanted to learn jekyll so I used this as an excuse.
Feel free to copy/fork anything you need ;)
Include the possibility to create a new Table from PairedTests objects similarly to MultipleTests: include the p-value, the win/tie/loss metric (and maybe which one is the 1st and the 2nd?)
Generate a new plot with the same concept of the ROC curve: x-axis is the worst method, y-axis the best one, and the line generated is as follows: for each dataset d (from 0 to D, where D is the number of datasets), if the best method wins line-up, if the best method lose line-right, and in case of tie line-up-right (diagonal).
When there are ties in the mean ranks now it is ordered alphabetically. I found the case where this order is not good because the win/tie/loss metric prints a comparison A vs B where B has more wins than B (in tabularTestSummary from a testMultiplePairwise).
I suggest to order the ties of the mean ranks by wins/ties/loss/alphabetically (cascade ordering).
When instantiating parameters the information of them is lost although their properties remain implicit in the data. It would be good for the user to have this information when printing the information about the experiment.
For this we need to include and optional configuration variable that should be checked when printing the experiment (both in toString and in the renderisation)
Solve all pending bugs to create an stable version of the old API before moving to the new major version.
Include a parameter for the tabularExpSummary function in order to allow generating a mean columnd or row with the mean results per method.
The function expReorder change the labels but not the data.
Therefore, if I plot an experiment using the plotExpSummary function, the output is the same before and after calling the expReorder function, only the labels of the changed elements are reorder, so data output is wrong.
Current templates not on bootstrap:
In this plot the labels that show the mean values are always shown, a new parameter to hide them can be added to the function.
There are some symbols that can be found in method, dataset, parameters, etc... names that would result in LaTeX error compilation such as "_", we should sanitize the generated code to escape all these symbols.
By default the character used as separator between method and parametes when instantiaion is performed is the comma. It would be desirable to parameterize this character.
In the plot of the rank distribution, the text in xaxis is rotated 90 degrees. This text should be aligned to the right to improve the visualization.
Now the methods are ordered automatically by name. Even if you can change them by hand, it would be desirable to allow to automatically order them by mean maximum or minimum output variable.
There are links to our website right now, but they are pretty hidden. It would be better to improve their visibility...
It would be desirable to allow changing the name of the experiment when necessary.
Also, it would be a more flexible function if wrong names introduced in the "parameter" input are ignored (also allowing it an empty list, so only name can be changed in an experiment).
The parameter to split the table in more than one does not work well when the parameter rowsAsMethods is equals TRUE: the number of tables is not equal to that parameter.
When the Friedman test is performed (in any multiple statistical test) and the p-value is greater than alpha, the warning does not show the message correctly (some missing variable in the sprintf function that prints the message).
Possibility to use multiple outputs in the output parameter, appending for each method N columns, where each one represents the output i (i from 1 to N).
Add an affix scroll spy with a summary of the report's content to the left of the website by using bootstrap capabilities.
It raises an error when trying to generate the ranking distribution plot using a multiple test pairwise as input (and it should work for both multiple test control and pairwise).
When rendering an experiment block in a latex report an error is raised if no parameters are found.
This is because an empty ìtemize environment is created (without any item object), this should be avoided by updating the corresponding template.
When render the exreport object, appart of generating the html it could be dessirable to generate a pdf.
Right now the experiment class does not save explicit information about the methods and problems that it contains, neither of the values for the parameters.
It would be nice if they could be stored in the object structure or in meta tags and be used in the template. Right now some of this information is being included on the template and on the print function but it is being calculated from the data.frame structure every time.
When using more than one output, the parameters do not work well. For example, when the parameter digits is an array c(2,3,4), the number of digits for the three outputs is not 2,3 and 4 respectively.
Instead of changing the name of a method/dataset/parameter in the file that contains the results, it could be more convinient to change them using a function. It could also be used to change the name after the instantiation.
When a wide table is rendered in the HTML report it can be wider than the container and produce an unwanted overflow.
The solution for this could be to scale the table within the container limits and show a warning to the user suggesting spliting the table.
We can use the function expReduce to remove one parameter and apply a function (mean by default) to the outputs of the common rows. It would be usefull a plot that shows the distribution (variability, mean...) of any output (Box-and-Whisker plot).
The current JS functionality embedded in the template code is:
When the number of methods is above 20, the function testMultiplePairwise spend a lot of time (several hours).
Analyzing the code it seems the internal function .shafferS is implemented inefficiently.
Replace our custom templating code
When the experiment parameter is an empty list an empty \begin{enumeration} is included causing a fail in compilation.
It would be useful to add the possibility to change the order of methods and datasets in tables and plots. Also, if data has been read froma csv file, the order could be the same as it is in that file.
Sometimes, the plot of the ranking distribution does not work properly: the colours of the boxes (white and grey) do not correspond to the results of the test (p-value greater or not than alpha).
When splitting the tables, the first one is treated different (the rest have exactly the same number of columns, and the first one the rest). It may occur that this first table has a small number of columns (even 0!), ora big number of columns.
The split should be done by making the tables more or less with the same size (not exactly the same number of column) but in order to have a variation of only +-1 column.
An optional parameter "description" can be useful for experiments, allowing to specify the source or the way they were generated.
By default, if description is not provided in expCreate function, it is empty.
Now, when load an experiment and generate a table or a plot from it, the methods and problems are ordered alphabetically. It would be desirable to choose between ordering alphabetically or by appearance when loading the experiment.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.
