gasparl / possa Goto Github PK
View Code? Open in Web Editor NEWPOSSA: Power simulation for sequential analyses and multiple hypotheses.
License: Other
POSSA: Power simulation for sequential analyses and multiple hypotheses.
License: Other
An obvious strength of the package is its flexibility to perform power analyses for almost any number of pretty much any statistical test in R. However, this requires that the user code their own test function such as:
customTest = function(sample1) {
c(
a = t.test(...)$p.value,
b = aov(...)[[1]][[1]][["Pr(>F)"]][2]
)
}
As you also point out in your vignettes, this can be somewhat tricky for ANOVA or regression fits but even adding the unusual $p.value
after the t-test call might be somewhat difficult to understand for the average R user. Would there perhaps be a way of supplying some wrapper function that automatically extracts the desired p-value from common (not all) test objects so that the user can just write something like the following?
customTest = function(sample1) {
c(
a = extract(t.test(...)),
b = extract(aov(...), "FactorA:FactorB")
)
}
This relates to the JOSS submission openjournals/joss-reviews#4643.
The outcomes of the actual experiment in case of a sequential design should be adjusted; see the related SO question for details and ideas.
There could be a function to plot the distribution (density) of p values, with vertical line(s) indicating the given alpha(s) and corresponding Type I and Type II errors highlighted under the density curves with color fills. It could also be interactive somewhat like this, perhaps using Shiny.
Then again, all this seems pretty complicated to implement with sequential testing, let alone with multiple hypotheses. Also it's not crucial. Still, it would be insightful and instructive, so would be good to have eventually.
Perhaps the sim
function could allow the user to create the sample for each look via a custom user-function (instead of subsampling internally from the maximum sample). However, this is not a priority; I can't actually think of a scenario where this would be needed, but it might be nice to provide this option, to be sure.
Same as local alphas, the futility bounds could automatically be adjusted to achieve any given level of desired power (beta). This is not a priority, but would be nice to have. (As for the mechanism, the same staircase procedure could be used as for adjusting local alphas for T1ER.)
Although the POSSA
internals are very fast and there seems to be little else to help speed, it would be great to somehow make the simulations faster.
For example, one way might be to have an option to provide total population once to the function (for each given sample), so that, instead of each time calling a sampling function, the population can be simply sampled in base R. Then again, the speed gain is probably not much with this, because the most time-consuming part is the statistical testing, which however seems inevitable.
Hi, @gasparl, you package looks nice! I see there are many package contains a code of conduct. I suggest you may consider to have that:)
Instead of any specific number, the number of iterations (i.e., how many times the simulated experiment is run) should be decided on in an objective manner โ preferably based on when the values of interest are "sufficiently accurate" (e.g., narrow enough CI, etc.).
Please add a reference for Pocock's correction in the JOSS paper.
This relates to the JOSS submission openjournals/joss-reviews#4643.
More practical examples should be added to a dedicated vignette (e.g., unequal samples and SD; ranked data; DeLong's test; AOV).
Your repo contains two LICENSE files, one plain-text (as required by JOSS) and one in Markdown (.md). Do you really need both? If so, could you make sure that the plain-text file is identical to the other?
This relates to JOSS submission openjournals/joss-reviews#4643
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.