kkmann / adoptr Goto Github PK
View Code? Open in Web Editor NEWAdaptive optimal two-stage designs for clinical trials made easy.
Home Page: https://kkmann.github.io/adoptr/
License: Other
Adaptive optimal two-stage designs for clinical trials made easy.
Home Page: https://kkmann.github.io/adoptr/
License: Other
haha, that's a fun one - I think we currently work with a single arm z test which is kinda boring. Extend this via the data distribution interface to two-stages (equal allocation for 0.1, everything else can come later ;))!
We need to decide what should go in the final package vignette.
I'd say let's just do the standard case + one design showing off Bayesian prior + conditional power constraint.
We also need to talk about prior capabilities + score/constraint arithmetic/construction.
Could do all in one file (slightly in favor of that) or split prior and score/constraints to different files (currently just dummies)
with the new AverageN2 regularizer, we do not penalize n1 at all. I am not saying that this is necessary, but maybe one would like to apply the same amount of regularization to n2 and n1 to avoid changing the shape of a design too much. This can be easily done by adding a regularizer just penalizing n1. Non, critical, I am not assigning a milestone but we might want to keep this in the backlog.
the n()
function should be plotted as non-continuous on the stopping boundaries, cf.
since we change the post processing default to TRUE in #98, we might want to manually switch to non-post-processing in most test cases (except those checking post processing ;))
we need to round the n1/n2 part after optimization and then refit the remaining parameters. This is a heuristic, but probably the best one can do...
We currently export everything. That's not really necessary. We should go through each item and decide whether or not this actually needs to be exported (e.g., users should not need to call AffineConditionalScore directly - ever). This also relieves some documentation burden. Important but not 0.1.0-important.
we need to make sure that we return the entire information from the optimization call
https://github.com/kkmann/otsd/blob/7dc292730e9c1ced854c150c8a8ba67428f80002/R/minimize.R#L104
not just the optimal desing
I think we should make it possible to fix
Is this straight-forward or do we even need a new class for this?
Well, I'd feel a lot more confident if we had something like 90+ % coverage. Test also need to be revised and less copy / paste!
We have to fix the ToDo in AffineScoreFunction regarding duplicated scores.
This should be the template for all other scenarios. Work out how exactly we are describing everything, the structure and how we handle test cases (where to throw errors). Also need to check how we get this build and deployed on Travis-CI without running into time-limit problems!
Now that we have such a wonderfully general interface - why not allow optimizing simple one-stage designs as well? Should be a piece of cake - only two parameter n and c, right?
In practice, there will be a few locations in the code where I implicitly assumed TwoStageDesign - those might be a little bit of a headache but in principle should it should be straight-forward to do.
The generic evaluation currently ignores uncontinuous jumps at continuation area vs rest. should also integrate in three parts (also faster!). @MatheMax That's an easy one...
https://github.com/kkmann/otsd/blob/83de4d296e7a3057c450fb70710e42ccc7b90d79/R/Scores.R#L83-L106
We might need to rename some stuff or at least discuss it. Feel free to collect any weirdnesses here ;)
We should have a generic way of getting summaries for design objects.
summary(design, unconditionalscore1, ..., unconditionalscorek)
should return a data.frame,
show(design)
needs to be implemented...
OTSD is as interesting and memorable as this ... other thing I recently heard about and cannot remember now.
I am scratching my head but cannot come up with anything better than BAdaSS for Bayesian Adaptive Sample Size - a controversial choice. I am not quite sure whether using 'BAdaSS designs' would be a pro or con in a grant application...
We need to review the entire documentation - see where it can be simplified etc.
Check what functions are used by stats::integrate and vectorize them all!
We should come up with a final suggest default for the order of integration.
Criteria are 'nicety' of the resulting n2/c2 functions (smooth enough?) and run time since I suspect that our crappy optimizer will converge a lot faster with fewer parameters.
Goes hand-in-hand with the default relative tolerance and maximum number of iterations.
This is not a priority, correctness and feature completeness are more important but it should be investigated systematically.
we currently only allow score <= / >= value, we should also add value <= / >= score and score >= /<= score for sake of completeness. boring and non-crucial but should be done sometime.
cf, #2
minimize() has become quite cluttered over time - any chance of simplifying that? Maybe we should have a separate method for one stage designs?
@MatheMax We finally want to support scores of the form s(design, x, theta)
and any form of expectation with respect to a joint distribution over (X, Theta)
. I.e., allowing to integrate over either X
, Theta
or both.
Currently a ConditionalScore
is only allowed to depend on x
and we do not really model joint distributions. This needs to be generalized for full flexibility.
A possible interface could be
# specify joint distribution of outcomes and parameters by specifying
# families for X | theta, n and theta
# i.e. in a normal model with uniform prior over theta (standardized effect size)
cp <- ConditionalPower(
x ~ Normal(),
theta ~ ContinuousPrior(function(x) = 1, c(0, 1))
)
evaluate conditional on both:
eval(design, cp, x = 1.55, theta = .0) # conditional type one error given z = 1.55
or only conditional on observed data
eval(design, cp, x = 1.55) # integrate over theta | x
or just the expected value (probability of rejecting):
eval(design, cp)
or condition before integration:
eval(design, cp, theta = c(.3, 1), x = 1.55)
btw 'power' would then be
eval(design, cp, theta = .4)
Anyway, looks much more consistent than the mixture we have right now. Easyly possible, just a bit 'fiddly'.
Means that people only really need to care about s(design, x, theta)
and all 'integral scores' are
truely derived consitently.
We need to implement a reasonable show() method for all custom classes. Otherwise, the confusing default list-like method is used. In almost all cases, I'd just print the class name since more detailed output will be too complex/confusing on the terminal, @MatheMax what's your take on this?
okay, just found another ifelse() that somehow ended up in the code - think that's due to the rounded/not rounded distinction.
There is VERY rarely a reason to use ifelse() when using it you have to be aware of it's quirks!
e.g. here
ifelse leads to breaking the vectorization of n2() since it's output dimension depends on the FIRST argument onyl (-> docs).
@MatheMax please make sure u remove/double check all uses of ifelse. Btw, these problems only show that we invested to little time in testing properly ;)
We should provide sensible methods for plot(). Should show sample size and critical value vs x1 and optionally an arbitrary list of conditional scores
plot(design, conditionalpower, ...)
would be a 1x3 plot with n/n2/conditional power.
Only use base plot, no ggplot2, to avoid additional dependency.
Hey,
Smoothness_n2 is an ugly mess at the moment. I hacked it together to avoid having to specify a prior etc. but it would be much nicer to fit it (at least approximately) in our system.
We simply need to specify a wide uniform default prior and it should work fine.
Also, do we want this for n2 and c2? seems to work fine for only n2.
Would be nice to have a simulate(design, prior, n) function returning respective z1/z2 values (can be NA), n2, c2, and final decision. This allows for easy cross-checking of power / toer.
we currently simply apply condition scores to the continuation set. That is a reasonable default but it should be possible to pass something any subset of list("futility", "continuation", "efficacy") and apply it accordingly.
How/where do we best put that?
Simple question: is the current implementation good enough to handle effect size weighted sample size?
I think yes, but @MatheMax have you thought about this?
https://github.com/kkmann/otsd/blob/83de4d296e7a3057c450fb70710e42ccc7b90d79/R/GQDesign.R#L68-L71
The syntax -score
should work for both conditional and unconditional scores. We currently need to write -1 * score
Following #47, it might be worthwhile playing around with other optimizers or at least think about ways to implement that. Stochastic global optimizers might be an option but I am not sure as to how well they handle non-linearly constraint problems.
Unfortunately, it turns out that the implemented integration by computing the posterior does not always work.
I think the problem is the following:
When computing the renormalizing constant, this constant can become very small.
This yields to quite large function values of the posterior density (on a small part of the support) and integrating the posterior does not necessarily return "1" anymore.
For now, I see 3 possible ways:
@kkmann Do you have any suggestions?
well, probably should raise an error since there is not really anything to plot, right?
We need (all with toer on boundary of null hypothesis):
Those should all eventually end up as test cases (ideally these would not be copy paste but
programmatically generated from matrix of hypotheses / conditions).
Constraints and final ESS should be checked usig simulate().
I would star, however, by picking 4-8 most important examples and implement them in a vignette. This vignette 'examples' automatically ends up in the online documentation (pkgdown).
Once we are happy with the results, these can be 'frozen' by using test_known_value again...
Well, exactly that. Defining GS as fixed n2, same as before - just that n2 is parameterized by a single parameter.
Since score arithmetics are preliminarily available, we could think about sketching up an optimization framework.
An interface could look like this:
problem <- minimize(objective, ...)
optimal_design <- solve(problem, ...) # rather optimize but solve(problem) was just too much of a temptation ...
for this to work, we need a class MinimizationProblem
(for which minimize is essentially a convenient constructor). objective
can be any kind of unconditional score ... hold arbitrary number of constraints.
problem
would then internally store everything needed to pass to nloptr and solve essentially calls nloptr and converts the result back to a design object (and accepts arbitrary optional arguments to pass to nloptr).
We also still need a class for conditional/unconditional constraints for this interface to work.
Thats just a thin wrapper around scores of the form
constraint(score, rhs)
which must also be evaluatable (essentially its just score - constant right?).
I will then implement a NSE version allowing constraint(score <= a)
to be a valid expression which is definitely cool, right ;)
So ultimately you could have something like
minimize(expected_sample_size,
constraint(power >= .8),
constraint(toer <= .05),
constraint(cp >= .7)
)
That should do.
We need to come up with a good validation strategy.
Besides proper unittesting, this should also involve an automatically generated documents with several test cases which can be checked visually for correctness and a detailed explanation of the overall testing strategy. In particular, we would probably include a simple function allowing to sample from a design - we could then validate our numerical solutions by simulation. This is probably the best we can do. Before we start doing that we need to define objectives etc. I.e. to what level should this software be usable in a real clinical trial? I feel that it should be possible to use it, maybe with an intermediate validation step required. We could even go as far as providing a SAS script for validation...
Since for a normal prior the posterior is known analytically, we could implement a subclass of ContinuousPrior and implement the normal distribution case.
well, yeah.
I think we are currently lacking tests to ensure that the internal version and the version used during optimization return roughly the same values, right?
Okay, @MatheMax next major insight (cf. PR #46 ): Turns out that you get an almost perfect solution even for order 9, it just takes a while. I can get rid of the ugly bits by just running the optimizer longer with higher precision (stopped at 10k iterations).
A few random thoughts:
before we let this loose we should ensure that the coding style in our test suite is consistent, we use enough whitespaces and all tests are sufficiently well documented.
Figure out if absolute value (l1-norm) works better for smoothness term
@MatheMax as of 83de4d2 gaussquad
is imported via the Depends: field in DESCRIPTION. This is unclean since it attaches the gaussquad namespace.
gaussquad functions seem to be unusable without attaching the namespace though - complains about legendre.recurrence not being exported.
This should probably be fixed upstream in the gausquad package - is the maintainer still active? Do we have alternatives?
Following #47, we might get faster and more stable convergence by using suitable regularizers.
Regularizing c2() is extremely difficult - at least I have no idea about how to chose a penalty parameter there. The scale is just too abstract. Regularizing n() is much easier.
I would suggest trying to directly penalize the the integral over n2 on the continuation region (computed via Gauss quadrature) divided by the lenght of that interval. @MatheMax that should be easy to implement and could help with convergence without changing the shape of the solution as much as the smoothness term.
We need to check that we always pass correctly vectorized functions since expectation assumes that!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.