
An important task in early phase drug development is to identify patients,
which respond better or worse to an experimental treatment. While a variety of
different subgroup identification methods have been developed for the situation
of trials that study an experimental treatment and control, much less work has
been done in the situation when patients are randomized to different dose
groups. In this article we propose new strategies to perform subgroup analyses
in dosefinding trials and discuss the challenges, which arise in this new
setting. We consider modelbased recursive partitioning, which has recently
been applied to subgroup identification in two arm trials, as a promising
method to tackle these challenges and assess its viability using a real trial
example and simulations. Our results show that modelbased recursive
partitioning can be used to identify subgroups of patients with different
doseresponse curves and improves estimation of treatment effects and minimum
effective doses, when heterogeneity among patients is present.

Identifying subgroups, which respond differently to a treatment, both in
terms of efficacy and safety, is an important part of drug development. A
wellknown challenge in exploratory subgroup analyses is the small sample size
in the considered subgroups, which is usually too low to allow for definite
comparisons. In early phase trials this problem is further exaggerated, because
limited or no clinical prior information on the drug and plausible subgroups is
available. We evaluate novel strategies for treatment effect estimation in
these settings in a simulation study motivated by real clinical trial
situations. We compare several approaches to estimate treatment effects for
selected subgroups, employing model averaging, resampling and Lasso regression
methods. Two subgroup identification approaches are employed, one based on
categorization of covariates and the other based on splines. Our results show
that naive estimation of the treatment effect, which ignores that a selection
has taken place, leads to bias and overoptimistic conclusions. For the
considered simulation scenarios virtually all evaluated novel methods provide
more adequate estimates of the treatment effect for selected subgroups, in
terms of bias, MSE and confidence interval coverage.

A common problem in Phase II clinical trials is the comparison of dose
response curves corresponding to different treatment groups. If the effect of
the dose level is described by parametric regression models and the treatments
differ in the administration frequency (but not in the sort of drug) a
reasonable assumption is that the regression models for the different
treatments share common parameters. This paper develops optimal design theory
for the comparison of different regression models with common parameters. We
derive upper bounds on the number of support points of admissible designs, and
explicit expressions for $D$optimal designs are derived for frequently used
dose response models with a common location parameter. If the location and
scale parameter in the different models coincide, minimally supported designs
are determined and sufficient conditions for their optimality in the class of
all designs derived. The results are illustrated in a dosefinding study
comparing monthly and weekly administration.

We consider the problem of testing for a doserelated effect based on a
candidate set of (typically nonlinear) doseresponse models using
likelihoodratio tests. For the considered models this reduces to assessing
whether the slope parameter in these nonlinear regression models is zero or
not. A technical problem is that the null distribution (when the slope is zero)
depends on nonidentifiable parameters, so that standard asymptotic results on
the distribution of the likelihoodratio test no longer apply. Asymptotic
solutions for this problem have been extensively discussed in the literature.
The resulting approximations however are not of simple form and require
simulation to calculate the asymptotic distribution. In addition their
appropriateness might be doubtful for the case of a small sample size. Direct
simulation to approximate the null distribution is numerically unstable due to
the non identifiability of some parameters. In this article we derive a
numerical algorithm to approximate the exact distribution of the
likelihoodratio test under multiple models for normally distributed data. The
algorithm uses methods from differential geometry and can be used to evaluate
the distribution under the null hypothesis, but also allows for power and
sample size calculations. We compare the proposed testing approach to the
MCPMod methodology and alternative methods for testing for a doserelated
trend in a dosefinding example data set and simulations.

Phase II dose finding studies in clinical drug development are typically
conducted to adequately characterize the dose response relationship of a new
drug. An important decision is then on the choice of a suitable dose response
function to support dose selection for the subsequent Phase III studies. In
this paper we compare different approaches for model selection and model
averaging using mathematical properties as well as simulations. Accordingly, we
review and illustrate asymptotic properties of model selection criteria and
investigate their behavior when changing the sample size but keeping the effect
size constant. In a large scale simulation study we investigate how the various
approaches perform in realistically chosen settings. Finally, the different
methods are illustrated with a recently conducted Phase II dosefinding study in
patients with chronic obstructive pulmonary disease.

In this paper we consider twostage adaptive doseresponse study designs,
where the study design is changed at an interim analysis based on the
information collected so far. In a simulation study, two approaches will be
compared for these type of designs; (i) updating the study design by
calculating the maximum likelihood estimate for the doseresponse model
parameters and then calculating the design for the second stage that is locally
optimal for this estimate, and (ii) using the complete posterior distribution
of the model parameter at interim to calculate a Bayesian optimal design (i.e.
taking into account parameter uncertainty). In particular, for an early interim
analysis respecting parameter uncertainty seems more adequate, on the other
hand for a Bayesian approach dependency on the prior is expected and an
adequately thoughtthrough prior is required. A computationally efficient
method is proposed for calculating the Bayesian design at interim based on
approximating the full posterior sample using kmeans clustering. The sigmoid
Emax doseresponse model and the Doptimality criterion will be used in this
paper.

Statistical methodology for the design and analysis of clinical Phase II dose
response studies, with related software implementation, are well developed for
the case of a normally distributed, homoscedastic response considered for a
single timepoint in parallel group study designs. In practice, however, binary,
count, or timetoevent endpoints are often used, typically measured repeatedly
over time and sometimes in more complex settings like crossover study designs.
In this paper we develop an overarching methodology to perform efficient
multiple comparisons and modeling for dose finding, under uncertainty about the
doseresponse shape, using general parametric models. The framework described
here is quite general and covers dose finding using generalized nonlinear
models, linear and nonlinear mixed effects models, Cox proportional hazards
(PH) models, etc. In addition to the core framework, we also develop a general
purpose methodology to fit dose response data in a computationally and
statistically efficient way. Several examples, using a variety of different
statistical models, illustrate the breadth of applicability of the results. For
the analyses we developed the R addon package DoseFinding, which provides a
convenient interface to the general approach adopted here.

This paper considers the topic of finding prior distributions when a major
component of the statistical model depends on a nonlinear function. Using
results on how to construct uniform distributions in general metric spaces, we
propose a prior distribution that is uniform in the space of functional shapes
of the underlying nonlinear function and then backtransform to obtain a prior
distribution for the original model parameters. The primary application
considered in this article is nonlinear regression, but the idea might be of
interest beyond this case. For nonlinear regression the so constructed priors
have the advantage that they are parametrization invariant and do not violate
the likelihood principle, as opposed to uniform distributions on the parameters
or the Jeffrey's prior, respectively. The utility of the proposed priors is
demonstrated in the context of nonlinear regression modelling in clinical
dosefinding trials, through a real data example and simulation. In addition
the proposed priors are used for calculation of an optimal Bayesian design.

Dosefinding studies are frequently conducted to evaluate the effect of
different doses or concentration levels of a compound on a response of
interest. Applications include the investigation of a new medicinal drug, a
herbicide or fertilizer, a molecular entity, an environmental toxin, or an
industrial chemical. In pharmaceutical drug development, dosefinding studies
are of critical importance because of regulatory requirements that marketed
doses are safe and provide clinically relevant efficacy. Motivated by a
dosefinding study in moderate persistent asthma, we propose responseadaptive
designs addressing two major challenges in dosefinding studies: uncertainty
about the doseresponse models and large variability in parameter estimates. To
allocate new cohorts of patients in an ongoing study, we use optimal designs
that are robust under model uncertainty. In addition, we use a Bayesian
shrinkage approach to stabilize the parameter estimates over the successive
interim analyses used in the adaptations. This approach allows us to calculate
updated parameter estimates and model probabilities that can then be used to
calculate the optimal design for subsequent cohorts. The resulting designs are
hence robust with respect to model misspecification and additionally can
efficiently adapt to the information accrued in an ongoing study. We focus on
adaptive designs for estimating the minimum effective dose, although
alternative optimality criteria or mixtures thereof could be used, enabling the
design to address multiple objectives.

The Laplace approximation is an old, but frequently used method to
approximate integrals for Bayesian calculations. In this paper we develop an
extension of the Laplace approximation, by applying it iteratively to the
residual, i.e., the difference between the current approximation and the true
function. The final approximation is thus a linear combination of multivariate
normal densities, where the coefficients are chosen to achieve a good fit to
the target distribution. We illustrate on real and artificial examples that the
proposed procedure is a computationally efficient alternative to current
approaches for approximation of multivariate probability densities. The
Rpackage iterLap implementing the methods described in this article is
available from the CRAN servers.