Statistical Concepts In The Design And Interpretation Of Clinical Trials In NeuroScience

ASENT 4th Annual Meeting

Thursday, March 14 - Saturday, March 16, 2002
Capital Hilton Hotel Washington, DC 

Speaker Abstracts: Statistical Concepts In The Design And Interpretation Of Clinical Trials In NeuroScience

DESIGNING STROKE TRIALS
Barbara C. Tilley, PhD, Yuko Palesch, PhD
Medical University of South Carolina
Department of Biometry and Epidemiology

  1. Introduction
  2. Definitions
    1. Phase I
    2. Phase II
    3. Phase III
  3. Phase I Trials
    1. Common in studies of treatment for cancer.
    2. Examples of designs
      1. up-and-down method
      2. continual reassessment method
    3. Outcome measures
    4. Current NINDS-funded Phase I trial
  4. Phase II Trials
    1. Past Phase II Trials in stroke
    2. Phase II Trials in cancer research
    3. Design of Phase II Trials for stroke
    4. Types of error
    5. Choosing alpha and beta
    6. K-Stage
    7. Examples
      1. NINDS t-PA Trial
      2. TOAST
      3. Fospheyntoin
      4. Atlantis
      5. Current NINDS-funded Phase II trial
    8. Advantages
    9. Disadvantages
    10. Other comments
  5. Phase III Trials
    1. Choosing outcome measures
    2. Clinical Issues
    3. Using Global test statistics
      1. Why not Bonferroni
      2. Why not Hotelling’s T2
      3. Non-parametric procedure
      4. GEE
      5. Assumptions
      6. Properties
      7. NINDS example
      8. Uses outside of ischemic stroke trials
    4. d. Sample size
      1. Considerations
      2. (example using Barthel)
    5. Global
    6. Wilcoxon Rank Sum Test
    7. For comparing binary outcomes
    8. Composite outcomes
  6. Summary

References:

CHOOSING ENDPOINTS IN NEUROLOGICAL TRIALS
Chris Weir, PhD

Several outcome measure frameworks represent the quality of life, impairment, disability and handicap dimensions of outcome. An ideal outcome measure will be valid, reliable and will be sensitive to important changes in health status. In the context of clinical trials it is also important that the outcome measure is relevant to the objective of the intervention under investigation. A further aspect often disregarded in clinical trials is that the outcome measure should address factors relevant to the individual patient. Taking clinical trials of putative neuroprotective or thrombolytic agents for acute ischemic stroke as a case study, I shall describe recent experiences in endpoint selection for several large influential clinical trials. Several types of endpoint have been utilized with varying degrees of success. Completed trials provide a resource of information regarding the appropriate dimensions of out come to measure, the statistical power of specific endpoints, and the size and pattern of treatment effects that may be observed. I shall outline how these experiences may also be used to guide the assessment of outcome in neurotherapeutic clinical trials in general. Finally, I shall describe in detail novel approaches that are currently under development for the selection of neurological trial endpoints. The performance of such methods will be compared to the properties of the ideal outcome measure and any potential gains in statistical power to identify efficacious therapeutics interventions will be assessed.

WHY NEUROLOGICAL THERAPEUTIC TRIALS WOULD FAIL
J Mau, G L Lenzi, and G J Del Zoppo
Department of Statistics in Medicine, Heinrich Heine University, Duesseldorf, Germany,
Department of Neurological Sciences, University "La Sapienza", Rome, Italy, and
Department of Molecular and Experimental Medicine, The Scripps Research Foundation, La Jolla, U.S.A.

Background & AIMS
Given the professional experience, the technical skills, the carefully planned protocols, and the peer expertise of advisers, one can hardly expect to see gross mistakes or a single cause for a negative stroke clinical trial. As the disease has many facets, so has each "drama" of a failed therapeutic investigation, and without sufficient knowledge about the pathobiology of ischemic stroke, neither will receive appropriate remedy. Past stroke clinical trials had tested quite a range of biochemical pathways, but even more impressive is the range of, or confusion about, metrics, designs and procedures to describe health deficits after an acute stroke and pertaining improvements quantitatively, to plan and to conduct the trials towards a "successful" trial outcome, respectively. Over-optimistic judgements and undue self-sufficiency in pharmaceutical companies, lack of quantitative knowledge about patient pools in hospitals, poor understanding of nosometrics, in particular in multidimensional disease outcomes, and little insight into specific needs for a trial setting at the design phase could represent some valid objections in recent and current clinical research. In a more systematic way, however, one would consider:

Dimensions and time frame of stroke recovery: What are the dimensions and time windows which one should target with a clinical efficacy trial?

Scales of efficacy outcome and "natural" course: What is known about scales in natural course of disease and where may one expect to see improvement?

Stroke clinical trials and societal context: Would differences of health care delivery affect multi-national trials and may one construe a legitimate claim for public access to data from industry-sponsored clinical trials?

Choice of endpoints: Combinations or multiplicity? Should multiple endpoints be combined to measure outcome in one dimension, and which endpoints does one declare as secondary?

Exploration of non-standard designs: Is there a point in multi-dose phase-III and in combination therapy trials? Are one-sided tests for efficacy ethical and scientifically sound?

Intercurrent mortality: Is it an endpoint or a confounder, are there other confounders, and does one have data on any of those others?

Methods
The core opinions of speakers at the International Workshop on STROKE NOSOMETRICS & DESIGN OF STROKE CLINICAL TRIALS, held in Duesseldorf, October 16-17, 1995, will be reported in a summary way. Some 30 clinicians, basic scientists and biostatisticians from Europe, North America and Japan, from academia, industry or governmental authorities, had discussed the clinical and methodological issues in the assessment of course of disease and, in particular, of outcome from therapeutical interventions after acute stroke.

Results

  1. Large parts of the brain remain disregarded by most scales, and individualized time "windows" from onset of symptoms to treatment may be a needed option.
  2. Short-term assessments may be desirable, and items within scales may be weighted more objectively, but longer and closer coverage during follow-up would also have merits.
  3. The impact of health care delivery systems must be understood and addressed, and public access to data from industry-sponsored clinical trials should become a legitimate issue.
  4. The multiplicity of relevant endpoints must be understood in their chronology of expression and their pathobiological correlations and weighed against the insight provided by "global" approaches; the variability of profiles of outcome scales should be studied and addressed more adequately.
  5. The "dogmatic" use of the two-sided tests of significance may be questioned more often at the design stage, and intercurrent mortality, in particular when related to treatment, should be incoporated into efficacy assessments.
  6. Combination therapy trials may become predominant in clinical research on acute stroke therapies, and the appropriate designs should be carefully selected according to clinical relevance; recently developed "adaptive" multi-stage designs will also need closer study.

Discussion
The workshop did not address the value of risk-benefit balances and the need for risk-profile analyses for individualized treatment decison for future patients. And it did not close with a checklist of avoidable errors, either. Instead, it had demonstrated the need for an awareness, to be taken more seriously and more openly into the many committees and boards by the experts and advisors, that day-to-day decisions are largely based on consensus than confirmed scientific evidence.

Conclusion
Encouraging tests in animal models and promising episodes of tolerable and effective treatment in few patients do not turn a subsequent negative phase-III trial into a deplorable failure for progress in therapeutical clinical research: it may well be a case of elimination of the obsolete. For an active agent, however, the race towards an NDA submission is endangered by substantial gaps of knowledge with respect to major constituents of a successful clinical test: understanding the pathobiology, and then applying insightful metrics, choosing endpoints which cover all relevant outcomes, and adopting a powerful design.

DECISION ANALYSIS IN NEUROLOGICAL DRUG DEVELOPMENT
Michael Krams, MD
Pfizer Global Research and Development, Sandwich, U.K.

Background
Developing new drug therapies for CNS indications can be an expensive high-risk operation. Much of the cost of bringing a new compound to market (>US$700M per successful candidate) is accrued during late phase II/III development. Traditionally, phase II/III trials do not formally look at a utility which accounts for cost of development and eventual pay-off, but concentrate on showing a clinically meaningful effect, using a desired effect size and estimated variability of the primary endpoint in isolation as the basis for sample size calculations. However, cost-benefit considerations are intrinsically governing the decision-making on go/no-go decision points for advancing drug development programs. Decision-making happens in discrete steps, at predefined time-points at which sufficient information is gathered. Improving the quality and speed of decision-making may translate into higher cost-efficiency in developing new drug therapies.

The failure to understand the dose-response of a new drug is one important reason for late attrition in drug development. We propose a decision-analytic approach to developing new CNS therapies and will use the example of developing a neuroprotectant for acute stroke to illustrate an adaptive design learning in real time about the dose-response and envisaging a seamless switch from a phase IIb dose-response finding phase to a confirmatory phase III trial. Rather than discussing specifics of choice of endpoint, compound related aspects and more complex issues on setting the overall utility, we will concentrate on the general concept of a highly innovative approach, parts of which have now been deployed in a real drug development program. The design presented here is discussed in more detail by Berry et al, 2002 (1).

Methods and Results
Let us envisage a neuroprotective treatment as a one-off iv-bolus, to be administered as soon as possible after the onset of an acute stroke. Primary endpoint is a continuous clinical outcome measure (say change from baseline to day 90 as measured by a neurological stroke scale). Let us assume that earlier measurements of the outcome parameter are predictive for final outcome. Relevant information about the patient is entered at baseline and at several follow-up time-points in real time, using a fax, web or palmtop based interface to a central computer system. Thus an information base accruing relevant patient outcome data is being built up. In a sequential adaptive dose-response finding study two decision problems are formally and continuously assessed, using a Bayesian decision-analysis.

  1. Which dose should the next patient be allocated to, in order to optimize learning about the minimal dose which will yield maximal efficacy (say ED95)?
  2. Should the dose-response finding phase continue in order to learn more about the dose-response (and the ED95 in particular), or is there sufficient information to now recommend stopping the trial, either for futility or to move into a confirmatory clinical trial, comparing the ED95 against control? Using Bayesian statistics the probability of achieving a predefined overall utility in a future confirmatory trial can be calculated. The overall utility may integrate desired clinical effect, cost of running the development program and estimated pay-off.

We used the Copenhagen Stroke Study (2) and a simulation software programmed by P Mueller (1) to run >100.000 simulated stroke trials and will present results comparing the simulated performance of a traditional dose-ranging study with the decision-analytic approach.

Discussion and Conclusion

A decision-analytic approach and simulation-guided clinical trial design have the potential to

  1. enhance the quality and
  2. speed up the process of decision-making, leading to remarkable advantages for the overall cost-efficiency in the development of neurological drug therapies.

The underlying philosophy of applying decision-analysis to clinical drug development is of interest well beyond the indication of acute stroke. In our presentation we will lay out the potential for future applications.

References

  1. Berry DA, Mueller P, Grieve AP, Smith MK, Parke T, Krams M (2002) Bayesian Designs for Dose-Ranging Drug Trials. In: Case Studies in Bayesian Statistics Vol 5, ed. by Gatsonis C, Kass RE, CarlinB, Carriquiry A, Gelman A, Verdinelli I, West M, Springer-Verlag, New York.
  2. Jorgensen HS, Nakayama H, Raaschou HO, Vive-Larsen J, Stoier M, Olsen TS (1995) Outcome and time course of recovery in stroke. The Copenhagen Stroke Study..Arch Phys Med Rehabil 76:399-4122