The clinical cooperative trial guide to a critical approach

:Acta-::--.. N&rochlrurglca

Acta Ncurochir (Wien) (1991) i10:1-5

9 Springer-Verlag 1991 Printed in Austria

The Correlated Studies Committee of the European Association of Neurosurgical Societies

The Clinical Cooperative Trial Guide to a Critical Approach Ch. B. Ostertag 1 and D. Werdier 2

Abteilung StereotaktischeNeurochirurgie,NeurochirurgischeUniversit/itsklinik, Freiburg i. Br., 2 BiometrischesZentrum Aachen, Aachen, Federal Republic of Germany In collaboration with the members of the Correlated Studies Committee of the EANS 3. Barcia, Valencia, Spain, A. Booth, Coventry, Great Britain, B. Cucciniello, Potenza, Italy, V. Fasano, Turin, Italy, J. J. A. Mooij, Groningen, Netherlands, J.P. Sichez, Paris, France, F. Silva-Santos, Lisbon, Portugal, J. Vajda, Budapest, Hungary

Summary

and therapeutic manoeuvres. The publication of the

The growingdifferencewith regard to academicand professional accountabilitybetweenneurosurgeonsand neuroscienceis alarming. Controlled clinical trials continue to be the most scientificallyvalid method of evaluating standard treatments against the risks and benefits of medical innovations. Clinical research becomes more rewarding as neurosurgeons gain familiarity with the particulars of the structure and relevance of expertlydesigned clinical cooperative trials, The basic principles and the significance of biometrical essentials (research hypothesis, comparability, randomized and nonrandomized trials, phases of studies and planning steps) are briefly discussed. A guide to the pertinent literature is provided.

EC-IC bypass study, which has generated bewilderment among neurosurgeons, is but one recent example 2'3'6'1~ Another is the controversy about the usefulness of endarterectomy ~. The neuroscience community is witnessing a growing difference in the standards of accountability between neurosurgeons and neurologists 9. Although many of us consider clinical trials an ordeal, they are indispensable for our professional accountability. "Scientifically based medicine requires more than post hoc testimonials or uncontrolled studies reporting post-operative improvements to justify its therapies, especially when the recommendations are expensive, potentially dangerous, or both" (F. Plum, H).Apart from scientific accountability, the changing economic context of neurosurgery will make economic accountability and cost effectiveness the way of life for surgeons 5. Controlled clinical trials continue to be the most scientifically valid method of evaluating standard treatments against the risks and the benefits of medical innovations.

Keywords:Clinicaltrial; controlledtrial; cooperativetrial; design.

Introduction

Neurosurgery is a young medical discipline whose methods of diagnosis and treatment are subjected to constant and s o m e t i m e s abrupt evolution. Many newly designed neurosurgical techniques often have escaped proper evaluation, even though the rationale for a n e w surgical technique should be as strict as that required for the development of a new drug. Furthermore, it is alarming that many neurosurgeons are rather reluctant to acknowledge the news presented by controlled clinical trials. Controlled clinical trials often tell us things we do not want to hear about many of our diagnostic

Controlled versus Uncontrolled Clinical Studies

Clinical and laboratory experimental studies are based on the principle of control by comparison. Ab-

2

Ch. B. Ostertag and D. Werdier: The Clinical Cooperative T r i a l - G u i d e to a Critical Approach

surd conclusions can be found in many research reports resulting from the lack of adequate controls. Many studies have been conducted and data collected over years which in the end, however, have turned out to be irrelevant because the biometrical essentials of experiment planning were neglected. We have no objection to uncontrolled clinical studies as such: Uncontrolled studies can pave the way to trying new treatment modalities in small groups of patients. If a treatment is to be extended to the general population of such patients, however, the new treatment must pass the most important test of all, which is to prolong survival and the quality of survival in a higher proportion of patients as compared with the standard treatment. To confirm that the new treatment is superior to the standard treatment requires a controlled prospective trial with a sufficiently large number of patients.

The Research Hypothes& Many procedures we have learned from our teachers are based simply on experience. How do we separate valid experience from invalid? How do we incorporate new scientific knowledge? H o w do we avoid making the same mistakes over and over again? Medical research follows - or at least it should - what is known as the scientific method, which consists of 4 basic steps: 1. Observations are made, that is data are gathered from selected populations under selected conditions; 2. Based on these data, hypotheses are formed on the effect of the disease or treatment of the said population under selected conditions; 3. One of these hypotheses is chosen to be tested; 4. The hypothesis is tested by an experiment conducted under controlled conditions. The first most fundamental step is formulating the research hypothesis. Let's say an old treatment is being directly compared with a new one. To find out a potential difference in the effectiveness of the two different treatment modalities, the research hypothesis must be formulated as a question. In many reports published in the literature, however, it is difficult to identify the research hypothesis. At best, they are reports on experiences ("argument by anecdote"). There my be excellent recordings and analyses of cases of diseases. Though the description of cases of a particular disease can be highly valuable, the singularity of a case or a number of cases only exists against the background of the general observations.

Comparability To be able to compare two treatment modalities, the treatment groups must also be comparable. Comparability is achieved by: 1. Clear and specific patient definition, e.g. age, sex, localization and type of lesion, etc.; 2. Definition of selection criteria; 3. Standardization of measurements, which includes a fixed time schedule; 4. Homogeneity with respect to well-known influencing factors. Another essential condition is the compliance of collaborating physicians and patients. Comparisons result in statements of differences (in our example, for instance, the efficacy of treatment A versus that of treatment B). Differences must be statistically analyzed and expressed in statistical terms. The large number and variety of biostatistical techniques make it impossible for most physicians to acquire enough statistical expertise for designing a controlled clinical trial. It is therefore mandatory to consult biostatistical experts who are experienced enough with clinical problems. Nevertheless, neurosurgeons have to acquire enough biometrical expertise to be able to critically evaluate published reports 4'7.

Possible Study Designs Clinical cooperative trials are ideally carried out as an experiment, i.e. a comparison of two or more treatment modalities. A clinical trial is definitely not characterized by a historical control of post-therapy against pre-therapy. Clinical trials are experimental studies that involve patients as study units. Clinical trials can be designed to include randomization with respect to different treatments or as non-randomized controlled trials of treatment effects. Randomization provides homogeneity of the study population only with respect to unknown influencing factors, in other words, it provides comparability. Homogeneity of well-known influencing factors in the treatment groups is achieved by stratification with respect to those factors and randomization within the strata. The number of strata should not be higher than 10-12. The study design is called "randomized block", if we have two groups that are observed during one period and a comparison is usually done between the groups, i.e. measurements of the primary variable under consideration are carried out at the beginnning and at the end of the period.

Ch. B. Ostertag and D. Werdier: The Clinical Cooperative Trial-Guide to a Critical Approach

Usually, a comparison is done "between patients" and not "between groups". Another possible design of a clinical trial is the socalled "change -over" design, in which at least two groups are observed over at least two periods. At the end of the first period the treatments change. If we have, for instance, two groups G and H, two periods, and two treatments A and B, group G will receive treatment A in the first period controlled by group H and treatment B. Group G will change to treatment B in the second period controlled by group H and treatment A. The advantage of this design is that every patient can be his or her own control if the so-called carryover-effect can statistically be regarded as zero. There do exist more sophisticated designs for several groups, several treatments and several periods such as the "latin square" design and others. In addition to the aforementioned designs, which are "non-sequential" designs, there are "sequential" designs in which data are not gathered during a fixed period. Data are gathered until a statistically relevant difference between treatments can be detected or a maximum number of patients has been recruited to ensure the test power for the decision. Upon both events, the trial is stopped.

Randomized Study Design The randomized study design which means the patients are assigned to treatment and control groups at random is the golden standard for a controlled scientific experiment. The study units (the patients) are observed before and after exposure to a treatment. A priori knowledge about well-known cofactors of the primary variable under consideration is used for prospective stratification. Since in many clinical trials patients are recruited into the study over many years, a sequential study design is usually preferred in order to minimize the duration of the trial. A randomized blind trial design avoids three major sources of error: bias due to prerandomization confounding factors, bias due to unintended intervention and by blinding the outcome bias due to desired results. However, many research questions are not suited for a strictly experimental design. It is therefore suggested to have experimental designs reserved for relative mature research questions - those that have already been examined by observational studies but which require stronger evidence for a health policy to be established 8.

Prospective Non-randomized Controlled Trial If it is not possible to use randomization in allocating the study units (i.e. the patients) to different treatment

3

groups by random (due to ethical considerations or technical problems), a non-randomized controlled trial is the next best choice. It can be carried out as a prospective or a retrospective study. The use of non-randomized controls in clinical trials is often mandatory when different treatment modalities (e.g. surgical techniques) are used in different institutions. An experienced surgeon is usually proficient in one, but not all the treatment modalities which are to be compared. To randomely assign patients to different hospitals is often not feasible for economic reasons and patient compliance. In such a situation randomization for better comparability is not always practicable. However, it may only be dispensed with if stratification (prospective matching) can provide homogeneity of the study groups with respect to known co-variates of the primary variable under consideration. Rather than choosing to drop the trial altogether because of problems with randomization, one should always resort to conducting a non-randomized study. A non-randomized study is especially feasible when sufficient a priori knowledge on co-variates of the primary variable under consideration is available. The results of non-controlled studies have to be interpreted cautiously. Although they can help generate hypotheses, they can not be used to confirm them. Prospective observational studies are in fact not designed for exceptional clinical situations only. They are a useful tool for the evaluation of prognostic factors which might be relevant for an individualized treatment.

Retrospective Controlled Studies Retrospective controlled studies are case-control studies, in which historical cases are used which were compiled under non-standardized conditions. The lack of comparability of diagnostic and evaluation criteria is one of many serious disadvantages of retrospective observational studies. The disregard of important prognostic factors in historical trials can lead to bias. To achieve a better structural homogeneity between control group and treatment group, the subjects are matched with comparable controls, i.e. the controls and treatment cases are matched for sex, age, tumour, type and location etc. In a matched pairs study design, however, all cases without a matching pair have to be ignored. The more criteria used for matching the more cases are lost. The choice of controls by matching in

4

Ch. B. Ostertag and D. Werdier: The ClinicalCooperative Trial-Guide to a Critical Approach

retrospective studies is critical because of many hospital and population - related factors such as entry criteria (preference for operable cases), composition of recruited population, etc. a. It can generally be stated that the results of clinical trials are more dependent on the method of control group selection than on the therapy under study 12.

Phases of Planning The search for safe and effective treatment modalities begins with basic research in laboratory and animal studies. At that time, it is unknown how the new treatment will work with patients or what the risks involved will be. The treatment producing the best laboratory results is then tried in patient studies with the hope that it will lead to improved treatment results over those obtained thus far. Such a study is then called a clinical trial, clinical meaning use in patients as opposed to laboratory animals. Although the researcher knows how laboratory animals react, no one knows how the patients will react, i.e. there is a significant risk in this phase of the trial. The purpose of this phase is to find out the most effective and safe way to carry out this new treatment. In a Phase I study the new treatment is offered to only those patients for whom no other known treatment modality exists. One example is the treatment of malignant brain tumours. There is no known effective treatment for malignant gliomas of the brain. For almost 100 years we have seen no progress toward developing a cure for these tumours. The techniques of basic sciences applied in chemo- and immunotherapy of brain gliomas now offer one of the most cautiously optimistic areas of brain tumour research. Apart from the use of interferons a new class of cells called lymphoid killer cells (LAK cells) was described which are capable of killing tumour cells in vitro. L A K cells are produced by the incubation of peripheral blood lymphocytes with interleukin-2 (IL-2) for 3 days. L A K cells do not kill normal cells. They kill, however, a wide variety of autologous and allogeneic tumour cells. It was shown that L A K cells can kill glioma cells in vitro and that single intracerebral injections of both L A K cells and interleukin-2 are tolerated without side effects in animals 9. At this point of the research for a new treatment the phase I clinical trial will start with a small number of patients who are selected and treated according to a detailed protocol. After the appropriate dosage of the new treatment regime has be determined, treatment moves into Phase

Hstudies, which test its effectiveness. The study is open to new patients for an indeterminate time until a sufficient number of patients is entered. The aim of a Phase II trial is to estimate the response rate for a new treatment and its limitations which form the basis for initiating the start of a Phase III study. If a treatment appears to have sufficient response rate and is still shown to be safe, it moves to a Phase III study. Here it is compared with standard treatment (the standard treatment in our example would be surgery and radiotherapy for maligant gliomas). Phase II trials of the intratumoural L A K cell and interleukin2 therapy of human gliomas are currently under way. It has yet to be shown how effective regional adoptive immunotherapy with L A K cells is before a phase III trial with a larger number of patients can be instituted. Setting up a Clinical Trial Any clinical trial has to be in full accordance with the "Declaration of Helsinki 1964, revised at Venice 1983". In the Federal Republic of Germany also the regulations of the "Arzneimittelgesetz" with regard to pharmacological testing of new drugs in humans have to be followed. Comparable legal regulations probably are existent also in most other countries. The first step in setting up a clinical trial is writing the protocol. The following checklist may serve as the "hat pegs" for a protocol: 1. Identify the investigators and their institution and state whether the study is open to other interested investigators. 2. Identify the chief coordinator of the study, joint coordinators, and the data manager. Identify the authors of the protocol. Designate the projected beginning and termination of the study. 3. Give a precise and specific account of the available knowledge of the subject to be studied. Formulate a research hypothesis in the form of a question and specify the objectives of the investigation (e.g. survival, life quality etc.). 4. Describe the design of the study (type of randomization, randomized block, change-over, latin square, observational-prospective, observational-retrospective case control, etc.). 5. Describe in detail the patient inclusion and exclusion criteria. 6. Document and define all aspects of the methodology (laboratory procedures, surgical modalities etc.). Specify how the quality of the collected data is controlled and how often.

Ch. B. Ostertag and D. Werdier: The Clinical Cooperative Trial-Guide to a Critical Approach

7. Describe the primary variable under consideration and the major and minor response variables. 8. Describe the major and minor end points. 9. Describe how unexpected side effects are observed and recorded. 10. Describe the handling of problem cases such as drop-outs or lost to follow up patients. 11. Include the: entire protocol into an informed consent of the patient. It must include in detail the purpose of the study and the possible risks which have to be described in layterms to the patient. 12. Submit the entire protocol to the university's ethics committee. The consent of the ethics committee serves both the patient and the investigators. 13. Define exactly how the results of the ongoing or completed study are publicised, whether in oral or written form. Writing a protocol for a clinical trial requires organizing a team 'with the necessary expertise and commitment. The protocol should indicate the advantages and disadvantages of the trade-offs in the study plan, the various sources of financial support, the distribution of guidelines and minimum requirements for the study protocol, which have to be followed. There are many practical and comprehensive textbooks available on fundamentals; of clinical trials and their design. A guide to the pertinent literature is provided so that the neurosurgeon who has not yet aquired special knowledge can make use of the available expertise and become familiar with current standards of biometry. References 1. Anonymous (1987) Committee on Health Care Issues, American Neurological Association: Does carotid endarterectomy decrease stroke and death in patients with transient ischemic attacks? Ann Neurol 22:72-76 2. Anonymous (1985)The EC/IC Bypass Study Group: The international cooperative study of extracranial/intracranial arterial anastomosis (EC/IC Bypass Study): Methodology and entry characteristics. Stroke 16:397-406 3. Anonymous (1985) The EC/IC Bypass Study Group: Failure of extracraniaI/intracranial arterial bypass to reduce the risk of ischemic stroke: Results of the international randomized trial. N Engl J Med 313:1191-1200 4. Elston RC, Johnson WD (1987) Essentials of biostatistics. F.A. Davis Company~ Philadelphia, pp 246-251 5. Enthoven AC (1985) The changing economic context of medical decision making. In: Clinical neurosurgery. Proceedings of the

5

congress of neurological surgeons, Honolulu, Hawaii, 1985. Williams & Wilkins, Baltimore, London, Los Angeless, Sydney, pp 81-91 6. Goldring S, Zervas N, Langfitt T (1987) The extracranial/intracranial bypass study: A report of the committee appointed by the American Association of Neurological Surgeons to examine the study. N Engl J Med 316:817-820 7. Haines SJ (1981) Six statistical suggestions for surgeons. Neurosurgery 9:414418 8. Hulley SB, Cummings SR (1988) Designing clinical research. An epidemiologic approach. Williams & Wilkins, Baltimore, Hongkong, London, Sydney 9. Jacobs SK, Wilson DJ, Kornblith BL, et al (1986) In vitro killing of human glioblastoma by interleukin-2-activated autologous lymphocytes. J Neurosurg 64:114-117 10. McDowell FH (1988) Neurology, Neurosurgery, Controlied trials and academic accountability. Stroke 19:1463-1465 11. Plum F (1985) Extracranial-intracranial arterial by-pass and cerebral vascular disease. N Engl J Med 313:1221-1223 12. Sacks H, Chalmers TC, Smith H (1982) Randomized versus historical controls for clinical trials. Am J Med 72:233-240 Suggested Literature

i. Biefang S, K6pke W, Schreiber MA (1979) Manual ffir die Planung und Durchfiihrung von Therapiestudien. Medizinische Informatik und Statistik, Bd 13. Springer, Berlin, Heidelberg, New York, Tokyo 2. Friedmann LM, Furberg CD, DeMets DL (1985) Fundamentals of clinical trials. 2nd ed. PSG Publishing Comp, Littletou, Ma 3. Hulley SB, Cummings SR (1988) Designing clinical research. An epidemiologic approach. Williams & Wilkins, Baltimore, Honkong, London, Sydney 4. Kelsey JF, Thompson WD, Evans AS (1986) Methods in observational epidemiology. Oxford University Press, New York 5. Kleinbaum DG, Kupper LL, Morgenstern H (1982) Epidemiologic research: Principles and quantitative methods. Lifetime Learning Publications, Belmont, Cal 6. Meinert C (1986) Clinical trials. Oxford University Press, New York 7. Pocock SJ (1983) Clinical trials: A practical approach. John Wiley and Sons, Chichester 8. Schlesselman JJ (1982) Case-control studies: Design, conduct, analysis. Oxford University Press, New York 1982 9. Silvermann WA (1985) Human experimentation: A guided step into the unknown. Oxford University Press, Oxford, New York i0. Spilker B (1984) Guide to clinical studies and developing protocols. Raven Press, New York Correspondence and Reprints: Prof. Dr. Ch. B. Ostertag, Neurochirurgische Universitfitsklinik, Hugstetter Strasse 55, D-W-7800 Freiburg i. Br., Federal Republic of Germany.