Issues in Randomization Outline

Issues in RandomizationLaura Lee Johnson, Ph.D.StatisticianNational Center for Complementaryand Alternative Medicinejohnslau@mail.nih.govFall 2012Objectives:Randomization Lecture• Identify– Reasons and mechanisms forrandomization– Types of randomized study designs– Nonrandomized experimental studies– Issues in implementation– Threats to randomization (and study)integrity• Compare randomized experimental studiesto nonrandomzied observational studiesOutline‣What is Randomization?• Randomized Study Design• What is a random sample?p• Introductory Statistical Definitions• Stat Software1

What do we mean by random?• Everyday speech– Process is random if there is no discernablepattern• Statistics– Random characterizes a process ofselection that is governed by a knownprobability rule for example 2 treatmentshave equal chance of being assigned– Random in statistics does not meanhaphazard4What is random treatmentallocation?“By random allocation, we mean that eachpatient has a known chance, usually equalchance, of being given each treatment, butthe treatment to be given cannot bepredicted.” (Altman 1991)5What is random treatmentallocation?• Randomization is the act of allocating arandom treatment assignment• A patient is said to be randomized to atreatment arm or group when they areassigned to the treatment group usingrandom allocation• Clinical trials that use random treatmentallocation are referred to as randomizedclinical trials62

Random Sample vs.Randomization• Random sample: chancedetermines who will be IN thesample• Randomization: chance determinesthe ASSIGNMENT of treatmentBasic• Two (or more) different treatments that wewant to compareRandomization: Definition• Not a random sample• Random Allocation– Known chance receiving a treatment– Cannot predict the treatment to be given• Eliminate Selection Bias– Outcome measurements can be biased• Similar Treatment Groups– Does not eliminate baseline differences– If baseline differences they arose bychance3

ONE Factor is Different• Randomization tries to ensure thatONLY ONE factor is different betweentwo or more groups.– Chance if 80% women in one group and20% in the other• Observe the Consequences• Attribute Causality• In truth, a rarity and cannot testWays to Randomize• Standard ways:– Random number tables (see text)– Computer programs– randomization.com• Three randomization plan generators• NOT legitimate– Birth date– Last digit of the medical record number– Odd/even room numberWho/What to Randomize - Independence• Person– Common unit of randomization in RCTs– Might take several biopsies/person• Provider– Doctor – Doctor– Nursing station• Locality– School– Community• Sample size is predominantly determined bythe number of randomized units4

Should I Randomize?Almost Always YES; But Pitfalls• Small sample size• Rare condition• Rare confounding factors• People do what they want anyway– Testing Life as practiced! (at your localgym, drug or health food store)– Wikipedia killed some blinding/masking• Post randomization exposed non-randomly– Medication mix up and control group getswrong investigational agentTypes of Randomization• Simple• Blocked Randomization• Stratified Randomization• Baseline Covariate AdaptiveRandomization/Allocation• Response Adaptive Randomization orAllocation (using interim data)• Cluster RandomizationSimple Randomization• Randomize each patient to atreatment with a known probability– Corresponds to flipping a coin• Could have imbalance in # / groupor trends in group assignment• Could have different distributionsof a trait like gender in the twoarms5

Block Randomization• Insure the # of patients assigned toeach treatment is not far out of balance– Block size = 6, 2 study interventions (Aand B)• AAABBB, BAABAB, ABABAB….• Variable block size (permuted)– An additional layer of blindness• Different distributions of a trait likegender in the two arms possibleStratified Randomization• A priori certain factors likelyimportant (e.g. Age, Gender)• Randomize so different levels ofthe factor are BALANCED betweentreatment groups• Cannot evaluate the stratificationvariableStratified Randomization• For each subgroup or strataperform a separate blockrandomization• Common strata– Clinical center, Age, Gender• Stratification MUST be taken intoaccount in the data analysis!6

Stratified Block Randomization ExamplePreexposure Prophylaxis Initiative (iPrEx) Trial• Double-blinded placebo-controlledrandomized trial examining efficacy of achemoprophylaxis regimen for HIV prevention• International multi-center study– Multiple advantages to achieving balancedallocation by site• 2499 HIV- men or transgender women wererandomly assigned in blocks of 10, stratifiedaccording to siteNEJM 2010 v363 (27): 2587-259919Adaptive Randomization• Previously discussed randomization methodsexamples of fixed allocation schemes– Order of treatment assignments can becompletely determined in advance of trial• Adaptive randomization schemes “adapt” orchange according to characteristics ofsubjects enrolled in trial• Note not all adaptive trials involve adaptiverandomization, namely group sequential trials20Adaptive Randomization• Sequence of treatment assignments cannotbe determined in advance• Probability of assigning a new participant aparticular treatment can change over time• Two major classes– Adaptive with respect to baselinecharacteristics– Adaptive with respect to patient outcomes217

Adaptive ?Randomization?Same Title, Different Meanings• Baseline Covariate– Minimization/Dynamic allocation– Pocock & Simon; Brad Efron (biasedcoin)• Adaptive Randomization/Allocation– Using interim outcome data– Play the winner or 2-armed bandit– BayesianBaseline Covariate AdaptiveRandomization/Allocation• Minimization/Dynamic Allocation– Balance on the margins• Table 1 looks pretty– Does not promise overall treatmentarms balanced in #• Pocock & Simon (biased coin)– Baseline covariates– Weighted probability (not 50/50)Why not just stratify?• Typically, many many variables• Will not have people in each “cell”if do traditional stratification– How many participants• Pittsburgh Site, Male, 40-64,• AND Grade 2, hormone therapy, 6-18 mopost treatment,• AND……8

Eye-Opening Experiencefor Minimization• Late Onset Treatment Study (LOTS)– 90 patients with late onset Pompe disease– Van der Ploeg et al (2010) NEJM 362, 1396-1406• For more details about statistical problemsminimization caused, see– Proschan, M., Brittain, E., andKammerman, L. (2011). Minimize the useof minimization with unequal allocation.Biometrics 67, 1135-114125Response AdaptiveRandomization/Allocation• Outcome data during trial (interim)• Unbalance # / arm in favor of the‘better’ treatment(s)– Ethically appealing to some• Difficult to do well– Computer programming, not simple– All blinded but statisticianAdaptive RandomizationDifficult• Programming is not easy• All blinded but statistician• Ignore covariates– Unknown can lead to problems– Treatment-covariate interactions• Imbalances may be backwards withinsubgroups– Time trends/drift9

Response Adaptive• May be group sequential designs• May use continuous interimanalysis to feed into randomization• May use set interim i analysis timepoints to feed into randomization• Do not want response to be toolong termCluster Randomization• Same ideas as before• Unit of randomization– School/Clinic/Hospitals– Providers• Outcome measurement– Students– Patients• Need to use special models for analysisbecause those providing outcomesnested in unit that was randomizedWhat do I recommend?• Depends on the study andresources available• My go-to for researchers– Permuted block randomization– Stratified by site– Choose block size(s) appropriate tosample size– Randomize smallest independentelement at last possible second10

Example• Try this at home!– Or at NIH Thursday evening session(sign up with Benita Bazemore)• Bags of hard shell chocolate candyExample• How many bags?• Different sizes of bags?• Number of types of candy?• Number of colors in each?11

Randomization Example• N = 56 (nice small study size)• Different types of randomization• 2 arm study• 6 colors: red, orange, yellow, blue,green, black• Compare to N = 20 exampleSimple Randomization• Perform a simple randomization• Record the results• Repeat as long as you have time(3-5 minutes)Simple Randomization #112

Randomize 56, 3 TimesSimple Randomization1 2 3N/arm 28:28Red 9 4:5Orange 8 4:4Yellow 3 2:1Blue 11 5:6Green 16 10:6Black 9 3:6Simple Randomization #2Randomize 56, 3 TimesSimple Randomization1 2 3N/arm 28:28 38:18Red 9 4:5 6:3Orange 8 4:4 7:1Yellow 3 2:1 1:2Blue 11 5:6 5:6Green 16 10:6 10:6Black 9 3:6 9:013

Randomize 56, 3 TimesSimple Randomization1 2 3N/arm 28:28 38:18 17:39Red 9 4:5 6:3 2:7Orange 8 4:4 7:1 4:4Yellow 3 2:1 1:2 1:2Blue 11 5:6 5:6 3:8Green 16 10:6 10:6 4:12Black 9 3:6 9:0 3:6Randomize 20, 5 TimesSimple Randomization1 2 3 4 5N/arm 9:11 9:11 11:9 6:14 14:6Red 4 2:2 2:2 2:2 0:4 3:1Orange 5 1:4 2:3 2:3 2:3 4:1Yellow 4 2:2 2:2 3:1 0:4 2:2Blue 2 2:0 1:1 1:1 1:1 2:0Green 3 1:2 1:2 2:1 1:2 2:1Black 2 1:1 1:1 1:1 2:0 1:1Block Randomization• Try again• Use (simple) Block Randomization14

Simple Block RandomizationRandomize 56, BlocksBlockN/arm 28:28Red 5:4Orange 3:5Yellow 0:3Blue 8:3Green 7:9Black 5:4Permuted Block Randomization15

Permuted Block RandomizationRandomize 56, BlocksBlock PermutedBlockN/arm 28:28 28:28Red 5:4 4:5Orange 3:5 5:3Yellow 0:3 3:0Blue 8:3 3:8Green 7:9 8:8Black 5:4 5:4Stratified Permuted BlockRandomization16

Stratified Permuted BlockRandomizationRandomize 56, BlocksBlockPermutedBlockStratifiedPerm. Bl.N/arm 28:28 28:28 28:28Red 5:4 4:5 5:4Orange 3:5 5:3 4:4Yellow 0:3 3:0 2:1Blue 8:3 3:8 5:6Green 7:9 8:8 8:8Black 5:4 5:4 4:5Randomize 20, BlocksBlockPermutedBlockStratifiedPerm. Bl.N/arm 10:10 10:10 10:10Red 2:2 2:2 2:2Orange 2:3 2:3 2:3Yellow 3:1 3:1 2:2Blue 2:0 1:1 1:1Green 1:2 1:2 2:1Black 0:2 1:1 1:117

Many Ways to Randomize• Choose one– Appropriate to sample size– Choose block size(s) appropriate tosample size• If I have to choose one– Permuted block randomization• Stratified by siteTime to Randomize?• When the treatment must change!• SWOG: 1 vs. 2 years of CMFVPadjuvant chemotherapy in axillarynode-positive and estrogenreceptor-negative patients.– JCO, Vol 11 No. 9 (Sept), 1993Randomize at the Time TrialArms Diverge• SWOG randomized at beginning oftreatment• Discontinued treatment beforerelapse or death– 17% on 1 year arm– 59% on 2 year arm– Main reason was patient refusal18

Even if 2 weeks later?• Long term use of beta blockerspost MI• 393 randomized 2 weeks prior tostarting therapy• 162 patients treated– 69 beta blocker– 93 placeboRandomized, Treated, Analyzed• 393 randomized• 162 patients treated• “…appears to be an effective formof secondary therapy ….”– Paper reported on analysis of n=162• What about the 231 randomizedbut dropped from the analysis?Intent to Treat vs. Completers• ITT = Intent To Treat analysis– Assume all study participants• Adhered to the study regime assigned• Completed the study– Only analysis to preserve randomization• MITT = Modified ITT analysis– ITT, but only include people who take thefirst dosage• Per Protocol-Completers-Adherers analysis– Only the well behaved19

Take Home• Permuted block randomization– Stratified by site– Appropriate to sample size– Choose block size(s) appropriate tosample size• Randomize smallest independentelement at last possible second• ITT (intent to treat) analysisOutlineWhat is Randomization?‣Randomized Study Design• What is a random sample?• Introductory Statistical Definitions• Stat SoftwareStudy Design Taxonomy• Randomized vs. Non-Randomized• Blinded/Masked or Not– Single-blind, Double blind, Unblinded• Treatment vs. Observational• Prospective vs. Retrospective• Longitudinal vs. Cross-sectional20

Ideal Study - Gold Standard• Randomized• Double blind / masked• Treatment• Prospective• Parallel groupsTypes of Randomized Studies• Parallel Group - classic• Sequential Trials – physical sciences• Group Sequential trials - classic• Cross-over – very useful if useable• Factorial Designs - independence• Adaptive Designs – gaining popularityThreats to RandomizationIntegrity• Improper masking or blinding– Bias will creep into data• Excluding subjects who withdraw fromtreatment– Can lead to bias• Drop-out/missing data– Without ITT: Breaks randomization– With ITT: Weakens treatment result– Long trials may need to have a screeningperiod to assess commitment of subjectsbefore randomization6321

Simple Important Steps to Avoid Problems• Check eligibility criteria thoroughly prior toenrollment– Unnecessary drop out if realize afterrandomization subject not eligible andforced to withdraw– No matter how quickly a subject iswithdrawn all randomized subjects includedin ITT• Avoid a delay between randomization andtreatment initiation– Delays can lead to unnecessary missingdata that affects ITT64Documentation• Protocol should specify who maintains/hasaccess to unblinded treatment assignment– Protocol should also specify criteria forunmasking and treatment of earlywithdrawals• There should be documentation of therandomization algorithm, randomization list,and how it was implemented– Reproducibility is key65Possible Implementations of BlindedRandomization• Sequenced sealed envelopes– Prone to tampering!• Sequenced bottles/packets– Generally carried out by pharmacy/drugcompany• Phone call to central location– Live response, IVRS (Voice ResponseSystem)• On site computer system• Web based• Best plans easily messed up in implementation22

OutlineWhat is Randomization?Randomized Study Design‣What is a random sample?• Introductory Statistical Definitions• Stat SoftwareRandom Sample• Draw from the population• Use a probability device• Select names out of a hat– Now randomize them to treatmentassignments• A few types of sampling to get namesinto the hat (not randomization)– Simple– Stratified– ClusterSimple Random Sample• Every possible subject chosenfrom a population for investigationhas an equal chance of beingselected from the population• Stop laughing23

Stratified Sampling• Select independent samples• Number of subpopulations,groups, strata within thepopulation• Might gain efficiency if donejudiciouslyCluster Sampling• Sample in groups• Need to look at intra-clustercorrelationRandomization Conclusions• Permuted block randomization– Stratified by site– Appropriate to sample size– Block size(s) appropriate to sample size• Randomize smallest independent element atlast possible second• Masking/blinding is also key• ITT (intent to treat) analysis• Analyze as you randomize• Proper documentation and implementation24

OutlineWhat is Randomization?Randomized Study DesignWhat is a random sample?‣Introductory Statistical Definitions• Stat SoftwareVocabulary• Sample size: N or n– May refer to total or per group!• Mean: average; sum / n• Median: 50%; middle ordered value• Variance: σ 2 (population) or s 2 (sample)• Standard deviation: σ or s• Standard error: σ/√n or s/√nVocabulary• Odds ratio• Relative risk• Proportion: ranges 0 to 1– For example 45% = 0.45• A|B is said, “A Given B”– P(A|B): “If B is true, what is the probabilityof A?” or “What is the probability of Agiven B is true?”25

Vocabulary• Y i = β 0 + β 1 x 1i + ε i• Y = outcome or response variable– Might not be an actual response• X = covariate, variable• β 0 = intercept– Average value of Y when X = 0• β 1 = slope, coefficient• ε = error, residual, difference betweensample fit or prediction and personY i = β 0 + β 1 x 1i + ε i• Subscript ‘i’ is person i; i = 15– Y 15 = 119 (SBP); x 15 = 1 (on treatment)• Y = β 0 + β 1 x 1 general sample model– Say β 0 = 150, β 1 = -20• Y 15 = β 0 + β 1 x 15 + ε 15– Thus 119 = 150 – 20*1 + ε 15– So ε 15 = 119 – 150 + 20 = -11– Difference between Y 15 and modelpredicted Y 15 = -11Vocabulary• Statistic: Compute from sample• Sampling Distribution– All possible values statistic can have– Samples of a given size randomly drawnfrom the same population• Parameter: Compute from population– Usually unknown to researcher– Several large studies in population26

Studies: Early, Stages, Phases• Early studies, intermediate or midrange,Conclusive or Definitive orLarge, Late stage– RCT (Randomized Controlled Trial)• Some are big, some are not• Phase 0-IV• Multistage• SeamlessPhase I to Phase IV Trials• National Cancer Institute: Dictionary of CancerTerms• Phase I - The first step in testing anew treatment in humans• Test the best way to give a newtreatment (for example, by mouth,intravenous infusion, or injection)and the best dosePhase I• Dose increased little at a time• Find the highest dose that does not causeharmful side effects• Little known about possible risks andbenefits of the treatments being tested• Trials usually include only a small number ofpatients who have not been helped by othertreatments27

Blocked Randomization Example:Flu Vaccine Dose Escalation Study• Phase I dose-finding flu vaccine study• Investigate 6 dose levels in a blinded,placebo controlled study– Considered single (one nare) and doubledose (both nares) of 0.25mg, 0.5 mg, and1mg administered intranasally• Primary outcome: safety and tolerability82Blocked Randomization Example:Flu Vaccine Dose Escalation Study• Dose cohorts had 5 active and 2 placebosubjects• Randomized block design, block of size 7• Order of the 5 active and 2 placeboassignments randomly permuted for eachdose group83Phase II• A study to test whether a new treatment hasan anticancer effect (for example, whether itshrinks a tumor or improves blood testresults) and whether it works against acertain type of cancer• Surrogate endpoints– Note: many things shrink tumors and donot improve survival/quality of life(ACRT/SCTS story)28

Phase III• A study to compare the results ofpeople taking a new treatment withthe results of people taking thestandard d treatment t t (for example,which group has better survivalrates or fewer side effects)• “Definitive”Phase IV• After a treatment has been approved and isbeing marketed, it is studied in a phase IVtrial to evaluate side effects or otherindications or populations that were notapparent in the Phase III trial• Thousands of people are involved in a PhaseIV trial• Different population from Phase III• Duration of useDistinguish“Observational studies are oftenanalyzed as if they had resultedfrom a controlled study, and yetthe tacit assumption ofrandomness can be crucial forthe validity of inference.”Copas, J.B. and Li, H.G. (1997). Inference for non-randomsamples (with discussion).Journal of the Royal Statistical Society, 59: 55–95.29

Interesting Intro Talk• http://ted.com/talks/view/id/1234• Fast talking but good interestingintro to research design andresults interpretationAnother Game?Random Example (Game 1)• Everybody get a coin– Designate one side “Heads” and theopposite side “Tails”• Flip the coin• How many heads (raise hands)• How are the heads clustered (vstails) clustered together?9030

Random Example (Game 2)• Everybody pick a random numberfrom this list0 1 2 3 4 5 6 7 8 9 1091OutlineWhat is Randomization?Randomized Study DesignWhat is a random sample?Introductory Statistical Definitions‣Stat SoftwareStatistical Resources• Software• Books• Articles• Colleagues• Internet31

Software• Most is expensive and some haveyearly license fees– NIH (through CIT) many times has thesoftware for free or cheaper thanretail; CDC and universities do, too• Some is hard to use, some is easySoftware: Programming Options• S-PLUS (Windows/UNIX): Strongacademic and NIH following;extensible; comprehensive– www.insightful.com• R (Windows/Linux/UNIX/Mac): GNU;similar to S-PLUS– www.r-project.org– www.bioconductor.orgS+ and R• Produce well-designed publicationqualityplots• Code from C,C++, Fortran can becalled• Active user communities32

Other Software• STATA (Windows/Mac/UNIX)– Good for general computation,survival, diagnostic testing– Epi friendly– GUI/menu and command driven– Active user community– www.stata.comOther Software• SAS (Windows/UNIX)– Command driven– Difficult to use, but very good onceyou know how to use it– Many users on the East coast– www.sas.com• SPSS, EpiCure, many othersStatistical Calculators• www.randomization.com• http://calculators.stat.ucla.edu/– “Statistical Calculators”– Down recently• http://statpages.org/• http://www.biostat.wisc.edu/landemets/• http://www.stat.uiowa.edu/~rlenth/Power33

Questions?34