Protocol Writing: The Influence of Statistics on a Positive Outcome

In 2017, TransCelerate Biopharma released the latest version of the Common Protocol Template (CPT) as an update to their first standard template, published in 2015. That same year, the U.S. Food and Drug Administration (FDA) and the National Institutes of Health (NIH) released a joint final protocol template. The light is shining bright on protocol development as it is such an important part of how clinical trials are conducted.

The overall industry alignment on protocol writing has been a massive effort over the past few years, according to Victoria Murray, Manager, Regulatory and Medical Writing, MMS.

Clinical Informatics News reported that since 2010, there has been an increase in new studies on ClinicalTrials.gov by approximately 20,000 per year. With each study, comes a protocol, and having these follow the same or similar templates is helpful for reviewers – in addition to an ease of understanding and better compliance for investigator sites.

What the biostatistician says

A study protocol is a detailed investigational plan which documents the objectives, design, and methodology of a clinical trial. Jing Dai, Sr. Biostatistician, MMS, says that “the biostatistician has a critical role and broad responsibilities in ensuring the scientific and ethical integrity of this product.”

During the protocol development, the biostatistician collaborates closely with the clinical team to define key elements including study objectives, primary/secondary endpoints, trial design, patient selection criteria, treatment regimes, visit schedule, safety monitoring, and statistical methodology to assure the adherence to ICH/FDA guidelines, the safety of the patients, and that the data collected are unbiased and reproducible.

Besides the above mentioned collaborative effort, the biostatistician is responsible for the writing of the data analysis/statistical methodology sections of the study protocol, as well as providing input to other sections such as those dealing with randomization and treatment masking.

Dai adds, “The required statistical input differs from study to study; however, some common components include bias minimization through randomization and blinding, sample size and power estimation, definition of analysis sets, specification of primary/secondary variables, description of hypothesis, statistical assumptions, and analysis models. The statistician also often determines mechanisms for handling missing values, censoring, early discontinuations, and deviations from trial conduct. Some trials also require the biostatistician’s input on pharmacokinetic and interim analysis.”

“The biostatistician must understand the clinical concept of the study well enough to provide guidance,” said Dai. “This often requires intensive involvement in clinical discussions, required literature review, and preliminary data analyses.”

Therefore, the best time to engage a biostatistician in the protocol development is always as early as possible.

“Personally, I have witnessed overly restrictive inclusion criteria leading to low accrual rate and significant extensions of study period,” noted Dai. “I’ve also seen studies with underestimated dropout rates leading to insufficient sample size. These oversights are not only an inefficient use of time and resources, but they can also lead to early termination of the trials; not to mention costs and recruitment difficulties with certain vulnerable patient populations.”

Tips from a medical writer

Murray emphasized the importance of statisticians in protocol writing, saying “they are the ones who call the shots. If the study is under-powered – a sponsor will spend millions of dollars on the clinical trial without the ability to establish the efficacy needed. All the time and effort from the sponsor, and more importantly the patients participating in the study, is wasted.”

In the case of unexpected dropouts, for instance, if the dropout rate was 40 percent, the trial would have a high potential to fail. With an average estimated dropout rate of 10-15 percent, the review and implementation of a protocol needs input from statistics upfront to ensure little to no surprises at the end.

“Biostatisticians have to be consulted to make certain that the statistical analysis plan (SAP) is consistent with the protocol later on,” said Murray. “The last thing anyone wants is for the FDA or EMA to see any discrepancies between protocol and SAP.”

Confessions from the field

Murray said she notices “smaller companies, who are not as established as industry giants, need help with protocol writing the most.”

In some circumstances, Murray has seen issues where sponsor teams did not want objectives in a protocol, only endpoints, and others did not have statisticians involved at all. For clients in earlier phases and those with fewer resources, Murray and her colleagues have created a protocol bootcamp to spread knowledge of what is required.

For efficacy, safety, and other analyses, “the way you write and analyze these endpoints is very important if you want the results you expect from the study,” said Murray. “Many clinical trials fail and it can come down to how primary and secondary endpoints are chosen and analyzed.”

Endpoints are written at a subject level, while objectives are written at the study level.

Murray added, “A biostatistician should be working alongside a medical writer as early as possible, right from the time objectives and endpoints are created. This is a key design element in the protocol concept sheet (PCS), and eventually it will have an impact on what is stated on the drug label.”