Quality Tolerance Limits (QTLs) Frequently Asked Questions

Q1) Should QTLs be defined in the protocol?

It may be possible to define them within the protocol; however, they may also be defined within a risk plan or other parallel document that is stored within the TMF and managed for a protocol.

Q2) With regard to bioequivalence (BE) studies, what are the regulatory expectations for QTLs?

Regulations do not specifically call out study types such as BE studies. Generally speaking, if patient safety risks or data integrity/interpretability risks exist for specific BE studies, then QTLs would apply.

Q3) What’s the expectation of completing this task for already ongoing studies?

Q4) Since QTLs have to be established at the start of a study, would currently active studies be out of scope even though the study might be ongoing for the next couple of years?

Regulations do not specifically define requirements at this level. It is anticipated that good judgment should be applied such that if an ongoing study may appropriately have QTLs established to ensure patient safety and data integrity/interpretability for the remainder of the study, then they should be established for the ongoing study.

Q5) Do you define the QTL for each study, or for each project, or only once for the entire company? Should the clinical study report (CSR) also report violations of “secondary limits” (below the QTL value)?

QTLs are to be established at a study level. It may be possible to have common QTLs across a project or company for common risks to patients or data interpretability. It should be recognized that specific studies within a given project may appropriately warrant some unique QTLs. Within the study report, it is not required to report on secondary limits; however, it may be leading practice to describe how these secondary limits and mitigations were an active part of your risk control.

Q6) When Pharma is outsourcing a study, could Pharma and the CRO have different QTLs? Are CRO QTLs more strict in order to mitigate issues before going above threshold and need to be addressed with Pharma?

Pharma and CROs should not have different QTLs for a specific study as the QTLs are related to the study risks for patients and data interpretability. The latter question of CRO limits being tighter would relate to the secondary limits, not the actual QTL.

Q7) Have there been any regulatory experiences where the agencies are actually looking for this as part of a sponsor inspection? Examples of findings?

Anecdotal testimonies exist of inspectors beginning to evaluate QTLs. Since ICH E6 (R2) is relatively new and regulators are allowing the industry to adapt, it may be too early to determine inspector trends.

Q8) Who would be the best placed in terms of role in a CRO to define and establish QTLs? Could this be just one role?

Establishment of QTLs should be a team effort including Medical Affairs, Biostatistics, Data Management, COA experts, Clinical Operations, and others. Patient safety or data interpretability risks could be identified by individuals from each of these groups.

Q9) Where do the QTLs get documented prior to the study start? Have there been any Sponsor findings to date related to QTLs?

QTLs are typically documented in risk plans prior to the study start, but may be placed in a protocol or other parallel document. Anecdotal findings have been reported relative to regulatory inspections.

Q10) Where do we document how we defined the rationale for the QTLs?

Documentation of rationale are best managed within the risk plan, protocol, or specific tool used to catalog the parameters and QTLs.

Q11) Protocol entry criteria – not all of them have an influence on the primary endpoint or patient safety. Shouldn’t we differentiate better based on detailed analysis of the criteria? Would you recommend to monitor medical risks, e.g., amount of specific adverse events as a QTL?

It is correct that a generic use of protocol entry criteria for selection of QTLs is not appropriate. Those that may affect patient safety or data interpretability should be assessed for potential designation with QTLs. Medical risks should indeed be considered as potential parameters with QTLs; specific AEs could be possibilities.

Q12) Have the agencies inspected on risk management and what were their comments?

With the new ICH E6 (R2) requirements, it is anticipated that agencies will indeed inspect on risk management; anecdotal testimonies have been reported.

Q13) Any regulatory guidance for implementation on ongoing trials (i.e., already ongoing by the Nov 2016 timepoint)?

Q14) Are there any recommendations from regulatory authorities on what will be the expected/key examples of QTLs (similar with the FDA recommendation quoted at the beginning of the webinar)? Are these parameters to be documented in the protocol, in the risk management plan, or any other study level plan or to be captured in the quality management system (QMS) (at the processes level)?

Regulatory agencies have not identified any specific recommendations for risk parameters and their QTLs. Parameters and QTLs may be documented within a protocol, risk management plan, or study level QMS. Sponsor companies should determine the option that is most effective.

Q15) Is there an expectation that a discrete section of a clinical protocol be dedicated to the description of the predefined QTLs?

Q16) Can you please give an example of a QTL?

Please refer to this webinar recording and slide deck for three specific examples.

Q17) Will QTLs be harmonized across Metrics Champion Consortium (MCC), TransCelerate, and Avoca and accepted by various regulatory agencies (FDA, EMA) across the world?

Avoca cannot speak for or speculate on the activities of other collaboratives such as MCC and TransCelerate, nor to the acceptance of such activities by regulatory authorities. As an organization focused specifically on quality and compliance in clinical research with a mission to improve the execution of every clinical trial, Avoca will continue to promote and support the use leading practices with respect to QTLs with its consulting clients, consortium members, and the industry at large.

Q18) Is it expected/seen that QTLs are established in investigator-initiated trials?

Regulations do not specifically reference investigator-initiated trials; however, the logic of the use of parameters and QTLs to manage patient safety risks and data interpretability would be as applicable to those studies as other studies.

Q19) Any thoughts about the best way to promote biostatistics “leadership” in defining QTLs?

Biostatistics is a critical contributor to QTLs since the agencies’ intent is to focus on systematic risks that may be evaluated statistically. The requirements are necessary for compliance.

Q20) Is the number of QTLs dependent on the size of the trial?

It is not expected that size would impact the number of QTLs.

Q21) How often do you recommend assessing the parameter during the study to assess if the limits have been reached?

The frequency of assessing the parameter is largely dependent upon the parameter, the speed at which enrollment and/or trial assessments occurs, and how these affect the parameter. It also depends on the lead time necessary to implement mitigations if a secondary limit is being used as a trigger to implement mitigations.

Q22) Is it appropriate to look at pre-clinical experience in setting QTLs and in particular for Phase I?

It is absolutely appropriate to use pre-clinical information to set QTLs, especially for Phase I.

Q23) What is the difference between a key performance indicator (KPI) and a key quality indicator (KQI)?

KPIs focus on operational performance (e.g., timelines, costs) while KQIs focus on quality measures.

Q24) For ongoing trials, what is your advice with regard to implementing QTLs despite not being done at the beginning of the trial? When is the right timing to start QTL monitoring, i.e., when a defined % of patients are randomized or rather X months after First Patients are dosed?

Regulations do not specifically state whether “ongoing” studies will be held to the new guidelines. However, it is generally anticipated that regulators and inspectors will hold sponsors accountable to using QTLs for studies started after the timepoint when they began expecting ICH E6 (R2) to be implemented (e.g., June 2017 for EMA). Timing for monitoring of QTLs is highly dependent upon the parameter to which a QTL is associated. There is no standard timing of when monitoring should begin as it relates to the parameter, the risk to patient safety or data interpretability and when metrics may approach secondary limits.

Q25) Can you please provide an example of secondary limit?

Using Lost to Follow Up (LTFU) as the example described within the QTL webinar, the QTL may be the point at which a study may no longer provide the power to distinguish a treatment arm from a control arm. The secondary limit would be set at a lower level whereby one can detect that the study is trending toward the QTL such that more extensive measures may be taken to reduce the number of LTFU subjects.

Q26) If a QTL is exceeded, would this lead a company to open an internal Corrective and Preventive Action (CAPA) program? If yes, would it be the sponsor, the CRO, or both that would open the CAPA?

Depending on the parameter associated with the QTL, it may warrant a CAPA to be initiated. The CAPA could be either at the sponsor or CRO.

Q27) I understand the rationale for establishing QTLs prior to study start, but when dealing with novel research approaches, it seems reasonable that when little, if any, historical data would be available, that regulators would accept/tolerate the fact that QTLs are adjusted once the trial is up and running. What would the process be for actually documenting that QTLs have been revised while the trial is ongoing?

The adjustment of QTLs after a study begins should be the exception rather than the norm. Adjustments may be warranted in situations when more scientific data become available (perhaps from other studies) or for other well justified reasons. In cases when QTLs are adjusted, they should be well documented within the risk plan and also within the Clinical Study Report (CSR) with explanations of the rationale.

Q28) You are referring to QTLs mainly related to data interpretability – what about patient safety aspects? How should QTLs for Phase 1 be handled, where the data are often available not as fast as the trial might be completed ((Study Data Tabulation Model (SDTM) datasets)), so that timely oversight of the parameters/QTLs might be difficult? Is there a consolidated source of information on suitable QTLs? Will, e.g., TransCelerate work on that area?

QTLs are to be established for parameters related to either data interpretability or patient safety. It is not mainly one or the other, but both. For trials of short duration, parameters and QTLs should be monitored as effectively as feasible based on technology that permits real-time access to data and analyses. Information on suitable QTLs are being published by organizations such as TransCelerate and Avoca.

Q29) What is the best way to implement QTLs mid-study if the exercise was not done at study startup? Also, how should this be addressed in the CSR?

The same approach should be made mid-study as is used at the beginning of a study. Reasonable assessment of risk should be considered (i.e., at what limit will patient safety be compromised or will data interpretability be compromised). The CSR should explain the reason for the parameters and QTLs being selected mid-study along with the rationale.

Q30) Would you advise protocol deviation rate or adverse-event (AE) reporting rate as appropriate Parameters for QTLs?

Protocol deviation rate or absolute numbers of deviations may be valid parameters for QTLs. AE reporting rate would need to be assessed in terms of potential to impact patient safety or data interpretability.

Q31) What needs to be documented in the Trial Master File (TMF) to show that a QTL has exceeded or not exceeded as reported in CSR? And do we really need to report non-exceeded QTLs in the CSR?

The TMF should include all essential documents that establish, monitor, and evaluate parameters where QTLs have been established. All parameters with QTLs established at the beginning of the trial should be described in the CSR irrespective of whether the limit was reached. The CSR should describe how the team monitored and controlled the risks even in the positive sense when the risk was effectively controlled thus preventing the limit to be reached.

Q32) Does it make sense to establish QTLs after a study has started?

Q33) Have you seen any publications where a pharmaceutical company has shared the QTLs they used for a given parameter?

Pfizer has shared some of their selected parameters and QTLs at conferences. The three QTLs listed in the QTL webinar are the three mandatories at Pfizer; study teams can add up to two additional QTLs if appropriate.

Q34) Has your experience with QTLs reflected a trend toward study specific parameters, or core parameters that are defined in a study specific way, in the same way that Key Risk Indicators (KRIs) may be core but use a study specific threshold?

Many organizations are having their study teams begin with considering core parameters that may require QTLs for a significant number of studies. They then require the teams to consider if there are study specific parameters that may also warrant QTLs.

Q35) Will QTLs be part of TransCelerate’s initiative to standardize across the pharma industry?

We are not in a position to speculate on TransCelerate initiatives.

Q36) Does Avoca know of any AQC Member companies that have submitted QTLs for FDA acknowledgement prior to start of trial? What is the best way to link Risk Assessment Categorization (RACT) critical data and processes and mitigations to KRIs? Furthermore, is it expected that the RACT high-risk line items be monitored by KRIs, and that their mitigation should match?

We are aware of companies that have submitted parameters and QTLs to the FDA, but we are not aware of the timing relative to obtaining FDA acknowledgement prior to study start. RACT by design should map critical risks to KRIs that serve as metrics to trigger mitigations. RACT high risk items are usually monitored by KRIs, and predefined mitigations are implemented based on trigger thresholds.

Q37) For the QTL planning and leading, our organization actually has a Clinical Risk Management Lead that is planning/implementing this process. We work with all of the noted stakeholders noted in the poll when choosing the QTLs.

This is an effective option.

Q38) Practically, where are investigations to exceeded QTLs or alert limits documented during the course of the study?

These may be documented within a RACT tool and issues log or other project tracking tool.

Q39) Do QTLs depend on available resources? If resources are unlimited, everything could be monitored. RBM software monitors many more parameters than 3-5 as you recommend.

QTLs need to be selected based on the key few that most effectively characterize key risks for your trial that need to be monitored to ensure data interpretability or patient safety. Regulators do not want unlimited numbers of QTLs but rather the key few that matter most. Levels of available resources should not matter in selecting a key few QTLs (3-5).

Q40) Have regulatory agencies inspected sponsors yet on the use of QTLs?

There are anecdotal accounts of regulators setting expectations when on site at sponsors that they will be assessing Quality Management Systems and QTLs per ICH E6 (R2) for future inspections; however, the AQC has not yet received confirmation of actual inspections against QTLs. Since the regulations became effective for many agencies in 2017, we anticipate a lag in actual inspections against the regulations, so the AQC is monitoring this closely.

Q41) When planning QTLs at the beginning of a trial, how do you know what limit should be a QTL versus a secondary limit?

The QTL is most likely the point at which patient safety or data interpretability are actually impacted. Secondary limits are lower thresholds whereby mitigations may be taken to minimize the probability that a QTL will be reached or to minimize the implications should a QTL be reached.

Q42) Why should we limit QTLs to not more than 5 per study? Are exceptions possible?

Regulatory agencies desire that sponsors use QTLs to focus on the most impactful risks to patient safety or interpretability of data (i.e., risks that matter most). They desire that these receive focus during study conduct and be succinctly described in the Clinical Study Report (CSR). There may be valid situations when a study should have more than 5 QTLs.

Q43) What if mitigations are implemented when the secondary limit is reached but the QTL is still reached?

If mitigations implemented when secondary limits are reached and these mitigations are not successful in preventing the QTL being reached, then appropriate steps should be taken as predefined when the QTLs were established.