Inspection Readiness (IR) Questions & Answers from Avoca Webinars

Throughout the following Q&As, AQC Knowledge Center documents are referenced and hyperlinked. The Avoca Quality Consortium (AQC) Knowledge Center is a Member-only online portal with more than 400 leading practices, guidelines, tools, and templates to increase efficiency, improve quality, and mitigate risk in clinical trials. Each item in the Knowledge Center is developed based on a rigorous research and review process by Avoca Subject Matter Experts (SMEs). Contact us to request your Knowledge Center credentials if your company is an AQC Member or for information about membership. Click here to view a video tour of the Knowledge Center.

Webinar Selection:
Confident Risk-Based Inspection Readiness
ICH E6 (R2) Risk-Based Requirements
Inspection Preparation With a Risk-Based Approach
Inspection Readiness With Proactive Risk-Based Oversight
Risk-Based Monitoring
TMF Inspection Readiness

Confident Risk-Based Inspection Readiness

Is risk assessment a mandatory requirement for a bioequivalence study as per ICH GCP (R2) requirements?

ICH E6 (R2) does not make a distinction regarding different types of studies, so the risk assessment guidance (and requirement, where ICH E6 (R2) has been adopted per regulation) applies equally to bioequivalence studies.

What are the biggest differences between US and EU regulatory inspections, and would you prepare differently for these?

There are differences in inspection approaches across regulatory agencies, and it is important to understand the focus of each agency to adequately prepare for inspections. Here are some useful links to start with:

If you are an AQC Member, you can access the Inspection Agency Grid, which provides an overview of different regulatory agencies, their particular areas of focus, and Member experiences related to recent inspections.

Would risk management differ for the different phases of clinical trials (i.e., Phase I versus Phase III trials)?

Yes, there are likely to be different risks at different stages of program development (Phase I, II, IV), as well as at different points of a clinical trial (start-up, execution, closeout). Risk assessments should be done routinely and particularly at key transition points (e.g., moving into Phase III).

Can you speak to CRO responsibilities for sponsor-managed vendors (i.e., ePRO vendor is contracted by Sponsor but CRO is contracted for site management activities)?

The CRO responsibilities would need to be considered and defined in the agreement with the Sponsor. For example, any issues detected by the CRO in relation to ePRO vendor deliverables would need a defined escalation path for notification of the Sponsor, and an established agreement of how such issues would be managed and resolved.

If you are a small company with limited resources to implement (R2), where should you start?

It may sound obvious, but the best place to start is at the top, by getting buy-in from senior leadership for the implementation (this applies to all companies). Then, ideally starting from protocol development stage, work through the risk assessment cycle of identification of critical data and processes, risk evaluation, control identification/measures. Focus your efforts on the key risk that will have the most impact. Next, determine the methods for risk communication, review, and reporting. We recommend you take a look at an article that was recently published in Applied Clinical Trials that includes more details: New ICH Guidelines Address Industry Inefficiency

How would an internal audit demonstrate the objective level of the company’s inspection readiness?

Audits are an essential part of a Quality Management System, and allow the organization to perform an internal check on compliance with SOPs, processes, systems, etc. Audits usually have a defined scope (e.g., process, system), and the audit outcome could be indicative of a general readiness in that specific area for inspection (certainly any findings need to be addressed). However, audits alone would not be able to reflect “objective inspection readiness”. We like the concept of Preparedness and Readiness. Preparedness is a commitment to quality which requires proactive risk management. Inspection Preparedness becomes part of an organization as they leverage prospective measures, like Quality by Design, to ensure an Inspection Readiness state throughout the execution of the clinical program. Elements of this mindset are embedded into SOPs and working documents. By adhering to these while performing daily activities, an organization will operate in a mindset of “Inspection Preparedness” and be in an ongoing state of “Inspection Readiness”.

Is this applicable to Medical Devices?

ICH E6 (R2) risk management guidance applies to clinical trials for all types of products, including any trials conducted on medical device products. There are, of course, some different requirements. Some classes of medical devices may not need a trial, and additional regulations that pertain to medical devices must be considered (e.g., EMA).

What tools would you suggest could be used to help with the constant review of the Inspection Preparedness state of the sponsor? Regular meetings? Inspection Readiness teams?

Inspection preparedness links with your risk management cycle. Therefore, the results of risk monitoring and review will provide valuable information into the state of preparedness. In addition, results of CRO oversight, audits, deliverable completion, etc., also provide preparedness information. It is important to consider and review these and other signals when evaluating the state of inspection preparedness. It would certainly be helpful to have inspection readiness teams comprised of SMEs, who meet regularly to review key indicators of preparedness (e.g., outcome of QMS assessments or progress on completion of functional preparation checklists).

From an inspection readiness point of view, what do you recommend showing to inspectors, regarding your management of key risks?

ICH E6 (R2) requires that there is documentation and communication of risk management activities, and this is what inspectors will be looking for. Any documentation that you compile related to each risk management activity (identification of critical data and processes, risk identification, risk evaluation, risk control, risk review, quality tolerance limits, root cause analysis and CAPAs for emerging risks) should be provided upon request in an inspection. There are a number of industry guidelines and tools to support the analysis, decision making, and the compilation of documentation (e.g., from EMA, MHRA, TransCelerate). If you are an AQC Member, you can access key references and related tools from the AQC Knowledge Center.

ICH E6 (R2) Risk-Based Requirements

What is an example of acceptable risk?

One example is a timeline delay of 3 months, if a study is not on a critical path timeline. Another example of an acceptable risk is treatment risks, such as headaches.

Should the risk management process be used at a study level or a portfolio level?

Individual study level risk plans should be developed and managed according to ICH E6 (R2). At the same time, lessons-learned processes, the risk blueprints, and overall risks being impacted by company or portfolio level standard operating procedures and processes, can be managed by assessing common risks at both the study and portfolio levels.

Are Risk Logs subject to inspection from competent authorities?

Yes, risk plans and logs are subject to inspection. ICH E6 (R2) establishes the expectations that risk plans are to be developed during the protocol development stage, and that they be actively managed throughout the trial, with a final assessment of the effectiveness of the risk plan included in the final clinical study report.

Which entities are risks communicated to? Does this apply for sites, too?

Risks relate to all stakeholders involved in clinical trials. If a stakeholder group will be impacted when a risk becomes an issue, or if they could be part of a mitigation or contingency to prevent or minimize a risk from becoming an issue, they should be part of the communication scheme. This includes sites, CROs, providers, etc.

Based on the Avoca tool or Risk Assessment Categorization (RACT), do you recommend a set of lessons learned checkpoints that should be reviewed for each study?

Common lessons learned checkpoints may include: protocol approval, study startup completed, 25% patients enrolled, 50% patients enrolled, 100% patients enrolled or last patient in, last patient completed, database lock, clinical study report complete. This does not mean that lessons learned should be done at all these timepoints, but they are logical timepoints to be selected.

Should QTLs be defined in metrics, or is classifying risks as high to low okay? If the high to low classification is okay, wouldn’t metrics still be necessary? Is there a standard convention available for classification of risk (Low, Medium, High)?

For QTLs, it is generally anticipated that these will be quantitative measures and metrics. If subjective categorical measures such as High, Medium, and Low are used, they should be tied to clear definitions of what constitutes each category. The AQC Risk Tool with Bubble Plot and the TransCelerate RACT both include standard conventions directly within the tools.

To what extent have we seen sponsors transfer the risk management obligations in ICH E6 (R2) to their service providers?

Many sponsors delegate risk management obligations to their service providers, but ICH E6 (R2) states that sponsors retain accountability. It is recommended, however, that sponsors recognize that risk tolerances of providers may differ from those of sponsors, so it is always important to ensure that a sponsor’s best interests are incorporated.

How is clinical training/protocol specific training impacted by these new requirements, as the sponsor is accountable for CRO delivery?

A sponsor should ensure that training conducted by their own organization, or by contracted CROs, addresses the requirements for risk management plans, QTLs, and other factors related to managing patient safety and data interpretability.

What would be the impact of QTL changes to any metrics reports that were captured prior to the change?

Each individual company should assess their risk tolerance and the requirement for QTLs, relative to agency expectations for metrics reports from trials that may have preceded the change. If a trial is a multi-year trial that will continue for quite some time after ICH E6 (R2), then an organization should consider if it is of greatest value for patient safety and interpretability of data, to implement QTLs for the duration of the trial.

Does ICH indicate the section of a Clinical Study Report where QTLs should be described?

ICH E6 (R2) Section 5.0.7 references (ICH E3, Section 9.6 Data Quality Assurance).

Inspection Preparation With a Risk-Based Approach

Is each agency releasing their own data integrity guidance? Is there a standard for what should be included in a data flow diagram, or an example template available?

Data flows are highly recommended; however, highly variable per sponsor, study, etc. It is important that the diagram include data from all sources (including sites, external vendors, etc.) through to the generation of the tables/listings and any other format that goes into the CSR. Below are key references on data integrity from the FDA, EMA, WHO, and MHRA:

Have there been any audits of sponsors which have implemented the Avoca oversight tools? Have they been found acceptable?

A case study was presented by Alexion during the 2015 Avoca Quality Consortium Fall Members’ meeting, regarding an agency inspection of their pharmacovigilance quality management system. They had used Avoca leading practices. Upon completion of the inspection, the inspector indicated that it was the most effective management system he had seen. This case study is available to all AQC Members on the AQC Knowledge Center. Other examples are available upon request.

When an inspector asks about SAE reports in the TMF, is it acceptable to document in the TMF/notify the inspector that the Safety Database is the central repository of safety data? Do we also need to enter all SAE information in the TMF if everything is archived in?

Yes, it is acceptable to reference the location of source safety data in your TMF index and direct an inspector to the Safety Database as the central repository for safety data. Guided access is acceptable for certain data records within an eSystem, in addition to TMF system direct access (examples include audit trails such as eCRF/CTMS, eCRF, CTMS). This is due to the technical nature of some of these systems, such as those containing data rather than documents.

How do inspectors typically want to view an eTMF’s audit trail? Do they want access to the system’s back-end, or is an export of the audit trail accepted?

The approach differs by agency, and certainly the MHRA are known to request access to TMF metadata, including audit trail. If/where there is reason for which access to system back-end is not available, provision of an exported audit trail can be negotiated, provided it is reliable (i.e., from validated system).

Previously, FDA had not accessed systems directly (in breach of their insurance) with the preference to request the documents or driven review. Has this now changed?

The FDA has continued to predominantly request that documents be brought to them; however, they are shadowing EMA and MHRA and gaining awareness and interest in potentially accessing eTMFs directly.

This question pertains to the eTMF and certified copy requirement. In a scenario where the site process is to retain original essential docs (i.e., 8.2, 8.3, 8.4 – site), and the site does not have a process in place to send in certified copies for filing an eTMF, how do you practically deal with this?

It is common for sites to retain their original (wet-ink signed) documents. Sites either send copies to the monitoring organization, post a scanned copy of the original in the eTMF, or the site monitors collect it for filing to the TMF/eTMF. Since the site will have the original documents in the Investigator Site File, which are verified by the site monitor, it is not typically expected that a certified copy process (since the monitor verification step covers this) is implemented.

Are there recurring cases where inspectors are not willing to take required training before receiving access to a system such as EDC or eTMF?

Yes. MHRA, and sometimes EMA, expect that the eTMF will be straightforward in terms of use, such that they should not require training. They request direct access with minimal orientation.

Has the FDA begun inspecting against ICH E6 (R2)? If so, what have their focus areas been?

Interviews with AQC Member companies have indicated that the FDA has not yet inspected companies against ICH E6 (R2), since most studies currently being inspected, occurred prior to implementation of the new guidance. Inspectors have, however, requested to see sponsor risk plans and QMS documents/procedures, which indicate that they are implementing new studies according to the guidance.

Did you see any sign that actual inspectors do not accept the risk-based approach for certain aspects of a trial? We received critical(!) finding from EMA recently for mistakes in 2 CSR listings. These listings were classified as “low risk” by our stats team. I still stand puzzled, as the finding was that we had mistakes in two listings, and not that the risk rating was wrong. The “low risk” listings were QC’d of course but not double programmed like the high-risk listings/tables.

We have received anecdotal comments from Member companies that individual inspectors may still identify findings within inspection reports, even when the issue had been identified within the risk plans as low risk, and therefore not managed by risk plans. These seem to be inspector-specific and not a broad trend.

What is the approach to using KRI in Phase Ib/IIa studies?

Key Risk Indicators (KRIs) are appropriately used for all phases of clinical research. For shorter duration Phase Ib/IIa trials with fewer patients, the risk triggers and metrics need to be selected in a fashion to permit early detection of appropriate thresholds.

Should we prepare a Phase I site for inspection, even though we are not sure if we will use the data to support the registration application?

This should be part of your risk-based plan as you determine the likelihood of an inspection, the potential implication for your product, market approval application, or other factors, should the site be inspected. Please note that some agencies inspect for reasons other than market authorization application.

Do you have examples of a vendor oversight plan that you could send us?

The AQC Knowledge Center has a jQMP Template as well as other oversight plan documents.

Why do we need storyboards when we have eTMF? How will these storyboards be of benefit? Are storyboards recommended?

Storyboards do not duplicate or conflict with what is stored in eTMFs. They are used to address a gap or issue identified by employees, or via an internal audit (either by a sponsor study team, a provider, or an investigative site). Should an inspector identify the gap or issue, it would be beneficial to have a pre-developed response explaining the situation, indicating if it did or did not have a material impact on patient safety or data integrity, and what was done to minimize the risk or implication. Storyboards are extremely useful for proactive inspection readiness, or when preparing for an impending inspection. They keep everyone on the same page (various functions, sponsor, CRO, and other vendors) so that when it comes time to explain the situation during an inspection, there is a consistent story that is told with confidence.

What is the overall trend for use of notes-to-file or storyboards, and how are they received during an inspection?

The use of notes-to-file is not recommended for compliance to industry standards. Storyboards, as described in the answer above, are becoming utilized frequently as internal prep documents for inspections. Use of notes-to-file should be limited, and they should only be used when they are comprehensive (for example, provide a full justification with the explanation/description of why there was no or low impact). They should also include a summary of the corrective/preventive action plan that was carried out.

Do you share the storyboards with inspectors?

Storyboards are usually for internal use only to prepare for inspections.

Inspection Readiness With Proactive Risk-Based Oversight

As a sponsor being inspected, how do you prepare your key vendors for the inspection, and work with them during the inspection?

The expectations for the vendor should be clearly set out in the quality agreement. Vendors need to be engaged in a state of inspection readiness during the study and leading up to the inspection. For example, during the study, vendors should focus on identifying and evaluating the risks pertaining to the aspects of the study they are responsible for. Upon doing so, the vendor must communicate appropriately with the sponsor, in order to effectively manage the risks that are likely to have the greatest impact. They should also work closely with the sponsor leading up to an inspection, ensuring essential documentation is in order, and participating in inspection interview training and logistics preparations as needed.

How do we teach/liberate the Clinical Operations staff to really embrace the risk-based philosophy of ICH E6 (R2)?

ICH E6 (R2) has now been adopted into legislation in many countries and is essentially a requirement. Moving forward, inspections are likely to focus on how organizations implement and apply risk-based approaches; some inspectors are already inquiring on how companies intend to comply. Hence, all parts of the organization will need to be educated and aligned. It is helpful to understand and promote the benefits of risk-based approaches; it is not possible to control everything, so by identifying and evaluating the risks, you can prioritize and focus on mitigating those with the highest potential impact.

What are some of the most common issues/hurdles you hear about, from companies trying to implement risk-based oversight tools and practices?

Common hurdles include resistance to change, lack of support and resources from senior management, and the absence of a quality culture. Additionally, varying knowledge levels pertaining to risk-based approaches within an organization (e.g., Clinical Operations versus QA) can be an obstacle. The 2017 AQC Research Report has further examples.

Please define “inspection readiness” in more detail. Many people believe that inspection readiness relates specifically to TMF, but what else should be considered when working towards being inspection ready?

There are varying definitions of inspection readiness. Avoca and the AQC define inspection readiness as a state of being, in which Sponsors, CROs, Clinical Service Providers, and the like, have SOPs in place to ensure that the applicable regulatory guidelines are adhered to. In order to achieve this, there must be a focus on GxPs and quality throughout the clinical program. Inspection Readiness is a culture in which there is a strong commitment to quality and proactive risk management. Elements of this culture are embedded into the organizations SOPs and working documents, and by adhering to these guidelines while performing daily activities, an organization will establish an ongoing state of “Inspection Readiness”. The term encompasses a more holistic focus on overall quality of trial processes and data, as well as compliance with all applicable regulations. A vital component of this is, as you pointed out, maintaining an “inspection ready” TMF.

What do you consider key success factors for reaching an inspection readiness state?

A culture and mindset of inspection readiness and focus on quality is promoted by having clearly visible and unwavering senior management support for quality initiatives. Organizational vision, values, and objectives relating to quality should be clearly articulated, and there should be performance expectations that clearly link to quality goals, for all individuals throughout the company. A culture of quality also promotes active, ongoing, and open engagement with customers to continually identify and address current and evolving risk and needs. If you are an AQC Member, you can obtain additional information on characteristics of a culture of quality here.

How are you preparing sites identified as “higher risk to be selected for future inspections”, for accelerated access submissions?

As part of a risk-based inspection approach, it is important to first identify those sites which are of higher risk to be selected for inspection. A risk-based approach should also be adopted in preparation for an inspection and is applicable to all clinical sites. This involves intentional and thoughtful identification, evaluation, review, and communication of the risks, especially those risks which may negatively impact the inspection. Those risks which pose the greatest threat to the inspection must be managed and mitigated. To assist with site preparation, the AQC has developed a wide variety of tools, such as a Site Preparation Checklist and a Site Inspection Coordination and Logistics Template. If your company is a Member of the AQC, you can access all the documents in the Knowledge Center.

What is the difference between a KRI and a QTL? Are QTLs derived from KRIs or are they unrelated?

Key indicators are metrics which are important in the conduct and control of a study or program. These key indicators may relate to performance (KPI), quality (KQI), or risks (KRI). The thresholds for these indicators must be relatively sensitive, in order to detect troubling variances which should be mitigated or minimized. Potential resolutions for troubling variances must be well defined, to ensure that appropriate action can be taken without delay. In contrast, QTLs (as defined in ICH E6 (R2)) are used “to identify systematic issues that can impact subject safety or reliability of trial results.” Simply put, a QTL is the limit of a parameter. If a QTL is reached, this should trigger an evaluation to address the issue, as subject safety and reliability of trial results are compromised beyond that limit. Please see additional details in the following article.

How do you incorporate inspection readiness as part of study start-up, and not just in the post-submission process?

A risk-based approach to inspection readiness involves intentional, thoughtful, systematic assessment and analysis of the risks, in relation to both study planning and conduct, as well as inspection preparation. This must be done while working abreast the specific areas of focus of regulatory agencies worldwide. From the start of the study, it is important to identify, evaluate, control, review, report/communicate the risks which may impact inspections. The aim is always to prioritize and direct efforts towards those risks which will likely have the greatest impact on the study, should they manifest.

How are tolerance limits integrated into the inspection readiness process?

With the new standards laid out in ICH E6 (R2), QTLs are essential to the inspection readiness process. QTLs must be defined at study start-up and monitored/acted on appropriately throughout the study. Please see additional details in the following article and related webinar.

Do you prepare “storyboards” well in advance? If so, how do you choose topics?

It is leading practice to develop storyboards as issues and/or needs are identified, rather than waiting until inspection preparation. The choice of topics can be based on areas of inspection focus (such as FDA, BIMO, or EMA GCP Inspection guidance) or output from risk identification and evaluation activities. For example, if a risk area such as high staff turnover is identified as part of a quality assessment, a storyboard can be prepared and maintained throughout the study, with a list of names/roles/responsibilities. Working in real-time prevents the burden of compiling information retrospectively. By doing so, the information is ready to be used for additional critical inspection preparation activities, such as interview training or mock inspections.

What is the difference between a storyboard and a BIMO? Can one feed into the other?

The Bioresearch Monitoring Program (BIMO) is the FDA inspection program, while storyboards are tools to support inspection readiness. For example, storyboards can facilitate SMEs in telling the study “story” during an inspection. A storyboard document can be put together to provide the key details behind a non-compliance issue, or to describe a complex process or study design. The FDA BIMO website describes areas of focus and inspection trends. This website, along with additional information from the various health authorities, assists in identifying areas of risk. Once risks are identified from this available information, storyboards can be created and used to help prepare SMEs on topics that may be brought up during inspection.

Does the AQC offer a Computer System Inventory Log, which would detail validation status, version, purpose, etc.? Inspectors want to see this.

To be inspection ready, organizations should compile details and records of all IT systems used. The records should contain written SOPs/business guidelines (including historical procedures for the period when the studies were conducted), Software Development, Validation, Access Control, Security of Systems, Change Control, Archiving, Backup/Recovery, and Disaster Recovery/Contingency Plans. An Inventory Log is a useful documentation tool for this purpose. The AQC has also developed a Functional Inspection Preparation Checklists (for Sponsor/CROs and Sites), which includes a list of all items that should be documented, pertaining to the computer systems used in a study. If your company is an AQC Member, you can access these checklists on the Knowledge Center.

Risk-Based Monitoring

How have sites adapted to a risk-based monitoring mindset? In our experience, sites oftentimes confuse RBM with remote monitoring.

This is completely true, and it is not just the site level; the whole industry continues to confuse risk-based with remote. Sites are only beginning to adapt to risk-based monitoring. To really adapt, sites must develop their own internal QA/QC programs, identify the type of monitoring plan prior to contract and budget negotiation, identify what qualifies as ‘high risk’, and improve accordingly. For clarity, remote (or off-site) monitoring can be a component of risk-based monitoring, as RBM encompasses a thoughtful balance of Centralized, Remote, and On-site monitoring in a risk-based fashion, focusing on risks that matter to patient safety or data integrity/interpretation.

What are your thoughts on early development or Phase 1 studies and risk-based monitoring?

All trials deserve a risk-based approach, but the earlier the phase, the higher the risk. Earlier phase studies deserve RBM even more.

Is there data that suggests CRAs should continue to 100% SDV during on-site monitoring visits (safety assessments, AEs, medical history, concomitant medication), in addition to the primary endpoint?

I do not believe 100% SDV provides benefit compared to a risk-based approach. Most datapoints have nothing to do with the trial endpoints. Tufts’ analysis shows that, on average, trials produce 1 million datapoints that never get used. There is a waste of time and resources due to 100% SDV, whether early phase or not.

Why don’t all sites implement a quality program? Sites should not be solely dependent upon a CRO or Sponsor to inform them of their quality standards/metrics.

I completely agree with you. Today, only the largest sites have their own quality programs. Real quality programs are expensive and difficult to implement and maintain. Sites aren’t reimbursed in a way that encourages or incentivizes them to implement a quality program. Most sites are too small or too poor to do so, but that needs to change.

Would it be beneficial if QTLs were outlined in the protocol for transparency to the sites?

Absolutely! We need full transparency on the data that matters, the algorithms that are used, and the datapoints that appear to be outliers.

If the monitoring plan were to be shared with the site, what benefit do you think it will have in the quality/performance of the site?

The monitoring plan has a budgetary impact on the site, but if you want the site to be a member of your team, the site must know the rules of the game.

In order to be more transparent to sites (KRIs, QTLs, etc.), would you prefer having an electronic dashboard or, for example, a monthly newsletter sent?

Great question! I would always prefer an electronic dashboard over a monthly newsletter.

Is there a risk of sharing risk-based monitoring plans with the site? I understand the need for transparency, but won’t the site be diligent only for those subjects/visits that are identified to be monitored?

Risk-based monitoring doesn’t mean you’re only going to look at certain subjects or certain visits. It means you’re going to look at all the data coming in, and look for outliers that suggest heightened risk, and datapoints that suggest anticipated behavior. Ultimately, all data is analyzed in the algorithm. Sites would not have the option/ability of doing less high-quality work.

What is/are the best way(s) to keep up with changing regulations in order to remain compliant?

Participating in industry conferences, establishing a regular cadence of reviewing regulatory webpages, and/or joining consortia like ACRP, Avoca Quality Consortium, etc.

What does quality tolerance limit (QTL) mean? Do you have examples?

A quality tolerance limit is a threshold on a parameter that necessitates an action. An example of a parameter that would have a corresponding quality tolerance limit is patients lost to follow-up. In a clinical study, it is often known how many patients need to complete the follow-up portion of the study to have a high enough statistical power to determine scientifically the effectiveness and safety of the drug. If too many patients are lost to follow-up, the study becomes unevaluable. Therefore, a QTL associated with this parameter would be a value that is lower than the known number of subjects that can be lost to follow-up, without losing power for the study. If that QTL were to be met, there would be actions required, such as re-training Investigators and sites on the importance of not losing patients, avoiding enrolling patients who may not complete the follow-up period, and determining if prior patients who were lost to follow-up can be confirmed deceased, etc.

If I have 150 sites, it would be a gargantuan effort to tailor the Site Monitoring Plan based on site experience. Do you agree?

Yes, a better plan would be to stratify the sites based on their level of risk and have a Monitoring Plan that designates how high/medium/low risk sites will be monitored.

If we agree that 100% SDV is largely a waste of time, should we also be re-evaluating the data that we are collecting? In some cases, it seems like the volume is a catch-all that doesn’t always provide value.

Absolutely! The complexity of many studies often has less to do with the primary endpoints, but the secondary and exploratory endpoints. By increasing the complexity to gain data that may or may not be used in the future, we are putting at risk the evaluability of the current trial.

How do we change the mindset away from 100% SDV?

Experience. Until the industry is provided a case of exemplary risk-based monitoring implementation, and everyone is aligned on why it is beneficial, it will be difficult to get everyone on board. Similarly, because it is new and there have not been many inspections on studies using risk-based monitoring, there are still too many unknowns. Our industry is risk averse, so until there are more experiences shared on risk-based monitoring inspections, findings or lack of findings, it will be difficult to change the mindset those that are accustomed to the status quo of 100% SDV.

If we decide <100% SDV is beneficial, how do we decide what percentage should be SDV? Is it across all patients?

I have seen studies where the first patient at each site is 100% SDV’d but then the SDV percentage decreases after to only key datapoints. The SDV percentage can also be based on the risk level of the site and not necessarily at the patient level.

Given Investigator responsibility for quality, shouldn’t there be a Site Quality Management Plan and or/quality metrics built into the Site Clinical Trial Agreement?

This is a brilliant idea and is an area of focus for the Avoca Quality Consortium in 2019.

Should CROs/Sponsors start thinking about establishing QAGs (Quality Agreements) with sites?

Absolutely, and this is an area of focus for the Avoca Quality Consortium in 2019.

It sounds like you are talking about sites that are busy with many studies. What about academic centers that run studies? How do Sponsors/CROs influence them to ensure quality is included (regardless that it has always been their responsibility)?

Academic Centers are equally as responsible for quality as other sites, and the only way to ensure that quality management is implemented, is by penalizing those sites who are not providing high quality. That penalty is to not utilize them in the future. If you still choose to utilize those sites, know that perhaps a more stringent Monitoring Plan would be required for this Academic Center, as they would be a higher-risk site.

TMF Inspection Readiness

Can you provide some guidance around what is considered “timely” filing of documents?

There is no specific regulatory guidance regarding timeliness, but it is expected that timelines for filing of documents should be defined in procedural documents or TMF plans. Within the industry, 30 days from creation of record to filing in the TMF has often been targeted, but with only varying degrees of success. Your actual targeted cycle time will depend upon the How, Where, and Who associated with the creation of the records. Documents created and approved within the eTMF are filed automatically (immediately), whereas records created at remote locations (e.g., sites, CROs) may take longer — particularly if scans are required. Understand the How, Where, and Who associated with your record types and plan or target accordingly.

What is the industry standard for the frequency of QC checks on an ongoing TMF?

There is no agreed industry standard for frequency of QC checks of TMF records. Some companies conduct QC checks on a monthly basis. At other companies, QC checks may be at trial milestones, and then annually thereafter for long-running trials. Documents that are created and filed through validated processes should not require as many QC checks as manually produced records. Conduct a risk analysis of your record types (e.g., criticality of the record, risk of record not being correct) and develop a quality checking process accordingly.

Do the regulatory agencies agree to be trained on the TMF before they are given access to it?

Regulatory agencies acknowledge that inspectors may need to familiarize themselves with the eTMF. If they opt for training on your system, it should take no more than an hour. Consider having an eTMF support person on standby, if questions of how to navigate the eTMF happen to arise.

How do you determine expectedness — is it based on the stage of the trial?

Yes, you will likely tie expectedness to a stage or stages of a trial. This helps with metrics and reporting. For example, at time of First Patient Visit within a specific country, what documents and records would you expect to have generated and filed by then?

Are we required to have an auditable electronic TMF/QMS for European studies?

European regulations do not mandate the use of electronic TMFs; paper TMFs and hybrid TMFs are acceptable.

How do you ensure email communication is inspection ready, always? Is there an industry standard for filing email communication?

Email is challenging. There is not an industry standard for filing email communications in the TMF. The TMF Reference Model is helpful, in that it establishes structured and standardized filing locations for relevant communications — but the greater issues are getting teams to recognize when they’ve generated or received relevant communication, and the logistical means for getting those communications into the TMF. EMA inspectors will prefer reviewing emails in their electronic state (and ideally in their native system) over having the emails stored in flat PDF files.

What is your opinion on documents which are collected in ISF, and mirrored in eTMF? ICH-GCP states that collection in ISF is sufficient. ISF is a part of TMF as eTMF is a part of TMF.

You are correct in that the ISF (Investigator Site File) is a part of the overarching TMF; the other half of the TMF is the sponsor TMF (not necessarily eTMF — eTMF is a medium of a sponsor TMF). ICH E6 (R2) GCP states that there are many records that need to be filed in both the site TMF and the sponsor TMF. Your comment, “collection in ISF is sufficient,” may be in reference to the source records. For a record that originates at the site but must be mirrored in the sponsor TMF, the original record (the source record) is filed in the ISF and a copy of the record is stored in the sponsor TMF.

With respect to the E6 R2 update involving sponsor oversight, how frequently would you recommend conducting a “health review” of the TMF at the CRO partner? Quarterly, semiannually, annually?

You may choose to outsource duties or functions to a CRO, but as sponsor, you remain accountable for the conduct of the trial — and for reflecting that conduct in the TMF. As for auditing and checking the CRO TMF, at the very least, you need to agree on the arrangements for quality oversight (including audits) with the CRO. Document the agreed approach and resulting QC reporting/auditing and file this agreement in the TMF. If the trial is completely outsourced, and only project management oversight is provided by the sponsor, then you may want to arrange regular (monthly, quarterly) reporting of TMF quality metrics. This should be followed by periodic deep dives at project milestones, and annually thereafter for long trials. Adjust your approach based upon your experience with the CRO; if they’ve had difficulty keeping their TMFs up to date, then increase the frequency of TMF quality reviews. The inspectors will likely review the CRO TMF audit trails to see if you have been accessing and reviewing the CRO records. This is necessary for you to discharge your trial oversight duties effectively.

Are there any differences in TMF as it applies to Phase 1, 2, or 3 trials?

The general requirements for a TMF apply to all phases of clinical trials. Phase 1 trials typically have fewer document types and total records. Larger trials are more likely to have complicated TMFs, and confirmatory trials are more likely to be inspected. Having said that, the stated requirements and guidelines apply to all phases of clinical trials.

What documents are required to be wet-ink signed?

From a regulatory standpoint, there are some documents that are required to be signed (e.g., protocol, 1572, contracts), but not many. It is more likely that internal policies and procedures will increase the number of records that need to be signed. The explicit requirement for wet-ink signature is also going to be driven more so by internal policies and procedures, rather than by regulatory requirements (a signature log being a notable exception — by definition, it requires wet-ink signatures). Minimize the number of records that need to be signed and maximize the number of those that can be executed with e-signature.

If a CRO has eTMF and sponsor has paper TMF, does the eTMF need to be incorporated in the paper TMF, in order to fulfill the requirements of a complete TMF, or can they remain distinct?

You are describing a hybrid TMF, which in this case, is a hybrid of a pTMF with an eTMF. They may remain distinct, provided it is communicated with the inspector that you have a hybrid TMF. You need to assemble a comprehensive inventory of all records and locations and establish how to make the records accessible to the inspector.

What should you do with paper files if your company is transitioning from a paper to an eTMF?

Develop a cutover strategy, such as, from date X-onward, all new trials are to be managed through an eTMF. For ongoing trials, perform a risk/cost/benefit analysis to determine where it makes sense to convert from pTMF to eTMF. For pTMF trials that are being converted to eTMF, a formal process must be employed. Note that some local regulations require the retention of original paper records. Be sure to conduct a legal review of your TMF documents before dispositioning any records for destruction.

Essentially, the paper fraction of the eTMF is site based. In what way would that be accessible in case of an inspection?

There are several records that are required to be filed in both the investigator TMF and the sponsor TMF. For those records that originate at the site, copies should be made and filed in the sponsor TMF. If the sponsor TMF is an eTMF, scanned copies of the originals may be filed.

What are the top findings that you’ve experienced during an FDA inspection regarding TMF findings?

The FDA, along with the other global regulatory authorities, regularly publishes Inspectional Observation Summaries that detail their findings. Specific to the FDA, they are more likely to take issue with the content of records, rather than issues with the TMF itself. Where there are references to documents missing, it is often regarding “Records of receipt, shipment, disposition” and “Records of unused drug disposition”.

Is there an industry standard EDL (Expected Document List)?

No, but the TMF Reference Model ia a good start. Depending upon the functionality of your eTMF and your specific approach to managing expectedness, your expected list may need to reflect the specific records (and record names) that your company’s and CRO’s procedures and processes generate.

I work at a small Sponsor using multiple eTMF vendors across our various clinical trials. Do you have thoughts on best practices for integrating these, to allow for access to all eTMFs via one system?

Currently, there is no standard means of interfacing or integrating the various eTMF systems. Consider instituting standardized processes, practices, and terminology — that is used across all CROs. The TMF Reference Model would be a good starting point. That way, the sponsor team members that are providing oversight know where to find trial information, regardless of the user interface required for accessing it. Some sponsor companies have procured their own eTMF and required the CROs to use it. This provides simplicity from a sponsor management and oversight standpoint, but it may introduce a complication to how the CROs execute their work, which in turn may result in added charges to the sponsor.

Have the regulators suggested electronic signature systems (e.g., Adobe eSign) to use, or provided guidance on expectations for e-signatures?

Both the FDA and the EMA accept Adobe e-signature functionality for electronic submission of certain regulatory forms in pdf format (e.g., IND). Guidance on Electronic Records/Electronic Signatures is available from the various regulatory agencies.

How do you measure or estimate the completeness of significant correspondence records, for which the expected number of records cannot be easily established?

Be judicious in focusing effort on getting the records filed in the TMF versus managing estimates of expectedness. Start with training the team on relevant communications, to make sure that they recognize when they’ve created or received relevant communications, and ensure it is known how and where to file them. At a minimum, have an informed team member conduct periodic checks in the relevant communication file locations. Does it look like the correct issues and communications are being filed? Make general approximations of expectedness based upon initial rates of filing.

More and more mobile applications are available for the creation of PDFs or scanned documents. What are the guidelines for cell phone created documents, in regard to Part 11 compliance.

It is possible for mobile phones to produce documents by utilizing their cameras to scan documents. Basically, the phone replaces the copier/scanner, but with enhanced connectivity. Check with your eTMF provider to determine if they have mobile phone applications for their system. Procedurally, sponsor companies and CROs may not allow the use of phone cameras to conduct record scanning.

Responses to questions provided by Avoca Subject Matter Experts (SMEs).