NEW YORK (360Dx) – Late last month, a working group assembled by the College of American Pathologists, with representation from the Association of Molecular Pathologists, released a practical framework for designing and implementing next-generation sequencing tests for inherited disorders. The framework, published in the Journal of Molecular Diagnostics, is meant to serve as a step-by-step guide for clinical laboratories.
As NGS testing continues to proliferate as a first-line test for patients with suspected inherited disorders, the practical framework aims to improve patient care by facilitating standardization across this line of testing. While there are guidelines and checklists for laboratories performing these tests, the working group designing the framework set out to provide more in-depth guidance than labs can find from other resources. Focusing on variant detection in inherited disease, the framework provides structured worksheets to guide laboratories through the lifecycle of NGS testing, focusing on test design, optimization, validation, and quality management, with additional guidance for clinical bioinformatics.
Two members of the working group – Avni Santani, director of clinical laboratories and strategic partnerships and innovation at the Children's Hospital of Philadelphia, and Birgit Funke, vice president of Clinical Affairs at Veritas Genetics – shared some of the back story of how and why this group came together to create this framework, and how they hope it can facilitate NGS testing in laboratories.
Below is an edited transcript of the interview.
How did this practical framework come about? How was it decided that further consensus was needed around the practical resources for NGS tests?
Birgit Funke: Clinical implementation of NGS began around 2010 and since then, a growing number of guidelines and recommendations have been issued by all major stakeholder organizations in this space (American College of Medical Genetics and Genomics, AMP, Clinical & Laboratory Standards Institute, CAP, Centers for Disease Control and Prevention, US Food & Drug Administration). The CAP is uniquely positioned to gauge the degree to which these guidance documents are translated into real-world clinical NGS testing because it accredits testing laboratories and conducts inspections to ensure regulatory compliance.
Through inspecting policies and procedures of clinical NGS laboratories, as well as their documentation, it has become evident that there is a high degree of variability in how guidance is implemented. One likely reason for this variability is the traditional style in which NGS guidance has been delivered: All existing guidance documents are text-heavy, review article style frameworks that specify what needs to be done but generally do not provide sufficient concrete instructions on how to translate them into laboratory practice.
The CAP therefore created a working group, led by John Pfeifer, professor of pathology and immunology in the division of anatomic pathology, Washington University School of Medicine, to address this gap by creating step-by-step guidance in the form of structured "worksheets" that lessen the need to translate a text-heavy recommendation into practical next steps.
How was inherited disorder selected as the disease area to begin?
BF: This was done mainly for practical reasons and to contain scope: We knew that our work would represent a major shift in how guidance documents are structured, and for that reason it was felt that this approach should be piloted with one NGS application first before rolling it out to additional applications. Because many of the early guidance documents focused on germline testing and the fact that germline NGS is generally somewhat less complex than somatic testing, it seemed logical to start there.
How do you expect this document could be used in connection with a CAP accreditation checklist? Or with other documents that are currently available to provide some guidance in this area? In what way do you see the information in this document enhancing other types of guidance?
Avni Santani: As mentioned earlier, there are a number of excellent guidance documents that have been published over the years and in this article, we provide a list of the recent publications. These are excellent resources to guide laboratories in implementing NGS-based tests. Laboratories would continue to rely on the CAP checklists to ensure they are in compliance. In order to incentivize laboratories to use this resource, we envision that the use of the worksheets is best done in conjunction with the CAP checklists. The checklists provide the requirements and the worksheets will provide them the tools to implement the details related to major NGS-related checklist items.
For example, we know that a clinical laboratory is required to have a quality management program for bioinformatics, but laboratories have to then implement "how" exactly to comply with that requirement. How can this QM program get implemented? What quality metrics should I be using for bioinformatics? Is coverage truly a good continuous quality improvement monitor? The worksheet allows them to make these decisions using some of the specific examples we have provided and also monitor their decisions about the specific metrics and their pass/fail metrics in a systematic fashion. We also envision that these worksheets may be useful to facilitate an inspection progress. For example, a CAP inspector with perhaps a limited background in NGS may be able to rely on these worksheets to guide the inspection process.
The paper mentions that approaches for standardizing NGS tests have been discussed in the industry. Is this paper compatible with those discussions? Is this meant to be a precursor for standardization?
BF: Yes, absolutely! Standardization will take a lot of time but we hope that our resource can serve as a baseline. The clinical NGS testing community is currently discussing various types of standards, ranging from prescriptive, fixed thresholds for metrics such as analytical sensitivity to more flexible concepts that define approaches and principles, and educate on how specific testing scenarios influence performance requirements. The current version of our worksheets was definitely inspired by these discussions. At this point, they mostly reflect concepts rather than prescriptive performance standards.
I should mention that we believe that NGS guidance should ultimately be delivered via a guided web process and our worksheets are an early step towards that goal. If further integrated with regulatory requirements such as the CAP checklist or the FDA NGS guidance, this could create tremendous momentum for catalyzing standardization as laboratories would be incentivized to use this resource while establishing and maintaining regulatory compliance.
Within the different lifecycle stages of NGS testing that this document covers, were some stages more difficult to provide advice for than others? Why or why not?
AS: We were comfortable with the test design, optimization, and validation worksheets because many of us in the working group have been involved in describing these efforts in the past. Our two challenges were developing a consistent format between all the worksheets and making sure we stayed consistent with CAP checklists. The two areas that presented with additional complexities were the bioinformatics and the quality management sheets. For bioinformatics, we discussed issues related to informatics, IT infrastructure, data privacy, and the appropriate tools a laboratory may need to achieving validation in a highly complex IT ecosystem that accommodates rapid innovation and changes.
Quality management is overarching for the whole laboratory and can include areas outside the purview of NGS such as safety, laboratory information systems policies and protocols. At the same time, we felt that the quality control and quality improvement monitors for NGS-based tests needed more granularity. We debated about creating an ideal balance between not being too prescriptive versus providing enough information so laboratories can find these sheets useful.
We also took the strategic decision to not address reporting and interpretation. We decided to limit the scope because of how complex this can get even within the germline space (panels vs exome vs genomes) as well as in terms of indications (diagnostic testing, screening, pharmacogenomics, carrier screening). However, this is something we have on our radar and we hope to work towards this in future iterations.
The paper mentions that this will be a living document. How and from whom do you hope to get feedback? How often do you hope to modify it?
AS: The CAP checklist undergoes periodic review and in order for these worksheets to have long-term utility for laboratories, we foresee that development of an infrastructure will be crucial to allow for timely and regular updates. Since this is the first set of worksheets that we have released, we would love to hear feedback from the community. The CAP website has the documents as well as an email address for the community to provide feedback. As part of the Genomic Medicine Resource Committee at CAP, we are working towards developing a mechanism to ensure that the community feedback is provided to the appropriate CAP committees so that CAP can evaluate and integrate feedback, update checklists and worksheets within the various committee members. In addition, we are hoping to initiate a dialog with other professional stakeholder organizations to work towards harmonizing guidance out there.
This guide looks at NGS testing through design, optimization, validation, quality management and bioinformatics/IT. Are there other areas you expect could be added in future versions?
AS: For germline conditions, there is additional guidance that could be provided during the interpretation process. We discussed this internally and made the decision to address that area in future efforts. Additional guidance on IT infrastructure, data privacy, and certifications related to data privacy and compliance, Sanger confirmation of variants, reporting policies are other areas that come to my mind, where our community could still use additional guidance.
Are there other disease areas that you might explore creating a similar framework for down the road?
BF: Definitely! As mentioned earlier, germline NGS was always viewed as a pilot area as it is technically a little less complex than other areas. Today, the NGS landscape is much more diverse with somatic NGS being widely implemented and other areas such as liquid biopsy, HLA testing, NIPT, and RNASeq following suit. Therefore, the hope is to create similar guidance for all these areas as soon as possible!
One way to accelerate this is via collaborations. For example, the CAP has formal representation on a working group recently formed by the Clinical Laboratory Standards Institute (CLSI), which has additional representation from ACMG, AMP, NYS, and FDA. This CLSI working group is using the CAP worksheets as a starting point to establish similarly structured guidance for these additional NGS applications with the ultimate goal of creating a standard that can be recognized by FDA.