FDA Engages Lab, Dx Players by Proposing NGS Regulatory Standards Come From Community

NEW YORK (GenomeWeb) – The US Food and Drug Administration indicated last week that it doesn't want to dictate analytical standards to developers of next-generation sequencing tests, but would rather the experts come up with them as a community.

"Our ideal case … is you, the community, develop standards in a way that we can recognize," David Litwack, a policy advisor within the Office of In Vitro Diagnostics and Radiological Health (OIVD) in FDA's device division, said during a public workshop Friday to discuss two draft guidances on NGS tests the agency released over the summer. 

One guidance outlines principles for demonstrating the analytical validity of NGS tests for hereditary diseases, and the other lays out how administrators of publicly accessible genetic variant databases might garner the agency's recognition, so data in these repositories could provide clinical validity support for genetic tests.

The FDA held the workshop to garner input from stakeholders, who can also submit formal feedback through the Federal Register until Oct. 6. During the meeting, most participants seemed generally supportive of FDA's proposal to advance a regulatory framework that can quickly evolve with rapidly changing technology, and constructively engaged with the agency.

One key suggestion that came out of the workshop was to expand the use of the PrecisionFDA cloud-based platform for NGS analytical validation. PrecisionFDA, which the agency launched last year, provides a forum for the life sciences community to share datasets, analysis pipelines, and bioinformatics tools. The FDA notes on the platform's website that the tool isn't used to make regulatory decisions but can "generate knowledge to inform future regulatory pathways and decision making."

"I would like to see the FDA precision website be able to handle the data we use for validation, so we can all kind of know that we validated [a test] the same as everybody else," said Joe Devaney, a staff scientist at Children’s National Health System. "

Devaney acknowledged that when he set up NGS at Children's, it was difficult to figure out how to proceed. "There might be a good way to interact with the community to figure that stuff out on the FDA precision website," he said.

The draft guidance on analytical validation currently only addresses targeted and exome sequencing for inherited conditions. At the workshop, stakeholders had different ideas on how prescriptive the analytical validity guidelines needed to be, and whether there needed to be separate recommendations based on intended use or based on the type of testing being performed, such as for germline conditions, for somatic markers, or mitochondrial disease. Experts noted, for example, that while the validation issues might be the same for amplification- and capture-based NGS tests, the technical specifications, such as library complexity and depth of coverage, would differ.

Regardless, workshop attendees seemed to be advising FDA against setting out specific analytical thresholds, particularly since stakeholders have already challenged some of the thresholds currently in the draft document. "The beauty of this draft guidance … is that it recognizes that there is a lot of complexity, and a lot of different ways to do this," John Pfeifer, vice chair of clinical affairs in pathology and immunology at Washington University School of Medicine, said at the workshop. "So, it's establishing general guidelines of what laboratories need to take into account in terms of being transparent in their assay design and being down in the weeds … to demonstrate the quality metrics of testing they're actually doing."

Pfeifer was at the workshop representing the accrediting body College of American Pathologists, which has published a checklist of requirements that labs performing NGS have to address. If the FDA relies on experts in the community to develop NGS standards, then CAP is likely to play a key role, although the organization hasn't officially said it would. 

"CAP's position is that the FDA is enlightened to pursue an approach in which they will rely on professional organizations that have a lot of depth of experience in this to actually help set the standards, develop the guidelines and the quality metrics," Pfeifer said at the workshop.

However, Sherri Bale, managing director of GeneDx, felt that the FDA's guidance on NGS is superfluous. "The validation recommendations that are described in the [American College of Medical Genetics and Genomics] guidelines, the CAP guidelines, the New York State Department of Health guidelines, and [the Clinical Laboratory Improvement Amendments] on how to validate a genetic test covers basically everything that we need in order to go forward," said Bale, who was an author of ACMG's NGS standards, which were published in 2013. "I don't think there is anything additional that strikes me that needs to be addressed."

She noted ACMG's position that any changes to regulation of lab-developed tests (LDTs) — a category that most NGS tests currently fall under — should be done through changes to CLIA, laws that have historically regulated lab procedures and have been administered by the Centers for Medicare & Medicaid Services. The FDA believes that CLIA is inadquate to ensure the safety and reliability of modern-day LDTs and plans to oversee them, a move that pathologist groups and members of the lab industry have lobbied hard against.

Many in the life sciences community looked to the draft NGS guidances suspiciously, as FDA's underhanded way of imposing regulation over a rapidly growing and particularly complicated subset of lab tests. The FDA workshop followed a Senate committee hearing earlier in the week where industry players testified that FDA regulation of LDTs would hinder patient access to the most innovative tests, particularly during public health outbreaks. 

The FDA has presented its NGS draft guidances as a dictate from President Barack Obama to ensure, as part of the Precision Medicine Initiative, that personalized medicines reach the people who will benefit based on accurate and reliable test results. "We think this is a sensible way to support and encourage innovation and make sure the tests work," FDA Commissioner Robert Califf said at the workshop, referring to the flexible framework the agency wants to advance for NGS tests.

"This actually is one of the key places that we can show that innovation and regulation don't have to be polar opposites," Califf said, seemingly responding to some of the criticisms volleyed at the agency during the Senate hearing where FDA was conspicuously absent from providing testimony. "In fact, good regulation can stimulate innovation."

The FDA gathered stakeholder input through multiple public workshops and private meetings ahead of issuing the draft NGS guidelines. Elizabeth Mansfield, deputy office director for personalized medicine at FDA's device division, acknowledged that the analytical draft guidance includes similar recommendations to those put forth by professional societies. "I think that should be taken as a good thing that we're not disagreeing," she said.

While lab professionals were more willing to discuss the NGS draft guidances than they have been to talk about the FDA's draft guidance on LDT regulation issued two years ago, they pointed out a number of areas that will be challenging for the agency to address because there is little consensus within the life sciences community.

Specifically, when discussing FDA's second draft guidance on the use of public genetic variant databases to support the clinical validity of NGS tests, workshop attendees debated how changing classifications in a recognized database would impact labeling and submissions for an approved or cleared diagnostic, and how to deal with discrepancies between two recognized public databases.

A number of workshop attendees noted that there is a lag between professionals deciding to change a variant's classification based on the latest evidence and when that knowledge finds its way into a database. Such delays have implications for diagnostic developers using these repositories to establish clinical validity and label tests. Mya Thomae, a regulatory expert at Illumina, noted for example that when information on a variant is removed from a database, it would likely impact test labeling.

The experts all agreed that databases need to be transparent when it comes to standard operating procedures, conflicts of interest, and discrepancies in variant classifications between labs and with other databases. One suggestion was for FDA to conduct annual reviews, during which the agency would conduct a kind of proficiency testing for databases. "When you're in the process of recognizing a database … you actually compare the assertions to assertions in other databases, if they house the same variants" and flag discrepancies, suggested Peggy Carter, who heads up drug regulatory affairs within Novartis' companion diagnostics division.

"We kind of need to know how the data were generated" and how reliable they are, she added. "It would be good to know the test that generated the data."

Christa Martin, director of the Autism and Developmental Medicine Institute at Geisinger Health System, also told FDA to create standards around data quality, noting inaccuracies in public databases where variant data is drawn from the literature. Inaccuracies in public variant databases is a major reason some labs say they are currently not submitting to repositories such as ClinVar, a freely available archive of genotype and phenotype relationships the NIH publicly launched three years ago. 

Although public databases are a key feature of FDA's vision for easing the regulatory path for NGS tests, labs that don't want to share their data in such repositories, don't have to. To date, the FDA has used Johns Hopkins University's freely accessible CFTR2 database to clear Illumina's MiSeqDx platform for cystic fibrosis, and Myriad's privately held database to approve its BRACAnalysis CDx, which is not an NGS test. Agency officials explained at the workshop that the draft guidance sets out a process for recognizing public databases that can be used to establish the clinical validity of NGS tests, but sponsors could also use proprietary databases to gain regulatory approval of their diagnostics.

The FDA, however, asked workshop attendees to weigh in on the definition of a "public" database, and whether that means the repository had to be free of cost to all or could charge fees. Workshop participants had divergent views on the best model. "We're in support of free and open data sharing," said Martin, who is involved in ClinGen — an effort, in collaboration with ClinVar, to build a genomic knowledge base that can be used in patient care.

Martin noted that patients today are savvy about looking up their genetic variant in the published literature and in databases. "If you put limitations and don't make it freely available for the community, then you're limiting the utility across all the stakeholders that are trying to learn from this knowledge," she said.

In contrast, speaking on behalf of the Association for Molecular Pathology, Madhuri Hegde expressed concern that FDA's focus on recognizing publicly accessible variant databases may restrict access to commercial repositories. For example, Quest and the French National Institute of Health and Medical Research last year launched BRCA Share, which allows free access to researchers, doctors, and patients, but charges annual licensing fees to commercial labs on a sliding scale. 

"Some of the best curated and useful databases … are available through licensing or subscriptions," said Hegde, executive director of EGL Genetic Diagnostics, adding that industry professionals regularly reference both free and commercial repositories when classifying a variant. "The requirement that all databases participating in the recognition program be publicly accessible may deter some proprietary database owners from participating in the program."