The article has been corrected to reflect Barbara Zehnbauer is an adjunct professor, not an assistant professor, of pathology at Emory University School of Medicine as previously stated. 360Dx regrets the error.
NEW YORK (360Dx) – Aiming to address what it calls an urgent need, a working group has embarked on a pilot project to develop reference materials that laboratories developing molecular tests can use as standards.
In particular, the group seeks to develop reference samples that laboratories can use to develop next-generation sequencing-based tests for determining which patients may respond to specific cancer drugs.
At issue is the lack of reference samples that have been properly vetted so that labs can evaluate how their assays stack up against other labs' tests — especially whether their tests would measure up against those that have been approved by the US Food and Drug Administration, or are in the process of being reviewed by the agency.
In an article published recently in the Journal of Molecular Diagnostics, authors describing the project called the Diagnostic Quality Assurance Pilot wrote that "the validation of laboratory tests in individual laboratories may not itself result in equivalent diagnostic test results. Development of technically sound strategies for demonstrating test equivalence is urgently needed."
In an interview, Barbara Zehnbauer, an adjunct professor of pathology at Emory University School of Medicine and the chair of the pilot's steering committee, added that the main limitation for labs developing new cancer molecular diagnostics has been "the absence of standardized reference materials, reference samples, [and] materials that have been characterized already that can be incorporated into any assay, any kind of platform that will guarantee a certain performance specification no matter which lab."
With so many different laboratory-developed tests available — many of which are never reviewed by the FDA — oncologists and regulators have been unable to separate the reliable ones from those that don't work, and "the need for some reference materials that could be well-characterized and that could be incorporated in any one assay regardless of the technology are a real need to assure dependable, quality, and reproducible performance lab to lab to lab," said Zehnbauer, who is also the editor-in-chief of JMD.
DQAP is the brainchild of Sustainable Predictive Oncology Therapeutics and Diagnostics, or SPOT/Dx, a working group launched in 2013 by Tapestry Networks, which describes itself as a professional services firm enabling businesses, governments, and other sectors to collaborate with each other to address difficult issues.
The goal of SPOT/Dx, according to the working group, was to equip the healthcare field with the tools necessary to "advance clinical decision-making, the diagnosis and treatment of cancer, and the regulatory and reimbursement infrastructure that underlies the field of precision medicine."
For this initial pilot, SPOT/Dx selected cancer NGS-based companion diagnostics as the focus after Illumina and Amgen "offered to put their companion diagnostic out there as a guinea pig, if you will," Zehnbauer said. The two firms began collaborating in 2014 to develop a companion diagnostic for Amgen's colorectal cancer drug Vectibix (panitumumab).
Zehnbauer noted that while cancer is one of the largest growth areas for newer technologies such as NGS, it has been an underserved area in terms of reference samples.
While there are established human cell lines used as reference samples for some cancers, they are mostly for "liquid" tumors, such as leukemias, and developing cell lines for many solid tumors "can be much more challenging," Zehnbauer explained, noting genetic variations, as well as variability in tumor stages and in mixtures of cell types in solid tumors.
"[T]herefore, individual cell lines may not be the best reference samples for solid tumors," she said, adding that tumor tissue may also arrive in the lab in different states, including fresh tissue, needle biopsy, frozen tissue, circulating tumor cells, or formalin-fixed paraffin embedded tissue, making it difficult to "represent all of those formats with reference samples."
Amgen is underwriting the costs to develop the reference samples for DQAP, along with costs associated with project management and lab resources to implement the project. Meanwhile, Illumina will provide technical specifications to DQAP.
SPOT/Dx "wanted to make sure that [an LDT test] somebody else was using would meet the same performance specifications that Illumina is putting forward in a companion diagnostic," Zehnbauer said. "I think the first step is to say that we can actually make performance specifications that are shown by a companion diagnostic, and LDTs can meet that same quality bar."
A spokesman for Illumina said company officials were preparing for an investor conference this week and would be unable to comment. Amgen did not respond to requests for comment.
Participants in the pilot are not required to use Illumina's NGS technology and are encouraged to use different methodologies so that DQAP can compare the technologies, Zehnbauer said.
DQAP will use human cell lines as the reference sample type, rather than actual patient tumor samples, due to challenges in sustaining and stabilizing tumor samples. In the JMD article, the authors wrote that human cell lines can be grown, harvested, fixed, and embedded into paraffin blocks, and divided into sections to approximate human tissue samples commonly used in clinical testing by laboratories.
Zehnbauer acknowledged that using human cell lines are not the same as using human tissue, but in the first phase the plan is to evaluate how the different NGS platforms will stack up with human cell lines. A possible next pilot project could compare the commutability of results across different sample types, including tumor tissue, isolated DNA, and plasma constructs.
And though NGS is the technology focus of the current project, once the cell lines are developed they could ostensibly be used for other technologies, although that is beyond the scope of the current pilot, Zehnbauer said.
"The question is getting something out there that has been demonstrated to give performance in a variety of settings," she said.
DQAP is not alone in trying to tackle the need for reference materials, and over the years, several groups have embarked on similar efforts, including working groups from the Association of Biomedical Resource Facilities and the Association of Molecular Pathology. The Centers for Disease Control and Prevention also has an ongoing project called the Genetic Testing Reference Material Coordination Program, or Get-RM.
Zehnbauer noted, however, several differences in the efforts undertaken by DQAP and the others, such as the pilot's sole focus on cancers which have acquired genetic changes.
Importantly, the organizers behind DQAP wanted to ensure that the pilot would involve multiple stakeholders including oncologists, regulators, industry, laboratory professional organizations, payors, and others, she said.
Participants in the original SPOT/Dx working group include individuals from Aetna, Laboratory Corporation of America, Quest Diagnostics, Abbott Molecular, and McKesson, among others. A patient advocacy group, Friends of Cancer Research, is also represented in DQAP. Meanwhile, liaisons from the FDA and the National Cancer Institute are participating in DQAP's steering committee as observers.
"Not all efforts that have been done by other groups with regards to reference material characterizations have included that kind of comprehensive approach," Zehnbauer said, adding that the diverse perspectives will better ensure the success of any reference samples resulting from the pilot.
In coordination with the College of American Pathologists, SPOT/Dx is preparing a request for proposals for submissions from vendors for the production and manufacturing of the cell lines. Zehnbauer declined to describe the desired specifications for the cell lines since they have not yet been made public.
DQAP will also evaluate how to select the labs to test the reference samples. The plan is to begin the hands-on work of DQAP during the first half of 2017, with the evaluation of the reference samples and technologies used to develop the assays to be completed by the end of the year, Zehnbauer said.
DQAP also expects to submit a journal article describing its work by year's end, she said.