This article and the headline has been updated to clarify that ClinGen is spearheading efforts to resolve variant classification discrepancies in ClinVar.
NEW YORK (GenomeWeb) – Clinical labs submitting genetic variant data to ClinVar, a rapidly growing public database of genotype-phenotype relationships, are using detailed case-level data and focusing on outlier interpretations to resolve classification conflicts in the repository.
At a US Food and Drug Administration public workshop this week on variant classification and interpretation efforts within precision oncology, Heidi Rehm, director of the Laboratory for Molecular Medicine (LMM) at Partners Healthcare Personalized Medicine, provided an update on ClinVar, a database the NIH launched in 2013.
As of Jan. 29, ClinVar contained more than 582,000 records from nearly 900 submitters from around 60 countries, including information on more than 376,000 unique genetic variants. Toward the end of 2016, the database had contained just around 250,000 total submissions.
ClinVar has grown rapidly despite certain genomics industry players refusing to partake in the data-sharing effort, criticizing it for containing discrepant variant classifications between submitting labs, and maligning it as a dumping ground of incorrect or outdated data.
Rehm has repeatedly emphasized that the purpose of ClinVar is to highlight variant classification discrepancies so that industry players will be motivated to collaborate and resolve those differences. "Keep in mind that no single data source, public or private, is ever comprehensive," she said at the FDA workshop. "Each one has different data than the next."
Think of information in ClinVar as "data sources, not correct claims," Rehm said. "You should use the actual evidence and take into account possible concerns about quality … and the claims should always be reassessed with the total body of evidence."
Considering all the data submitted to ClinVar, 17 percent of variants interpreted by two or more submitters have conflicting classifications. But when considering which of these discrepancies would impact patient management — for example, if one lab calls a variant likely pathogenic or pathogenic, and another lab deems it a variant of unknown significance (VUS), likely benign, or benign — that number drops to 3.6 percent.
A lot of these conflicting interpretations comes from data culled from research and the published literature, which are reassessed and updated at a slower clip than the data submitted by clinical labs. Around 80 percent of data in ClinVar is from clinical labs, and most high-quality labs submitting to the repository are reassessing their submissions every few months based on the most up-to-date information from the published literature, ongoing research, and internal case-level data.
Rehm noted that it has been difficult to present an accurate snapshot the classification conflicts within ClinVar because it contains older classifications based on outdated research or published data. But if one focuses on the medically significant variant classification discordances within the data submitted by clinical labs, only 1.7 percent of the submissions are conflicting.
"Quality of the evidence does vary based on the submitter," Rehm acknowledged, noting that supporting evidence is not available for about 19 percent of entries. The other 81 percent of entries in ClinVar largely contain summarized evidence, links to external databases, and published literature.
There is currently limited case-level data supporting the variant classifications within ClinVar. The experts involved in ClinGen — an NIH-funded resource that aims to define the clinical relevance of genes and variants in precision medicine and research, and a sister effort to ClinVar — have been trying to come up with strategies for bringing in more case- level data into the repository.
One option is the online registry, called Genome Connect, where patients can upload their de-identified genetic test reports and health details, and consent to share that information within ClinVar. As of last year, around 1,000 patients had completed the consent process within Genome Connect, 80 percent had taken a health survey, and 24 percent had uploaded a genetic test report.
Healthcare provider groups, such as Geisinger Health and the Center for Inherited Cardiovascular Disease at Stanford University, are starting to submit de-identified, case-level phenotypic data from patients tested by commercial labs. According to Rehm, this information has been useful in resolving classification discrepancies.
She cited the example of a pregnant patient who had received prenatal genetic testing and was very worried about genetic disorders in the fetus after receiving a result indicating a VUS in the BRAF gene. This patient contacted Rehm's lab, because LMM was one of two labs that had submitted a VUS interpretation for this variant to ClinVar.
The patient's outreach prompted the experts at LMM to work with the other lab, GeneDx (owned by Opko Health), and the woman's family to gather more information on this variant and reclassify it as likely benign. The patient's family also signed up for Genome Connect and submitted the relevant phenotypic data. "All this information went into the ClinVar entry for the variant," Rehm noted. The couple gave birth to a healthy baby.
Last week, Rehm's lab assessed another variant as a VUS that three labs had also submitted as a VUS into ClinVar. But by working together to evaluate case-level data on this variant, these labs were able to reclassify the VUS to 'likely pathogenic.'
Collaborative efforts of this type between clinical labs are becoming more common and have demonstrated that while conflicting variant classifications happen, labs can resolve them in most cases by sharing data. For example, Ambry Genetics (owned by Konica Minolta), GeneDx, LMM, and the University of Chicago evaluated more than 6,000 variants that at least two labs had submitted to ClinVar, and found their classifications differed for 724 of them. The four labs worked for several months to tackle 242 of these variants and reached a consensus on 87 percent of those, but weren't able to agree on the remaining 13 percent because the labs applied variant classification guidelines differently.
In another pilot project, nine laboratories comparing their interpretations for 99 variants found that their determinations initially agreed 34 percent of the time. After sharing data and talking through the evidence, they aligned their classifications for 71 percent of these variants.
ClinGen has now scaled this type of collaborative project and is advancing "an outlier approach" where for the variants where the majority agree, only the outlier labs are asked to review their determination. The conflicting interpretations that remain unresolved after reassessment by outlier labs are then addressed through further collaboration. "We've now scaled this project to encompass every clinical lab that's submitting to ClinVar in the germline space," Rehm said. "That process allows us to resolve the majority [of conflicts] without all the labs having to reassess."
Data-sharing collaborations around somatic variants are also occurring within cancer-specific knowledgebases. The Variant Interpretation Cancer Consortium (VICC), which formed in 2016, for example, is working to organize variant data across existing knowledgebases.
As of last October, the consortium had integrated data on 17,000 variants from six databases, including OncoKB, MolecularMatch, the Clinical Knowledgebase, Precision Medicine Knowledgebase, Clinical Interpretations of Variants in Cancer, and the Cancer Genome Interpreter. VICC has assembled a prototype interface for querying variant interpretations across those knowledgebases and is preparing to submit data on somatic variants to ClinVar.
In an effort to provide greater transparency to ClinVar users about the information in the repository, ClinGen last year published a list of labs that meet certain data-sharing requirements, such as outlining the criteria they use to make assertions about submitted variants and updating reclassified variants at least once a year. Labs that meet minimum requirements can also get "badges" for additional activities, such as working with other labs to resolve variant classification conflicts. Six months ago, the list included 10 labs, and has since grown to 16 labs.
"This list will allow providers, insurers, and others to determine who they may order tests from, or to reimburse [labs] based on whether they are meeting certain basic requirements for data sharing," Rehm said, adding that in the future, labs on the list will also be required to provide the evidence underlying variant interpretations in ClinVar.
Currently within ClinVar, variant records are marked with stars to indicate they meet certain criteria. For example, submissions from individual labs without assertion criteria have zero stars, submissions with assertion criteria get one star, and submissions without any discordance among multiple submissions get two stars. When a submission has been reviewed by a ClinGen-approved expert panel, it gets three stars, which overrides interpretations by individual labs. And when a submission has been reviewed by a practice guideline body, it gets four stars.
The star system is useful for prioritizing which submitters to follow up with when a user has a different variant interpretation than what's in ClinVar. "We don't generally follow up with a zero-star submitter but we follow up with a single star and above," Rehm said.
Users should always consider the underlying evidence on a variant when referencing ClinVar, Rehm cautioned, and not simply assume that a four-star entry in ClinVar is always up to date, or that a two-star variant with agreement between labs can't be inaccurate. "Expert panel interpretations can get out of date as evidence continues to amass," she noted. "Just because labs all agree with each other doesn't mean it's correct."
The star system has been a source of confusion for users, and ClinVar is considering doing away with it, Rehm acknowledged, since some people have mistakenly thought that the more stars a variant has, the more pathogenic it is. In the future, ClinVar may simply annotate variant records using descriptive terms, such as "practice guideline."