Skip to main content

AMP-Led Workgroup Publishes Guidelines for Validating NGS Bioinformatics Pipelines

NEW YORK (GenomeWeb) – A workgroup led by the Association for Molecular Pathology with representation from the College of American Pathologists and American Medical Informatics Association has published recommendations and guidelines for validating next-generation sequencing bioinformatics pipelines to improve the accuracy of NGS-based testing.

The group, which also included representatives from the American Medical Informatics Association, developed a set of 17 recommendations that laboratories should consider when validating bioinformatics pipelines, and published them today in the Journal of Molecular Diagnostics.

"While the recent widespread adoption of next-generation sequencing methodologies have transformed our ability to detect somatic and germline variants, the constant technology evolution and absence of professional guidelines have contributed to variability in clinical laboratory practice," Somak Roy, an assistant professor of pathology at the University of Pittsburgh Medical Center and the chair of the working group that developed the recommendations, said in a statement.

Such variability can lead to inaccurate results that may have "negative consequences for patient care," Roy and co-authors wrote in the JMD study.

The guidelines are the third issued by AMP and CAP that address NGS workflows. Earlier this year, the organizations jointly published guidelines for the validation of NGS-based oncology panels and last year published guidelines on interpreting and reporting cancer-related sequence variants.

In today's publication, the authors noted that the recommendations apply only to the bioinformatics portion of a test and assume that the sequencing potion has already been validated. In addition, the recommendations deal with SNVs, as well as indels and multi-nucleotide substitution that are 21 base pairs in length or less. The guidelines cover both germline and somatic variants.

The guidelines address NGS bioinformatics pipeline overview, design, development, validation, implementation, and quality control metrics, emphasizing the importance of trained molecular medical professionals.

In generating the guidelines, the group first reviewed the peer-reviewed literature on bioinformatics pipeline validation, identifying "a clear absence of uniformity in the methods."

One main problem the authors noted is that often there is a disconnect between the clinical laboratories that ultimately run an NGS test and the researchers or commercial vendors that originally developed a given algorithm or software tool.

Different algorithms and software tools are sometimes combined for a given clinical test to put together a complete pipeline — for instance, one tool may be used for alignment and another for variant calling and annotation. In addition, some commercial vendors offer various informatics services and so clinical labs will sometimes outsource portions of the pipeline.

The workgroup had several recommendations to address these issues, including recommending that labs be familiar and validate each step of the bioinformatics process, including even portions that it outsources. It also said that the pipelines should be validated for the types of variants and samples that the lab would be testing. In cases where labs rely on software for variant annotation, that process must be validated and labs should have an alert system for cases that require manual review.

Another set of recommendations sought to emphasize the importance of and need for appropriate trained professionals to oversee and be involved in the validation process.

In addition, the workgroup said it would be important for bioinformatics pipelines to ensure patient privacy was protected.

Quality control is another issue, and the workgroup outlined several recommendations to address this, including ensuring that the bioinformatics pipelines are appropriately accredited, that they include ways to track and maintain sample identity, and that quality control and quality assurance metrics should be evaluated against a test's desired sensitivity, specificity, and positive predictive values. If sequence reads are altered or filtered, those steps must be documented and validated to ensure correct interpretation.

And finally, the group said that if any component of the bioinformatics pipeline is changed or updated, labs should perform a supplemental validation.

NGS testing is "rapidly becoming a method of choice for somatic and germline variant detection by clinical laboratories," the authors wrote. However, the methods can be challenging and require "thorough analytical validation to ensure the high quality of sequencing results."