Benchmarking data analysis software

Researchers can now assess mass-spectrometry-based, label-free quantitative proteomics in a comprehensive, reproducible manner, thanks to new software created by an international collaboration. Published in Nature Biotechnology, 'LFQbench' is a new tool for benchmarking different types of instruments, acquisition methods and data-analysis software.

Only as good as the tools you use

“Consistent, accurate quantification of proteins by mass-spectometry-based proteomics depends on how well the instruments, acquisition methods and data analysis software perform”, says Yasset Perez-Riverol, Senior Software Engineer in the Proteomics team at EMBL-EBI, which develops the PRIDE data resource.

In this study, University Medical Center Mainz and EMBL-EBI researchers developed a software framework for the automated evaluation of quality metrics based on the existing knowledge of a hybrid proteome sample set. 

Software developers then demonstrated how they used LFQbench to compare and improve their products in an easy, reliable way. 

The researchers critically evaluated five of the most widely used software methods for processing SWATH data: OpenSWATH, SWATH 2.0, Skyline, Spectronaut and DIA-Umpire.

Enabling optimisation

In an iterative evaluation process, LFQbench enabled the developers to detect issues in their analysis algorithms and to improve them. After optimisation, all tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics.

Making proteomics software better

“The new methodology is useful for improving quantitative proteomics software tools, including performance control, and enables benchmarking of algorithms for peak detection interference removal and improved strategies for peptide to protein inference” says Perez-Riverol.

LFQbench is also invaluable for assessing and optimising quantitative proteomics workflows, and offers an important resource for the global proteomics community.

Read more about this work

Source article: Navarro P, et al. (2016) A multicenter study benchmarks software tools for label-free proteome quantification. Nature Biotechnology, published online 3 October. doi:10.1038/nbt.3685 

GitHub Project: https://github.com/IFIproteomics/LFQbench 

Genome Web story about this study

Spectronaut story on the Biognosys website