CAPRI: Critical Assessment of PRediction of Interactions
CAPRI community wide experiment on the comparative evaluation of protein-protein docking for structure prediction
Hosted By EMBL/EBI-MSD Group
CAPRI: THE EXPERIMENT AND ITS RULES
The Critical Assessment of PRedicted Interactions (CAPRI) experiment aims to do test methods that model macromolecular interactions in blind predictions based on the three-dimensional structures of proteins. CAPRI is neither a competition nor a commercial enterprise, it is a community-wide experiment open to all research groups in a spirit of cooperation.
The prediction experiment
A CAPRI prediction round starts when an experimentalist offers an adequate target complex. Participants must register with the CAPRI Web site as predictors, servers and/or scorers. Registration remains open during the round. When the round starts, the predictors are given passworded access to atomic coordinates of the components, and they must submit models of the complex before a deadline set some weeks later. They are also invited to upload sets of up to 100 models that are merged before being communicated to the scorers. The scorers will re-rank the scoring set and make their submissions before a second deadline. Submissions must contain no more than 10 models, ranked 1 to 10, and follow the format described on the CAPRI Web site. i
A Web server participating in the prediction experiment should return its submission within 24 hours of the time its manager downloads the target; a longer period may be arranged with the CAPRI organizers. Outside that period, the server should be accessible to the public, including to the CAPRI predictors. Servers are expected to contribute to the scoring set.
CAPRI targets are protein-protein complexes, or possibly other complexes involving proteins, for which there is an 'unbound' experimental structure of the free components, and an unpublished structure of the complex. If only one component structure is available at the time of the prediction, the second component may be 'bound', that is, taken from the complex. Predictors may use all the information publicly available at the time the round starts, but predictions will be canceled if the target structure becomes public before the round is completed.
The nature of the targets, their experimental coordinates and the model coordinates submitted by the predictors, are covered by a confidentiality agreement that all CAPRI participants must sign on the Web site.
Evaluation of the models
Models submitted by registered participants in time and in the correct format, are evaluated in comparison with experimental coordinates made available by their authors to the CAPRI assessors. The origin of the submissions is masked from the assessors. The criteria for the evaluation are described in Figure 1 of Lensink et al. Proteins 2007 69:704.
Once the evaluation is completed, the results are made accessible to all registered participants through the Web site. Participants are free to disagree with any conclusions, but submission of predictions explicitly waives all potential claims against the CAPRI Management. The results may be not be used for advertisement or commercial purposes, but they may appear in scientific publications with proper reference to the CAPRI experiment (Janin et al. Proteins 2003 52:2-9).
The CAPRI Management Committee