CAPRI: Critical Assessment of PRediction of Interactions

Community wide experiment on the comparative evaluation of protein-protein docking for structure prediction

Hosted By EMBL/EBI-MSD Group

First CAPRI Evaluation Meeting


La Londe-des-Maures, France,

September 19-21, 2002


We thank the following organisations and companies for supporting the 1st CAPRI evaluation meeting.

The French Centre National de la Recherche Scientifique (CNRS)
The Belgian Fonds National de la Recherche Scientifique (FNRS)
The International Society for Computational Biology (ISCB)
Action Informatique Mathematiques Physique pour la Genomique(IMPG) (French Ministry of Education)
Organon International
Hewlett-Packard Company


Thursday Sept. 19

14:00 Registration

An Overview of CAPRI Rounds One and TwoChair: M. Sternberg (London)

15:00 Introduction by J. Janin (Gif-sur-Yvette, France)

15:15 The targets for Round One
Sonia Fieulaine, Sylvie Nessler, Marie-Christine Vaney, Felix Rey, Carole Barbet, Marcel Knossow & Joel Janin (CNRS, Gif-sur-Yvette, France)

16:00 The targets for Round Two
Christian Cambillau (AFMB-CNRS, Marseille, France)

16:45 Pause

17:00 Evaluation of the Blind Predictions Made in the CAPRI Experiment
Raul Mendez, Leonardo DeMaria, Raphael Laplae and Shoshana J. Wodak. (SCMB, Universite Libre de Bruxelles, Brussels, Belgium)

Friday Sept. 20

CAPRI Rounds One and Two: Procedures, Algorithms and Results

Each participating group will make a 20 min presentation + 5 mn discussion, of specific features in their prediction procedure and of their results in Rounds One and Two.

8:30 Session 1 Chair: L. Ten Eyck (La Jolla)

Protein Docking Using the 3D-Dock Package
Graham R. Smith (Cancer Research UK) & Michael J. E. Sternberg ( Imperial College, London, UK)

Guided docking - First step to locate potential binding sites.
Paul W Fitzjohn & Paul A Bates (Cancer Research UK, London, UK)

Evaluation of protein docking predictions using Hex3.1 in CAPRI rounds 1 and 2.
David W Ritchie & Diana C Mustard (University of Aberdeen, UK)

GAPDOCK: A genetic algorithm approach to protein docking in CAPRI round 1
Eleanor J. Gardiner, Peter Willett & Peter J. Artymiuk (Sheffield University, UK)

Prediction of the unknown: Inspiring experience with the CAPRI experiment.
Efrat Ben-Zeev, Alex Berchanski, Alex Heifetz, Boaz Shapira & Miriam Eisenstein (Weizmann Institute, Rehovot, Israel)

10:35 Pause

11:00 Session 2 Chair: I. Vakser (Stony Brook)

ZDOCK: An integrated approach to protein-protein systems
Rong Chen, Weiwei Tong, Julian Mintseris & Zhiping Weng (Boston University, USA)

A lesson from CAPRI: Our best not always wins - how weakly bound complex structures can fool (computational) thermodynamics.
Carlos J. Camacho (Boston University, USA)

Protein-protein docking predictions for the CAPRI experiment
Jeffrey J Gray, Stewart Moughon, Tanja Kortemme, Ora Furman, Kira Misura & David Baker (University of Washington, Seattle, USA)

Taking Geometry to its Edge : Fast Rigid and Hinge-bent Docking Algorithms.
Haim Wolfson, Dina Duhovny, Yuval Inbar, Vladimir Polak & Ruth Nussinov (Tel Aviv University, Israel, and NCI-Frederick, USA)

12:45 Lunch

Free afternoon

20:30-22:35 Session 3 Chair: S. Vajda (Boston)

Finding needles in haystacks: Ranking the results of DOT calculations.
Dennis Law, Julie C. Mitchell & Lynn Ten Eyck (UCSD, La Jolla, USA)

ICM Docking by Global Energy Optimization with Fully Flexible Side-Chains
Juan Fernandez-Recio, Max Totrov & Ruben Abagyan (Scripps Research Institute and Molsoft, La Jolla, USA)

Docking of shapes for CAPRI targets: flexibility needed.
Andrei Tovchigrechko & Ilya Vakser (SUNY at Stony Brook, USA)

Third Solvent Clusters Fitting: Self-Evaluation of CAPRI Round 1 and Round 2.
Katsuichiro Komatsu, Youji Kurihara, Mitsuo Iwadate, Mayuko Takeda-Shitaka & Hideaki Umeyama (Kitasato University, Tokyo, Japan)

A modified soft docking approach for predicting sructure and interactions of protein-protein complex.
Cun-Xin Wang, Chun-Hua Li, Xiao-Hui Ma and Wei-Zu Chen (Beijing Polytechnic University, Beijing, China)

Saturday Sept. 21

9:00 General discussion of CAPRI Rounds One and Two

Chair: J. Janin (Gif-sur-Yvette)

Lessons learned and future directions

Is the protocol for submitting predictions to CAPRI adequate?

Are the evaluation and presentation of CAPRI results adequate?

10:30 Pause

Challenges in Predicting and Modelling Interactions

Chair: S. Wodak (Brussels)

11:00 Using Biophysics to Understand Genome-scale Studies: There's More to Life Than Statistics!
Adrian Elcock (University of Iowa, USA)

11:45 Functional organization of the yeast proteome: systematic analysis of protein complexes
Anne-Claude Gavin (Cellzome AG, Heidelberg, Germany)

12:30 Lunch

14:00 Interrogating and predicting protein interactions with 3D structures
Rob Russel (EMBL-Heidelberg, Germany)

14:45 General discussion : The future of CAPRI, CAPRI meetings and the Benchmark experiment.

16:00 Meeting closes

Prospective Participants (incomplete list)

Each group that participates in CAPRI coordination, target submission, prediction and evaluation, is expected to send one person to the Evaluation Meeting.

Ruben Abagyan (Scripps Institute, La Jolla, USA)
Christian Cambillau (AFMB-CNRS, Marseille, France)
Carlos J. Camacho (Boston University, Boston, USA)
Florencio P. Cabaleiro (Universitad Autonoma, Madrid, Spain)
Ye Che (Washington University, Saint Louis, USA)
Miri Eisenstein (Weizmann Institute, Rehovot, Israel)
A. Elcock (University of Iowa, USA)
Paul Fitzjohn (ICRF, London, UK)
Elizabeth Gardiner (University of Sheffield, UK)
Anne-Claude Gavin (Cellzome AG.b Heidelberg Germany)
Jeffrey J. Gray (University of Washington, USA)
Kim Henrick (EBI-EMBL, Hinxton, UK)
Mitsuo Iwadate (Kitasato University, Japan)
Joël Janin (LEBS-CNRS, Gif-sur-Yvette, France)
Marcel Knossow (LEBS-CNRS, Gif-sur-Yvette, France)
John Moult (CARB-University of Maryland, Rockville MD, USA)
Sylvie Nessler (LEBS-CNRS, Gif-sur-Yvette, France)
Raquel Norel (Columbia University, New York, USA)
Ruth Nussinov (NCI, Frederick MD, USA)
Arthur Olson (Scripps Institute, La Jolla, USA)
Nuno Palma (University of Lisbonne, Portugal)
Anne Poupon (LEBS-CNRS, Gif-sur-Yvette, France)
Paulino G. Puertas (Universitad Autonoma, Madrid, Spain)
David Ritchie (University of Aberdeen, UK)
Dietmar Schomburg (University of Cologne, Germany)
Brian Shoichet (Northwestern University, Chicago, USA)
Graham Smith (ICRF, London, UK)
Michael J.E. Sternberg (Imperial College, London, UK)
Lynn Ten Eyck (San Diego Supercomputer Center, La Jolla, USA)
Sandor Vajda (Boston University, Boston, USA)
Ilya A. Vakser (SUNY, Stony Brook NY, USA)
Marie-Christine Vaney (LGV-CNRS, Gif-sur-Yvette, France)
Zhiping Weng (Boston University, Boston, USA)
Cun-Xin Wang (Beijing Polytechnic University, Beijing, China)
Haim Wolfson (Tel Aviv University, Israel)
Shoshana Wodak (Université Libre, Brussels, Belgium)
Raoul Mendez (Université Libre, Brussels, Belgium)

Additional participants are expected from amongst members of the scientific community with interest in predicting protein-protein interactions and its applications.

Critical Assessment of PRedicted Interactions(

First CAPRI Evaluation Meeting

Agelonde, La Londe-des-Maures (France)
September 19-21, 2002

Predicting protein-protein interaction in the post-genome era:

Computer-based prediction methods have long been a recognized complement to experiment in structural biology, and demand in this field has greatly increased in the post-genomic era. Genome sequences yield information on tens of thousands of proteins, mostly of unknown structure and function. Bioinformatics has gone a long way towards predicting three-dimensional structures of individual proteins, mostly through homology modelling and fold recognition methods. The coverage and accuracy of these methods increase fast as the catalog of known structures grows thanks to high-throughput experimental methods and "structural genomics" programs. On the other hand, function is mediated as much by interaction as by structure, and protein-protein interactions are a central element in post-genomic biology [1-8].

Tens of thousands of gene products are known or suspected to interact with many others, based on genetic (yeast two-hybrid), biochemical (co-expression, co-localisation, co-purification or co-precipitation) or bioinformatics (phylogenetic profiles, gene synteny) methods [9-12], forming millions of putative complexes. A very small fraction of these protein-protein complexes will be available, let alone crystallized, in the near future. Predicted modes of association can be useful guides for genetic and biochemical experiments, but the prediction methods and the algorithms must first be extensively tested and their validity assessed. Docking algorithms, first suggested by S. Wodak & J. Janin in 1978 [13-14], operate on the atomic coordinates of two individual proteins and generate a large number of candidate structures for complexes in the computer. These candidate structures are then ranked using several criteria: geometric and chemical complementarity, electrostatics, empirical or mean field potentials and so on. A number of algorithms and many different scoring functions have been developed since 1991, and the field has become extremely active in the last year. Algorithms and scoring functions are first tested against the crystal structures of complexes deposited in the Protein Data Bank, mostly protease-inhibitor and antigen-antibody complexes. In the best case, the top of list is reasonably close to the crystal structure [15-26]. However, no prediction procedure achieves that goal on all test complexes and, in any case, the best predicted structures remain largely inferior to the crystal structures when it comes to, for instance, guiding mutagenesis experiments.


The Critical Assessment of Techniques for Protein Structure Prediction (CASP - see experiment is a blind test of the capacity of prediction algorithms to produce three-dimensional models based on sequences. Judging by the progress from CASP1 in 1994 to CASP4 in 2000 [27-28], CASP succeeded not only in establishing the state of the art in structure prediction, but also in stimulating the entire field. It relies on the willingness of crystallographers and NMR spectroscopists to provide experimental structures as targets for the predictions. There were 43 in CASP4, with predictions coming from over 160 groups around the world.

The Critical Assessment of PRedicted Interactions (CAPRI) experiment aims to do the same for macromolecular interaction, now a central theme in functional genomics. CAPRI is a blind test of the ability of protein-protein docking algorithms to predict the mode of association of two proteins based on their three-dimensional structure. It was designed in June 2001 at the Conference on Modeling Protein Interactions in Genomes organized in Charleston, SC, by Ilya Vakser (Medical University of South Carolina) and Sandor Vajda (Boston University). A report of this Conference is available at Unlike CASP, which has a fixed time schedule, CAPRI is data-driven: it starts whenever an experimentalist offers an adequate target and ends six to eight weeks later with the submission of predicted structures. The targets are protein-protein complexes, or possibly protein-DNA complexes, for which there is an experimental structure of the free components to start with, and one of the complex at the time of evaluation. If only one component structure is available, the second component may be taken out from the complex. Taking both component structures from the complex biases docking too much towards the correct solution, but a prediction starting from randomly reoriented backbone coordinates can also be considered.

The CAPRI Management Group nincludes: Kim Henrick (Hinxton, UK), Joël Janin (Gif-sur-Yvette, France), John Moult (Rockville, MD), Lynn Ten Eyck (San Diego, CA), Michael Sternberg (London, UK), Sandor Vajda (Boston, MA) and Shoshana Wodak (Brussels, Belgium). J. Janin takes care of finding suitable targets for prediction, S. Wodak, of evaluating the predicted structures. Kim Henrick is in charge of the Web site where participants register, target coordinates are deposited and predictions are submitted.

Rounds One and Two of CAPRI

Round One of CAPRI began in July 2001 with three target protein-protein complexes and nineteen groups of predictors. The targets were: the complex between HPr kinase from Lactobacillus and its substrate protein HPr, offered by J. Janin and S. Nessler (LEBS-CNRS, Gif-sur-Yvette, France); two complexes between monoclonal antibodies and either the flu virus hemagglutinin (presented by C. Martin and M. Knossow, LEBS-CNRS, Gif-sur-Yvette, France) or the VP6 protein of a rotavirus hemagglutinin (presented by M.C. Vaney and F. Rey, LGV-CNRS, Gif-sur-Yvette, France). The predictors were from 10 American, 8 European and one Japanese groups.

At the end of September, 271 predictions were submitted on the Web site that has been opened for this purpose at the European Bioinformatics Institute (Hinxton, UK). The submissions are being evaluated by comparison with the X-ray coordinates of the complexes, kindly communicated to S. Wodak prior to publication by the crystallographers who determined their structure.

Meanwhile, Round Two was being prepared. A set of three new targets, all complexes of the same protein antigen (an a-amylase) with three different camelide antibodies, was kindly presented by C. Cambillau (AFMB-CNRS, Marseille). Atomic coordinates for the component proteins will be made available to registered participants on Jan. 11, 2002, with a deadline of March. 30 for submitting predictions. Evaluation should be completed by the end of June.

The Evaluation Meeting

By the summer of 2002, blind predictions will have been made on at least six different protein-protein complexes by some twenty different procedures, and the results will have been compared to the experimental structures. There is a strong demand from both the predictors and the management committee for holding an evaluation meeting at this stage. Its major purposes will be to:

  1. validate and, if necessary, adjust the format of the prediction experiment itself: which targets are best suited? how should the predicted structures be presented? what criteria should be used for their evaluation?

  2. assess the performances of docking algorithms, scoring functions and other elements of the procedures used for prediction

  3. evaluate progress in the field: are we getting closer to efficient and reliable methods for predicting protein-protein interaction?

The CAPRI Evaluation Meeting will be similar to the CASP evaluation meetings which have been held every other year in Asilomar (California) since 1994. These meetings have been extremely stimulating to all participants and brought entirely new point of views on protein structure prediction. We hope that, like the Asilomar meetings, the meeting in La Londe des Maures will foster new ideas and give a strong impetus to the field.


1. C. Kleanthous (2000), ed. Protein-protein recognition, Oxford Univ. Press

2. Janin J. & Wodak. S.J. (in press) The structural basis of macromolecular recognition. Adv. Prot. Chem. 61

3. Jones, S. and Thornton, J. M. (1996). "Principles of protein-protein interactions." Proc Natl Acad Sci U S A 93, 13-20.

4. Marcotte, E. M., Pellegrini, M., Thompson, M. J., Yeates, T. O. and Eisenberg, D. (1999). "A combined algorithm for genome-wide prediction of protein function." Nature 402, 83-6.

5. Xenarios, I., Rice, D. W., Salwinski, L., Baron, M. K., Marcotte, E. M. and Eisenberg, D. (2000). "DIP: the database of interacting proteins." Nucleic Acids Res 28, 289-91

6. Enright, A. J., Iliopoulos, I., Kyrpides, N. C. and Ouzounis, C. A. (1999). "Protein interaction maps for complete genomes based on gene fusion events." Nature 402, 86-90

7. Bader, G. D., Donaldson, I., Wolting, C., Ouellette, B. F., Pawson, T. and Hogue, C. W. (2001). "BIND--The Biomolecular Interaction Network Database." Nucleic Acids Res 29, 242-5.

8. Park, J., Lappe, M. and Teichmann, S. A. (2001). "Mapping protein family interactions: intramolecular and intermolecular protein family interaction repertoires in the PDB and yeast." J Mol Biol 307, 929-38.

9. Uetz, P., Giot, L., Cagney, G., Mansfield, T. A., Judson, R. S., Knight, J. R., Lockshon, D., Narayan, V., Srinivasan, M., Pochart, P. (2000). A comprehensive analysis of protein-protein interactions in Saccharomyces cerevisiae. Nature 403, 623-7.

10. Ito, T., Chiba, T., Ozawa, R., Yoshida, M., Hattori, M. and Sakaki, Y. (2001). "A comprehensive two-hybrid analysis to explore the yeast protein interactome." Proc Natl Acad Sci U S A 98, 4569-74.

11. Hazbun, T. R. and Fields, S. (2001). "Networking proteins in yeast." Proc Natl Acad Sci U S A 98, 4277-8.

12. Rain, J. C., Selig, L., De Reuse, H., Battaglia, V., Reverdy, C., Simon, S., Lenzen, G., Petel, F., Wojcik, J., Schachter, V. (2001). "The protein-protein interaction map of Helicobacter pylori." Nature 409, 211-5.

13. Wodak, S. J. and Janin, J. (1978). "Computer analysis of protein-protein interaction." J Mol Biol 124, 323-42

14. Janin, J. and Wodak, S. J. (1985). "Reaction pathway for the quaternary structure change in hemoglobin." Biopolymers 24, 509-26

15. Jiang, F. and Kim, S. H. (1991). ""Soft docking": matching of molecular surface cubes." J Mol Biol 219, 79-102

16. Cherfils, J., Duquerroy, S. and Janin, J. (1991). "Protein-protein recognition analyzed by docking simulation." Proteins 11, 271-80

17. Shoichet, B. K. and Kuntz, I. D. (1991). "Protein docking and complementarity." J Mol Biol 221, 327-46

18. Katchalski-Katzir, E., Shariv, I., Eisenstein, M., Friesem, A. A., Aflalo, C. and Vakser, I. A. (1992). "Molecular surface recognition: determination of geometric fit between proteins and their ligands by correlation techniques." Proc Natl Acad Sci U S A 89, 2195-9

19. Norel, R., Fischer, D., Wolfson, H. J. and Nussinov, R. (1994). "Molecular surface recognition by a computer vision-based technique." Protein Eng 7, 39-46

20. Totrov M, Abagyan R (1994) Detailed ab initio prediction of lysozyme-antibody complex with 1.6 A accuracy. Nat Struct Biol 1, 259-63

21. Vakser, I. A. (1995). "Protein docking for low-resolution structures." Protein Eng 8, 371-7

22. Meyer, M., Wilson, P. and Schomburg, D. (1996). "Hydrogen bonding and molecular surface shape complementarity as a basis for protein docking." J Mol Biol 264, 199-210

23. Gabb, H. A., Jackson, R. M. and Sternberg, M. J. (1997). "Modelling protein docking using shape complementarity, electrostatics and biochemical information." J Mol Biol 272, 106-20

24. Ackermann, F., Herrmann, G., Posch, S. and Sagerer, G. (1998). "Estimation and filtering of potential protein-protein docking positions." Bioinformatics 14, 196-205

25. Camacho, C. J., Weng, Z., Vajda, S. and DeLisi, C. (1999). "Free energy landscapes of encounter complexes in protein-protein association." Biophys J 76, 1166-78

26. Ritchie, D. W. and Kemp, G. J. (2000). "Protein docking using spherical polar Fourier correlations." Proteins 39, 178-94.

27. Moult J. (1999) Predicting protein three-dimensional structure. Curr Opin Biotechnol 10, 583-8

28. Venclovas C, Zemla A, Fidelis K, Moult J. (1999) Some measures of comparative performance in the three CASPs. Proteins Suppl 3:231-7