CAPRI: Critical Assessment of PRediction of Interactions

Community wide experiment on the comparative evaluation of protein-protein docking for structure prediction

Hosted By EMBL/EBI-MSD Group

Schedule of Round 26 extension

Monday Feb. 20 - Round extension opens

Download additional information - 20th Feb 2012

Sunday March 18 Deadline for submission.

Round 26

Round 26 ID mapping from group number to Accessor Number for Target 53
Round 26 ID mapping from group number to Accessor Number for Target 54
NOTE The Coordinates are not yet available for download

T53 result summary
T54 result summary

Results for targets 55 and 56

General Summary
T55 result summary for Round 1
T55 result summary for Round 2
T56 result summary for Round 1
T56 result summary for Round 2

Target 53

Results T53 - click here to see the results

Full results (regular assessment):

Clash threshold	62.31
average	18.21
std dev	22.05

	Predictor	Uploader	Scorer
Nr groups	42	15	13
High Accuracy (***)	1 ( 1)	0 ( 0)	0 ( 0 <- 0)
Medium (**)	11 ( 9)	9 ( 5)	12 ( 9 <- 4)
Acceptable (*)	31 (17)	92 (12)	24 ( 8 <- 4)
Incorrect	321 (39)	1090 (14)	85 (13 <- 12)
Clashes	20 ( 5)	209 ( 4)	9 ( 3 <- 2)
Low ID	0 ( 0)	0 ( 0)	0 ( 0 <- 0)
Total	384 (40)	1400 (14)	130 (13 <- 13)

Numbers in parentheses refer to the number of different source groups producing these models (ie z scorer groups recognize y out of x uploader groups)

Target 54

Results T54 - click here to see the results

Full results (regular assessment):

Clash threshold	64.14
average	17.43
std dev	23.36

	Predictor	Uploader	Scorer
Nr groups	41	16	13
High Accuracy (***)	0 ( 0)	0 ( 0)	0 ( 0 <- 0)
Medium (**)	0 ( 0)	1 ( 1)	0 ( 0 <- 0)
Acceptable (*)	6 ( 4)	18 ( 7)	0 ( 0 <- 0)
Incorrect	350 (39)	1196 (14)	119 (13 <- 14)
Clashes	21 ( 6)	185 ( 4)	11 ( 3 <- 2)
Low ID	0 ( 0)	0 ( 0)	0 ( 0 <- 0)
Total	377 (39)	1400 (14)	130 (13 <- 14)

Numbers in parentheses refer to the number of different source groups producing these models (ie z scorer groups recognize y out of x uploader groups)

Round 26 - Results for targets 55 and 56

All metrics are calculated across all positions, as well as for just the mutations at interface positions (T55: 33, 46, 47, 49, 50, 53, 54, 56, 57, 60, 61, 64, 65, 66, 69; T56: 12, 13, 14, 15, 17, 18, 20, 21, 24, 25, 27, 36, 39, 40, 42 ). The first round metrics were calculated across all provided prediction for non-"wild type" mutations. The second round metrics were calculated across all the provided predictions which were not in the released data set. The number of predictions considered for each group is specified by the Nmut columns.

The Kendall values are the all-against-all Kendall tau rank order correlation for the continuous value predictions against the experimental log2(enrichment ratio) values. To allow for metrics where predicted values may not be comparable across different positions, the Per Res Gamma metric is a derivative of Goodman and Kruskal's gamma, where the counted concordant and discordant pairs are formed solely between mutations at a single position. Frac correct is the fraction of correct class predictions, using experimental log2(enrichment ratio) cutoffs of +2.0 and -2.0 for determining the beneficial/neutral/deleterious class specification.

Download results for targets 55 and 56

References:

Optimization of affinity, specificity, and function of designed Influenza inhibitors using next generation sequencing
Whitehead TA, Chevalier A, Song Y, Dreyfus C, Fleishman SJ, De Mattos C, Myers CA, Kamisetty H, Blair P, Wilson IA, Baker D.
Nature Biotechnology vol. 30, June 2012
Download complete experimental data set
Next-generation protein engineering targets influenza New and Views
SJ Wodak
Nature Biotechnology vol. 30, June 2012

Target 55 Round 1

Group	Wtseq	Nmut	Nmut (Int.)	Per Res Gamma	Per Res Gamma (Int.)	Kendall	Kendall (Int.)	Frac correct	Frac correct (Int.)
Mitsuo Iwadate	exp	18	4	0.000	-1.000	0.105	0.333	0.444	0.50
Julie Mitchell	PDB	1007	285	0.072	0.084	0.061	0.159	0.568	0.71
Chaok Seok	exp	1007	285	0.043	0.097	0.080	0.142	0.461	0.55
Martin Zacharias	PDB	1007	285	0.016	0.102	0.098	0.216	0.473	0.37
Paul Bates	PDB	1007	285	-0.004	0.004	0.094	0.157	0.504	0.57
Haruki Nakamura	exp	1007	285	0.013	0.010	0.141	0.223	0.484	0.43
Sergei Grudinin	PDB	1007	285	0.012	0.107	0.077	0.089	0.461	0.47
SurFit,Daron Standley	exp	1007	285	0.030	0.007	0.066	0.084	0.502	0.49
Xiaoqin Zou	exp	1007	285	0.001	0.130	0.163	0.247	0.341	0.49
Zhiping Weng	PDB	1007	285	0.119	0.084	0.224	0.217	0.557	0.55
Jeffrey J. Gray	PDB	1007	285	0.031	0.028	0.166	0.188	0.539	0.52
Dmitry Korkin	exp	1007	285	0.009	-0.063	0.039	0.012	0.439	0.28
Alexandre M.J.J. Bonvin	exp	988	266	0.013	0.030	0.088	0.200	0.498	0.47
Juan Fernandez-Recio	PDB	1007	285	0.180	0.077	0.295	0.278	0.628	0.66
Yves Dehouck	PDB	1007	285	0.151	0.130	0.286	0.252	0.582	0.59
Huan-Xiang Zhou	exp	1007	285	0.074	0.033	0.165	0.236	0.504	0.52
Mayuko Takeda-Shitaka	PDB	1007	285	0.069	0.172	0.123	0.152	0.478	0.55
Daisuke Kihara	PDB	1007	285	-0.022	-0.002	0.054	0.179	0.259	0.51
Yi Xiao	exp	1007	285	0.128	0.103	0.131	0.111	0.500	0.49
Sandor Vajda	PDB	164	163	0.210	0.210	0.134	0.133	0.677	0.67
Meng Cui	PDB	1007	285	0.013	0.040	0.068	0.166	0.417	0.49
Samuel Flores	PDB	601	260	0.112	0.151	0.232	0.258	0.448	0.58

Target 55 Round 2

Group	Wtseq	Nmut	Nmut (Int.)	Per Res Gamma	Per Res Gamma (Int.)	Kendall	Kendall (Int.)	Frac correct	Frac correct (Int.)
Julie Mitchell	exp	530	150	0.097	0.150	0.263	0.298	0.532	0.65
Chaok Seok	exp	530	150	0.083	0.183	0.088	0.103	0.440	0.51
Paul Bates	PDB	530	150	0.303	0.324	0.514	0.400	0.547	0.69
Haruki Nakamura	exp	530	150	0.039	0.115	0.097	0.217	0.443	0.39
Sergei Grudinin	PDB	530	150	-0.021	0.088	0.041	0.048	0.455	0.46
Zhiping Weng	exp	530	150	0.130	0.110	0.335	0.249	0.643	0.66
Dmitry Korkin	PDB	530	150	0.002	-0.018	0.038	0.018	0.404	0.31
Alexandre M.J.J. Bonvin	exp	530	150	0.128	0.168	0.157	0.119	0.502	0.53
Juan Fernandez-Recio	exp	530	150	0.261	0.239	0.495	0.403	0.696	0.79
Mayuko Takeda-Shitaka	NA	490	134	0.211	0.225	0.381	0.310	0.600	0.75
Daisuke Kihara	PDB	530	150	-0.045	-0.052	0.032	0.158	0.255	0.48
Yi Xiao	exp	530	150	0.090	0.089	0.199	0.253	0.532	0.49
Zhiping Weng	855	285	0.053	0.076	0.227	0.254	0.468	0.54

Target 56 Round 1

Group	Wtseq	Nmut	Nmut (Int.)	Per Res Gamma	Per Res Gamma (Int.)	Kendall	Kendall (Int.)	Frac correct	Frac correct (Int.)
Mitsuo Iwadate	exp	332	114	-0.030	-0.052	0.019	-0.015	0.654	0.72
Julie Mitchell	PDB	855	285	0.053	0.022	0.056	0.092	0.572	0.61
Chaok Seok	exp	855	285	0.110	0.179	0.035	0.133	0.427	0.52
Martin Zacharias	PDB	855	285	0.119	0.176	0.029	0.132	0.382	0.44
Paul Bates	PDB	853	284	0.028	0.057	0.085	0.200	0.402	0.49
Haruki Nakamura	exp	855	285	0.091	0.102	0.079	0.092	0.357	0.35
Sergei Grudinin	PDB	855	285	0.052	0.076	0.041	0.174	0.435	0.55
SurFit,Daron Standley/td>	exp	855	285	0.041	0.060	0.129	0.248	0.432	0.49
Xiaoqin Zou	exp	855	285	0.077	0.064	0.044	0.054	0.586	0.52
Zhiping Weng	PDB	855	285	0.085	0.125	0.214	0.310	0.566	0.57
Jeffrey J. Gray	PDB	822	285	0.097	0.119	0.139	0.223	0.482	0.47
Dmitry Korkin	exp	855	285	0.035	0.018	0.077	0.070	0.357	0.34
Alexandre M.J.J. Bonvin	exp	855	285	-0.015	0.030	0.016	0.124	0.337	0.35
Juan Fernandez-Recio	PDB	855	285	0.099	0.147	0.172	0.289	0.532	0.58
Huan-Xiang Zhou	exp	855	285	0.075	0.048	0.147	0.161	0.421	0.49
Mayuko Takeda-Shitaka	PDB	855	285	0.023	0.021	0.054	0.126	0.488	0.53
Daisuke Kihara	PDB	855	285	-0.001	0.070	0.015	0.098	0.290	0.41
Yi Xiao	exp	836	285	-0.168	-0.494	-0.029	-0.002	0.287	0.27
Sandor Vajda	PDB	200	160	0.207	0.144	0.173	0.141	0.665	0.63
Meng Cui	PDB	854	284	-0.016	-0.047	0.047	0.140	0.370	0.43

Target 56 Round 2

Group	Wtseq	Nmut	Nmut (Int.)	Per Res Gamma	Per Res Gamma (Int.)	Kendall	Kendall (Int.)	Frac correct	Frac correct (Int.)
Julie Mitchell	exp	450	150	-0.028	0.170	0.191	0.302	0.638	0.71
Chaok Seok	exp	450	150	0.075	0.205	0.061	0.153	0.451	0.55
Paul Bates	PDB	447	148	0.148	0.132	0.406	0.458	0.633	0.69
Sergei Grudinin	PDB	450	150	0.011	-0.007	0.029	0.142	0.451	0.53
Xiaoqin Zou	exp	450	150	0.028	-0.007	0.116	0.049	0.592	0.55
Zhiping Weng	exp	450	150	0.044	0.083	0.209	0.321	0.580	0.65
Dmitry Korkin	PDB	450	150	0.041	0.079	0.045	-0.124	0.364	0.29
Alexandre M.J.J. Bonvin	exp	450	150	0.087	0.262	0.130	0.242	0.509	0.59
Juan Fernandez-Recio	exp	450	150	0.196	0.247	0.418	0.447	0.747	0.78
Yves Dehouck	exp	450	150	0.131	0.163	0.393	0.438	0.689	0.71
Mayuko Takeda-Shitaka	NA	325	91	0.188	0.220	0.278	0.239	0.702	0.76

ROUND 26 Timeline
2nd Nov. 2011 - Registration opens for Round 26
7th Nov. 2011 - Round 26 opens - Prediction of targets 53,54 and Scoring for targets 55,56
20th Nov. 2011 - Deadline for submitting models for target 53, 54
24th Nov 2011 - Scoring starts for targets 53, 54
27th Nov. 2011 - Deadline for scoring submission for targets 53, 54
22nd Dec. 2011 - Deadline for scoring submission for targets 55, 56

Target T53 a protein-protein complex (homology/unbound)
Target T54 a protein-protein complex (homology/unbound)
Target T55 (scoring only) is a large set (>500) of point mutations made in a protein engineered to bind another; given the X-ray structure of one complex, the challenge is to predict the effect of each mutation on the interaction.
Target T56 (scoring only) is a large set (>500) of point mutations made in a protein engineered to bind another; given the X-ray structure of one complex, the challenge is to predict the effect of each mutation on the interaction.

Documentation and Targets Round 26

Participants

Management Group

Prediction formats

Download Opens 7th Nov 2011

Upload Opens 7th Nov 2011

Scoring download open 24th Nov 2011

Entries

Current submissions

SCORERS

Scorers can download the server set of submissions from
http://www.scmbb.ulb.ac.be/capri/ using the same login user/pass as distributed for Capri round26. Full details and deadlines are given on this site.

Email Problems or Queries to Sameer Velankar