You are here

2015/2016 Model Challenge


  • Establish a benchmark set of 3DEM maps in the 3.0-4.5 Å resolution range, where significant growth in the number of maps is anticipated over the next few years and where a number of technical challenges exist to map interpretation and fitting

  • Encourage developers of modelling software packages and biological end users to analyze these maps and present modelling results with the best practice

  • Evolve criteria for evaluation and validation of 3DEM map-derived models

  • Compare and contrast the various modelling and analysis approaches in a positive spirit

Model Challenge Registrants: 41
Current Count, 2019 Model Metrics Challenge Submissions: 63

Model Committee

Paul Adams (Chair), Axel Brunger, Randy Read, Torsten Schwede, Maya Topf, Gerard Kleywegt, Cathy Lawson, Wah Chiu

Model Challenge How to Participate

All members of the Scientific Community--at all levels of experience--are invited to participate as Challengers, and/or as Assessors.

Challengers will create and submit their own atomic coordinate models of one or more challenge targets using the supplied target maps.

There are four modelling categories:

  1. Optimize the current cryoEM model for the target, improving fit to the cryo-EM map density and/or model stereochemistry
  2. Fit known related cryoEM, crystallographic, or comparative models to the cryo-EM map, optionally followed by model optimization
  3. Ab initio model building, without reference to any existing cryo-EM or crystallographic models related to the cryo-EM map
  4. Any other method that seeks to interpret, or create a model based on, the cryo-EM map

For each submission, challengers will provide their final model, target map (if modified from original), and basic information about their modelling process. Researchers are expected to disclose all of the prior information used in their method. For full challenge rules and additional information see the model challenge guide.

Assessors will contribute to the challenge assessment phase.  Following a short initial review period by the model committee, challenge data and files will be made publicly available (entry authorship suppressed) for anyone to assess. The intention is to enable comparisons of the various modelling methodologies available and their options in a positive spirit.  Assessors will be able to share their results via a planned workshop (~Fall 2016) as well as manuscript submissions to a Journal special issue.

****All participants--challengers and assessors--are required to register****

Model Challenge Timeline

February-June Model Committee meets monthly to discuss possible scope of challenge
June 20/21 Face-to-face meeting to define challenge targets, goals, and parameters
July-September Finalization of challenge formulation. Requests to map contributors to provide any missing data (e.g., half-maps, unfiltered, unsharpened, unmasked maps)
October 14 Pre-Challenge Announcement, Challenger and Assessor Registration Opens
January-June 17 Challenge Site Open for Model Entry Submissions
June 17 21:00 UTC
Model entry submission window closes


Challenge Data review by the Model Committee
Challenge Data review opened to volunteer assessors (blind analysis--entry authorship suppressed)
May 29 Face-to-Face Meeting to discuss preliminary outcomes of assessments
mid July Assessors provide written reports on their results
mid July Reports on Models sent to Challengers; Full workflow info provided to all challengers and assessors
Aug 15 Feedback from Challengers on their reports
Sept 1 Deadline for Assessors to update their reports
Oct 5-8 Challenges Meeting (Maps and Models)
Post-workshop Challenge Writeups (multiple articles) for a Journal Special Issue

Model Challenge Targets

The eight targets chosen by the model challenge committee are shown below.  All are based on recently reported 3DEM structures.  The target order is according to unique molecular weight, from 19 kDa (TMV) at left to 2500 kDa (ribosome) at right. For challenge rules and additional information about each target see the model challenge guide.  

April 28: Corrected symmetry center info and BIOMT matrix files for the proteasome and beta-galactosidase targets, see the model challenge guide.

May 3: Corrected BIOMT matrix file containing Icosahedral transformations for Brome Mosaic Virus, see the model challenge guide.

Target Map Download: You can use this rsync script to download target maps from one of three wwPDB ftp sites. Alternately, you can download individual maps from EMDB atlas pages (click on EMDB entry link in the table below, select "download" tab).

  1. Tobacco Mosaic Virus 2. T20S Proteasome 3. GroEL 4. TRPV1 Channel 5. Brome Mosaic Virus 6. β-Galactosidase 7. γ-Secretase 8. 70S Ribosome


Map "A" EMDB entry
Primary Citation
Reported Resolution (Å)
Fromm et al
Li et al
Liao et al
Wang et al
Bartesaghi et al
Lu et al
Fischer et al
Reference Model(s) Map "A"

4udv (EM)

1ei7 (Xray)

1yar (Xray)

3j9i (EM)

3cau (EM)

1ss8 (Xray)

1svt (Xray)

3j5p (EM)

3j7l (EM)

1js9 (Xray)

3j7h (EM)

1jz7 (Xray)

4upc/superceded by 5a63 (EM) 5afi (EM)
Map "B" EMDB entry
Primary Citation
Reported Resolution (Å)
Campbell et al
Bartesaghi et al

Bai et al

Li et al
Reference Model(s) Map "B"  

1yar (Xray)


5a1a (EM)

1jz7 (Xray)

 5a63 (EM)

3ja1 (EM)

Imposed Map Symmetry


Dihedral (D7)

Dihedral (D7)

 Cyclic (C4) Icosahedral (I)  Dihedral (D2) None (C1) None (C1)
 Sample MW (MDa) --  0.7 0.8  0.3  4.6  0.47  0.17 2.5
 Unique MW (kDa) 19  50 56  80 80  120  170 2500
Map Contributors (Thank You!)

Simon Fromm, Carsten Sachse

Jean-Paul Armache, Yifan Cheng

Melody Campbell, Bridget Carragher

Soung-Hun Roh, Corey Hryc, Wah Chiu Jean-Paul Armache, Maofu Liao, Yifan Cheng Zhao Wang, Wah Chiu Alberto Bartesaghi, Sriram Subramaniam Xiaochen Bai, Sjors Scheres

Niels Fischer, Holger Stark

Wen Li, Zheng Liu, Joachim Frank

EMDataResource Validation Challenges are supported by NIH National Institute of General Medical Sciences

Please send your challenge questions, comments and feedback to

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer