The goal of our assessment plan is to develop an unbiased method of evaluating the map quality, not based in the reported resolution (FSC graph), but in it's ability to correctly guide an atomic model prediction software (ROSETTA).

The planned method is divided in three stages:

1st Stage
The method starts by identifying a suitable section of the map where the protein structure is known. The selected section of the map is then extracted from the original map. Utilizing the protein sequence and the extracted map section the Rosetta protocol denovo_density can be used to obtain an initial atomic model.

2nd Stage
The initial atomic model, the protein sequence and the extracted map section can be used to produced several thousand atomic model using the Rosetta function, rosetta_scripts.

3rd Stage
Lastly, all generated atomic model are compared to the native atomic model. The resulting root mean square (rms) of the comparison is plotted in a scattered plot to visualized the rms spread. Good quality maps will consistently guide the modelling to an atomic model reconstruction similar to the native structure, while low quality maps will poorly guide the modelling. In other words, good quality maps will show a small rms values and small spread while low quality maps will show higher rms values with a larger spread.

We only evaluated TRPV1 maps. We re-calculating FSC from half maps using the same spherical mask, to evaluate the nominal resolution of each submitted map. We then use EMRinger to assess side chain quality of each maps, and rank the scores. Last we visually compare all maps to see how specific features are resolved, in particular the density contributed by specifically bound lipid molecules. We finally summarize evaluation of all maps in this entry and compare them. Here, the first two comparison will be quantitative, but the last one is more qualitative.