You are here

Home

EMD-20026 half vs. full maps

Summary: below is an email thread discussing the relative quality of half-maps for EMD-20026 (1.8 Å apoferritin) vs. the full maps. By some of the metrics investigated by Tom and Greg (as described below), one of the half-maps appears to be of equivalent or better quality to the sharpened full map.  Tentative conclusion (Tom): " Maybe...we should be concluding that the full and half-maps at 1.8 A are not all that different (except for sharpening). And maybe this is saying that the errors in these maps are now coming from things that are not random, so that averaging in more data is no longer making a difference?"
 
From: Tom Terwilliger  Subject: EMD-20026 Date: April 24, 2019 at 3:59:58 PM EDT
Hi Cathy, I am checking out the maps for the challenge and it seems that the half-maps for 20026 are better than the full maps. These are all CC values to a docked model (same model for all maps)  (Of course I could have made a mistake in here somewhere...but the other ones seem to make sense):
  • emd_20026_unsharpened.map      CC_mask  : 0.6337
  • emd_20026.map                              CC_mask  : 0.7514
  • emd_20026_half_map_2.map         CC_mask  : 0.7795
  • emd_20026_half_map_1.map:         CC_mask  : 0.7797
Does this make sense? All the best, Tom T
 ====================
From: "Pintilie, Greg" Subject: Re: EMD-20026 Date: April 24, 2019 at 6:24:11 PM EDT
The half maps are not better than the full map, and CC is not a good indicator of map quality - very often you can get higher scores at lower resolutions. The problem is not the score per-se but rather different scaling of densities across maps - if the densities in the maps are scaled the same, i.e. have the same rough mean and standard deviation, or use CC about the mean, then that would be a better indicator. But  Z-scores are even better indicators :) See attached paper …. in particular Figure 5B which shows how CC is not a good indicator at different resolutions.
By Z-scores (and by eye - I took a look in Chimera to make sure), the full map has better resolvability.
  • emd_20026_unsharpened.map      CC_mask  : 0.6337  Avg side chain Z-score: 1.67
  • emd_20026.map                              CC_mask  : 0.7514 Avg side chain Z-score: 3.07
  • emd_20026_half_map_2.map          CC_mask  : 0.7795 Avg side chain Z-score: 2.28
  • emd_20026_half_map_1.map:         CC_mask  : 0.7797 Avg side chain Z-score: 2.28

================

On Apr 25, 2019, at 9:16 PM, Tom Terwilliger  wrote:
Hi Greg,  I had another look at two maps in question: emd_20026_half_map_2.map   and  emd_20026.map.  If I run phenix.auto_sharpen on each of these, including a model in the auto-sharpen process (I docked 3ajo for this purpose), I get a very nice map in each case, but to me the model-sharpened map from half-map 2 is clearly better than the model-sharpened map from the full map (looking at the aromatic residues and definition of branched side chains which are much clearer in the model-sharpened half-map for example).  I did this blind twice and each time I picked the model-sharpened half map over the model-sharpened full map.
So...I am not quite sure that the full map is really better than the half-map in this case. I'd be interested in your opinion of a visual analysis of model-sharpened versions of these maps.

==========

On Thu, Apr 25, 2019 at 9:32 PM Pintilie, Greg wrote:
Hi Tom, I tried the same thing, but could not replicate exactly... the sharpened full map looks much better to me than the sharpened half map. I used phenix.auto_sharpen with fitted 3ajo.pdb, resolution=1.75. The sharpened full map actually looked pretty much identical to the full map we deposited, at least to my eye. The sharpened half map looked worse than the initial half map, so maybe I did something different than you. Could you perhaps share the maps you are using, fitted model, and Phenix commands?
 
About the previous message, I think that comparing CCs of a model in the same map (say at different resolutions of the model-map as you do in your paper) makes sense, but I still think it’s not robust/indicative of map quality to compare CCs of the same model in different maps - unless, perhaps, all map densities are first scaled to have the same mean/stdevs.
 
In our paper we did use the same resolution cutoff for the models against all maps, but no B-values (not fully sure what you mean here actually). I used Chimera molmap to generate the model map, so nothing fancy. Would be interesting to see a similar analysis using the Phenix map-model CC calculations - I don’t think I know the internals enough to fully trust myself with that, or a way to automate it for many maps/models - will leave it for future consideration/discussion :) Greg
===========
On Apr 26, 2019, at 3:18 PM, Tom Terwilliger wrote:

Hi Greg, I redid everything to check and have attached:

EMD-20026_best_docked_model_ncs_real_space_refined.pdb  -- full docked model rigid body refined. Used for model-based sharpening.
EMD-20026_best_docked_model_ncs_real_space_refined_A.pdb -- just chain A of docked model. Used to extract a part of the map for viewing.
emd_20026_model_sharpened_box_A.ccp4   --   emd_20026.map model-sharpened with full docked model, then region of chain A extracted with map_box
emd_20026_half_map_2_model_sharpened_box_A.ccp4  -- emd_20026_half_map_2.map model-sharpened with full docked model, then region of chain A extracted with map_box
I sharpened with this command (same model for both maps):
phenix.auto_sharpen EMD-20026_best_docked_model_ncs_real_space_refined.pdb emd_20026.map resolution=1.8 sharpened_map_file=emd_20026_model_sharpened.ccp4
To me, the emd_20026_half_map_2_model_sharpened_box_A.ccp4 map looks better than the emd_20026_model_sharpened_box_A.ccp4 map.   Additionally, the model-sharpening process provides a FSC for model vs map, and this FSC is a little higher for the half map in almost all resolution shells.  Here is a plot of those FSC values.
==============
On Fri, Apr 26, 2019 at 2:57 PM Pintilie, Greg wrote:
 Hi Tom,Thanks for those details... I ran the same commands. While the model_sharpened map now looks almost as good as the full map (I must have done it wrong before), it still looks just a little worse to me and has slightly lower Z-scores - see attached table. What really impressed me is that sharpening the half map without a model produces an almost-as-good result as using the model! Nice job on that…
 
I tabulated and plotted the CC values vs d_min from auto sharpen… second attachment. It confirms the full map is still mostly better than the half map, except past d_min 1.9. I wonder if these higher frequencies are different in the two half maps, and average out when combining them. If that’s true, it still produces a full map that is slightly better, by eye and by Z-scores.
 
What I still think is even more interesting in all this is to look at CC and other scores, and whether they truly represent how good the map is visually - that’s been one of our goals anyway.
 
Finally, this analysis is also saying is that the half map is just as good as the full map, except it’s not sharpened. Actually it should be as good, otherwise the gold standard FSC for the full map would be wrong. It also says your auto_sharpen is pretty awesome - even I can run it without knowing too much about SPR, B-values, etc. Other than that, I wonder if you have the full map and half map 2 switched somehow - you mentioned you did it blind, could that be possible?
===============
On Apr 26, 2019, at 5:10 PM, Tom Terwilliger wrote:
Hi Greg, Thanks for looking into it more!  Maybe...we should be concluding that the full and half-maps at 1.8 A are not all that different (except for sharpening).  And maybe that is saying that the errors in these maps are now coming from things that are not random, so that averaging in more data is no longer making a difference?
I did check...I think I have the maps correctly labelled as I compared them to the originals dated 1 April as downloaded from the EMDB.
===============
On Apr 26, 2019, at 7:24 PM, Pintilie, Greg wrote:
I think that’s a fair conclusion; looking forward to more discussions and seeing some results from the modelling side. One thing we didn’t consider is whether the model is as good as it can be yet, based on the density.  Greg
 

EMDataResource Validation Challenges are supported by NIH National Institute of General Medical Sciences

Please send your challenge questions, comments and feedback to challenges@emdataresource.org

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer