[ccp4bb] significant drop in Rfree ?

Protein crystallography Programs for crystallography X-ray detectors
Basic tutorials:
Protein Peptide

CCP4bb navigation

CCP4bb <-- 2008 <-- February 2008 <-- 24 February 2008

Previous message:
Subject: SHELX precompiled binaries
From: "George M {- dot -} Sheldrick" gsheldr {- at -} SHELX {- dot -} UNI-AC {- dot -} GWDG {- dot -} DE
Date: 2008-02-24

Next message:
Subject: Re: significant drop in Rfree ?
From: Ethan A Merritt merritt {- at -} U {- dot -} WASHINGTON {- dot -} EDU
Date: 2008-02-24

Subject: significant drop in Rfree ?
From: Bernhard Rupp bernhardrupp {- at -} SBCGLOBAL {- dot -} NET
Date: 2008-02-24

Dear All:

I often read and also used myself the phrase
'a significant drop in R-free' was observed or not
upon doing something to the refinement.

I am not sure about the determination of 'significance'.

We discussed on ccp4bb that the estimated error for R-free is
R-free/(n^1/2).
This is useful for the error in the absolute value of Rfree:
say I use multiple different R-free sets of same count, then the
distribution of the
resulting R-frees I get from the same data set and *same refinement*
will have this error (s.u.). Fine.

But I cannot use this error to judge whether
something I do to the refinement is a significant improvement
or not. Indirect proof through argumentum absurdum:

Assume the su of Rfree is 0.008, and I build n waters. Rf drops
from 24.4 to 13.9, by 0.005, less than the su. I build n+m waters,
say drops by 1%. Larger than su. It seems nonsense to say that
building n waters is insignificent, but n+m waters is.

I think this su as defined above cannot be used for any
determination whether changing the model was a significant
improvement. It is only a measure for the su of the absolute
rfree number.

Soooo...how can we quantify whether something gave a 'significant
improvement
in Rfree' or not? What constitutes an objective measure for a significant
improvement in R-free? What test discriminates hypothesis A from B
in terms of improvement of R-free?

Any drop? Any drop until the gap exceeds the expected ratio?

Or do I need a full blown Hamilton test (Acta 18:502 1965) to
answer that? For a simple neutron case I once used the Himmelblau
test, that worked (Ted Prince, ActaB 38:1099(1982) and does not need
R-values.

Maybe I missed such a discussion on BB before, leads welcome.
Probably addressed in some Rf-ree paper?

Thx, br

CCP4bb navigation

CCP4bb <-- 2008 <-- February 2008 <-- 24 February 2008

Previous message:
Subject: SHELX precompiled binaries
From: "George M {- dot -} Sheldrick" gsheldr {- at -} SHELX {- dot -} UNI-AC {- dot -} GWDG {- dot -} DE
Date: 2008-02-24

Next message:
Subject: Re: significant drop in Rfree ?
From: Ethan A Merritt merritt {- at -} U {- dot -} WASHINGTON {- dot -} EDU
Date: 2008-02-24