 |
| THE QUALITY OF MEASUREMENTS: How Can Validity Be Improved? |
|
| THE QUALITY OF MEASUREMENTS: How Can Validity Be Improved? |
 |
 |
 |
 |
OBJECTIVES |
| At the end of this session you should understand the main methods of improving validity - by minimising selection, information and confounding bias. |
Following the classification of bias, select the study groups appropriately.
The group you are studying should be representative of the population to which you hope to generalise your study results. Much depends on your study objectives and the study design. Strong contrasts with respect to the variable of interest (usually an exposure) are advantageous. Internal controls are often better (less heterogenous) than external controls. Several control groups especially for case-control studies may be problematic. You need to be careful of the healthy worker effect. In case control studies you need to obtain as many of the cases in your study period as possible that are representative for a geographical region. The controls need to be representative of those who would have come for treatment or registration to the same health facility or vital registry if they had developed the disease under study.
Think about the sources of information that may give rise to bias systematically as in Table 1.
| Observer | Intraobserver variation - training |
|---|---|
| Inter-observer - standardisation of procedures and training | |
| Instrument | Questionnaire or other instrument - standardise, pretest to validate, translate and backtranslate |
| Machine- calibrate and frequent checks before, during and after testing | |
| Respondent | Information recall - dummy questions to test for overreporting unrelated to the study aims and objectives, choose cancer controls or serious disease controls in case control studies where serious disease is the outcome that you are studying in relation to an exposure (which is known not to be linked to the cancers or serious conditions in the controls); use same type of respondent throughout (eg proxy or subject should not be mixed). |
| Timing of information recalled - don't ask for unrealistically distant recall. | |
| Emotional state - subject not too distracted. | |
| Study setting - minimise undue pressures to answer in the affirmative or negative. | |
| Consider biological (diurnal and other cycles) always and their potential impact on what you are measuring. | |
Care should be taken to minimise sources of misclassification, and to correctly interpret findings when you suspect that there is misclassification. This is done by careful transcription of records or back-checking for exposure and disease status for study subjects.
If misclassification is non-differential, effects tend to be diluted and may be missed
If misclassification is differential with respect to exposure status or outcome it is possible to have a either a false positive effect or also to miss a real effect. Sometimes one can anticipate the direction of bias introduced, and that is helpful. An example might be that those with a condition were investigated more thoroughly with regard to their exposure status in the past to a chemical than those without that condition. It might then be expected that this would strengthen any effect found (if real) or create a false positive effect (if in truth there was no effect).
Randomisation of exposure allocation to experimental studies is the best method but this depends on whether this is feasible and ethical. Mostly it is not.
Restriction of a potentially confounding variable to one category, or a narrow enough range of a continuous variable to rule out confounder variation within this range can be done. For example, if one is worried about confounding by age of an exposure-response relationship, this can be looked at within a narrow age range to see if there is still an effect, or if this effect is different from what you found for the whole population including all age groups. This is not often practically feasible as subject numbers are too small to stratify the group you have studied into many different cells.
Individual or group (frequency) matching is a generalised form of restriction. This can only be safely applied to follow-up studies where there are often logistical and cost-efficiency problems. It is problematic in case-control studies because it may bias the selection of controls who then no longer represent the source population generating the cases.
Control for confounding during the analysis phase, and this is most frequently done by stratified analysis or multivariate statistical methods.
