Systematic reviews of measurement instruments

VU medisch Centrum

E&B
EMGO institute for health and care research

Systematic reviews of measurement properties

A systematic review of measurement properties is a systematic review in which the content and measurement properties of measurement instruments are critically appraised and compared. Systematic reviews of measurement properties are useful tools for selecting a measurement instrument for a certain purpose. In addition, it are useful tools to identify measurement instruments that need further validation.

Different types of systematic reviews of measurement properties
Several types of systematic reviews of measurement properties exist. Examples are:


Published systematic reviews of measurement properties of health status measurement instruments
We systematically collect all systematic reviews of measurement properties of all available measurement instruments that intend to measure (aspects of) health status or (health-related) quality of life. To this aim, regular searches are performed in Pubmed, Embase, of Psycinfo. The number of such systematic reviews has increased from none or one review per year in the beginning of the 1990s up to more than 50 in 2010.

Download a file with 335 reviews (updated through March 18, 2011) here 
We are currently updating this file up to 2012.




The first 148 published systematic reviews have been described and their methodological quality has been evaluated. This study was published in Quality of Life Research in 2009 (open access). It was concluded that systematic reviews of measurement properties often are of low methodological quality. The most important methodological weaknesses of the reviews were a poor search strategy, a lack of adeqaute reporting of the methods used to perform the systematic review, and a lack of use of standards and criteria of adequacy to assess the methodological quality of the included studies.
The poor quality of these reviews hampers evidence-based selection of instruments.


Performing a systematic review of measurement properties
In general, a systematic review of measurement properties is performed according to eight steps. The steps are briefly described below.

Objective
We recommend to clearly define the type of review that one intends to perform. In addition, it is important to define the construct of interest and the population of interest.

Inclusion and exclusion criteria
The focus of the study should be the development or evaluation of the measurement properties of a measurement instrument. Often all studies are included that report on at least one or more measurement properties. Studies in which another measurement instrument is validated against the measurement instrument of interest, are often excluded because these studies only offer indirect evidence for validity of the measurement instrument of interest.

Literature search
Studies on measurement properties are sometimes difficult to find in Pubmed or other databases due to poor indexing, large variation in terminology, and poor reporting of measurement properties. Therefore we recently developed two search filters for finding studies on measurement properties in Pubmed: 

The search filters were recently published

Terwee et al. Terwee CB, Jansma EP, Riphagen II, de Vet HCW. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res 2009:18:115-1123. (open access)

We recommend to perform the selection of abstracts and full-text articles by two persons, independently from each other. Consensus should be reached afterwards, if necessary with a third independent person. This strategy is also recommended for systematic reviews of other studies, e.g. clinical trials and diagnostic studies.

Data extraction
It is also recommended to perform the data extraction by two persons, independently from each other. Consensus should be reached afterwards, if necessary with a third independent person.

Assessment of the methodological quality of the included studies
In a systematic review it is important to take the methodological quality of the included studies into account. If the results of high quality studies differ from the results of low quality studies, this can be an indication of bias. The COSMIN checklist can be used to evaluate the methodological quality of studies on measurement properties of health status measurement instruments.

Assessment of the results of the included studies
In this phase of a review criteria for what constitutes good measurement properties should be applied to the results of a study on measurement properties, to assess the quality of the included measurement instruments. An example of a checklist providing such criteria was published by our group:

Terwee et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60:34-42.

Conclusions
It is often useful to present a sort of ‘consumer’ table of the ratings of the measurement properties of all included measurement instruments. Examples of such tables can be found in these example reviews:

Bot SD et al. Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature. Ann Rheum Dis. 2004;63(4):335-341.
de Boer MR et al. Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004;24(4):257-273.

Reporting
Clear reporting of the methods and results of the systematic review are important. In published systematic reviews it is sometimes unclear if things were not done (e.g. data extraction performed by at least two independent reviewers) or if they were not reported. This makes it difficult to judge whether the review had been adequately performed. Recommendations for reporting systematic reviews of measurement properties can be found in this article:

Mokkink LB et al. Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Qual Life Res 2009;18:313-333. (open access)