Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

RATIONALE, AIMS AND OBJECTIVES: Although there is wide acceptance of the necessary domains for the evaluation of measures of health outcome, there has been little development or evaluation of methods of data extraction, such as checklists. This study aimed to assess the inter-rater reliability of data extraction from published articles using a new electronic database developed for structured reviews of health outcome measures. METHODS: Domains considered important in the evaluation of measures of health outcome provided the foundation for a checklist for data extraction using a new electronic database. The database utilized Microsoft Access 2000 software (Microsoft Corporation, 2000). Following training, three independent reviewers extracted data from seven selected articles using the electronic database, and assessed article and measure quality against pre-defined criteria. Data extraction was compared for consistency and item content. Inter-reviewer agreement for categorical data was assessed by multiple kappa correlation. RESULTS: Analysis demonstrated strong agreement between reviewers for all aspects of data extraction. However, lack of clarity in published articles affected the ability to clearly identify measures of outcome. CONCLUSIONS: The high level of inter-reviewer agreement supports the use of multiple trained reviewers in data extraction for reviews of measures of outcome using the checklist and Access software described. The electronic database supports standardized data extraction from published articles, benefiting from the combination of data extraction and data entry in a single step. Adaptation of the database to support structured reviews of measures of outcome adopted in the evaluation of other health states is proposed.

Original publication




Journal article


J eval clin pract

Publication Date





329 - 337


Ankle Injuries, Connective Tissue, Humans, Observer Variation, Outcome Assessment (Health Care), United Kingdom