What are Claim Evaluation Tools?
The Claim Evaluation Tools is an item bank containing multiple-choice questions that assess an individual’s understanding of and ability to apply Informed Health Choices Key Concepts. The Key Concepts are principles that can help people assess treatment claims and to make informed health choices [Austvoll-Dahlgren 2015].
Who are these questions for?
All the questions in the Claim Evaluation Tools item bank have been developed for children (from the age of 10) as well as for adults. They can used to test peoples’ knowledge in:
- school and other teaching settings
- randomised trials evaluating outcomes of educational interventions
- cross-sectional studies to gauge ability in a population, and thus provide background information to help tailor interventions addressing people’s educational needs.
Flexible and tailorable
Instead of a standard, fixed questionnaire, we have developed the Claim Evaluation Tools as a flexible battery of multiple-choice questions from which teachers, researchers and others can select relevant questions for a specific purpose.
Each multiple-choice question is designed to address one Key Concept. This means that you can create your own test or questionnaire based on which Key Concepts you want to teach or measure.
Use in IHC Primary School and Podcast trials
In a project in Uganda (2012-2017), we created IHC primary school resources and IHC podcast for parents that were designed to teach 12 and 9 Key Concepts respectively (13 Key Concepts in total, as several overlapped). We evaluated the effect of the resources using a test with 26 multiple-choices questions – each Key Concept was evaluated by two questions.
The test also included questions that assess intended behaviours, self-efficacy, and attitudes. In addition, we included four questions that assess literacy, as a covariate in exploratory analyses in the trial of the primary school resources.
Open access
The Claim Evaluation Tools item bank is open access and free for non-commercial use.
If you are interested in trying out the multiple-choice questions, please see our manual that will help you to develop your own test or questionnaire in a few easy steps.
Multiple-choice questions in the Claim Evaluation Tools item bank are currently available in several languages, including: Chinese, English, German, Luganda, Norwegian, and Spanish.
For access to translated questions, please download and fill in the ‘Intention to use’ agreement (PDF) and contact us:
Astrid Dahlgren (general inquiries, English and Norwegian versions)
Anke Steckelberg (German version)
Giordano Pérez Gaxiola (Spanish- Latin American version)
We used an absolute (criterion referenced) standard to set passing scores for the test – one for the 12 concepts covered by the IHC primary school resources and one for the nine concepts covered by the IHC podcast. We used a combination of Nedelsky’s and Angoff’s methods to determine the criteria [Davies 2017], which are cut-offs for passing scores. In addition, we determined a second cut-off for scores that indicate mastery of the concepts, using the same methods.
We have developed the item bank based on extensive qualitative and quantitative feedback from methodological experts, health professionals, teachers and members of the public [Austvoll-Dahlgren 2017b].
We have validated the questions using Rasch analysis in several countries, including, China, Germany, Mexico, Norway, Uganda and Croatia [see Austvoll-Dahlgren 2017a and Publications].
Manual
Austvoll-Dahlgren A, Oxman AD, Chalmers I. Manual for preparing a test or questionnaire based on the Claim Evaluation Tools database. IHC Working Paper. 2019.
Surveys
Pivac I, Markić J, Poklepović Peričić T, Aranza D, Marušić A. Evaluating health claim assessment skills of parents with preschool children: A cross-sectional study using Informed Health Choices Claim Evaluation Tool. J Glob Health. 2023;13:04156.
Dahlgren A, Furuseth-Olsen K, Rose CJ, Oxman AD. The Norwegian public’s ability to assess treatment claims: results of a cross-sectional study of critical health literacy. F1000Research. 2021;9:179.
Oxman AD, Austvoll-Dahlgren A, Garratt A, Rosenbaum S. Understanding of key concepts relevant to assessing claims about treatment effects: a survey of Norwegian adults. IHC Working Paper. 2017.
Development and validation
Dahlgren A, Semakula D, Chesire F, Oxman AD, Mugisha M, Nakyejwe E, et al. Critical thinking about treatment effects in Eastern Africa: development and Rasch analysis of an assessment tool. F1000Res. 2023;12:887. https://doi.org/10.12688/f1000research.132052.1
Hinneburg J, Gasteiger-Klicpera B, Kasper J, Lühnen J, Maitz K, Martens T, et al. Evaluating student’s ability to assess treatment claims: validating a German version of the Claim Evaluation Tools. BMC Public Health. 2023;23(1):262.
Nsangi A, Aranza D, Asiimwe R, Munaabi-Babigumira S, Nantongo J, Nordheim L, et al. Measuring lower secondary school students’ ability to assess claims about treatment effects: establishment of a standard for passing and mastery. BMJ Open. 2023;13:e066890.
Seifert S, Maitz K, Pendl D, Gasteiger-Klicpera B. [Is health literacy a subjective assessment or an objectively measurable competence? A comparison of different instruments to measure health literacy in relation to reading literacy and cognitive skills of adolescents]. Diagnostica. 2022;68(4):197-208.
Aranza D, Milavić B, Marusic A, Buzov M, Poklepović Peričić T. A cross-sectional study on adaptation and initial validation of a test to evaluate health claims among high school students: Croatian version. BMJ Open. 2021;11(8):e048754.
Wang Q, Austvoll-Dahlgren A, Zhang J, Yu Y, Zhou Q, Yang N, et al. Evaluating people’s ability to assess treatment claims: Validating a test in Mandarin from Claim Evaluation Tools database. J Evid Based Med. 2019;12(2):140-6.
Pérez-Gaxiola G, Austvoll-Dahlgren A. [Psychometric validation of a questionnaire to measure the ability of the public to evaluate claims about treatments]. Gac Med Mex. 2018;154(4):480-95.
Davies A, Gerrity M, Nordheim L, Okebukola P, Opiyo N, Sharples J, et al. Measuring ability to assess claims about treatment effects: establishment of a standard for passing and mastery. IHC Working Paper; 2017.
Semakula D, Nsangi A, Oxman AD, Sewankambo NK, Guttersrud Ø, Austvoll-Dahlgren A. Measuring ability to assess claims about treatment effects in English and Luganda: evaluation of multiple-choice questions from the “Claim Evaluation Tools” database using Rasch modelling. IHC Working Paper; 2017.
Austvoll-Dahlgren A, Guttersrud O, Nsangi A, Semakula D, Oxman AD, Group IHC. Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the ‘Claim Evaluation Tools’ database using Rasch modelling. BMJ Open. 2017;7(5):e013185.
Austvoll-Dahlgren A, Semakula D, Nsangi A, Oxman AD, Chalmers I, Rosenbaum S, et al. Measuring ability to assess claims about treatment effects: the development of the ‘Claim Evaluation Tools’. BMJ Open. 2017;7(5):e013184.
Austvoll-Dahlgren A, Nsangi A, Semakula D. Interventions and assessment tools addressing key concepts people need to know to appraise claims about treatment effects: a systematic mapping review. Syst Rev. 2016;5(1):215.
(Check also Publications page, in case newer studies haven’t been added here.)
Are you interested in translating the multiple-choice questions to other languages or contributing to the ongoing development and validation of evaluation tools?
Here are some protocol examples of what these kinds of studies can look like:
First, download the ‘Intention to use’ agreement (PDF). Then, contact us:
Contact us for general inquiries or to send feedback about the Claim Evaluation Tools.