Online Article

Which one comes first? Reliability test or validity test?

5th February 2023

Related topic: Quantitative research

Author: Ady Hameme N. A.

The question of which comes first in psychometric testing, reliability or validity, is a debated topic in the field of psychometrics. Some argue that reliability must be established before validity, while others argue that validity must be established before reliability.

According to Nunnally and Bernstein (1994), a test's reliability should be established before its validity can be accurately determined, as unreliable test scores can lead to invalid conclusions. Reliability refers to the consistency or stability of test scores over time. In other words, if a person were to take the same test twice, their scores should be consistent. This consistency can be established through several methods, including test-retest reliability, alternate form reliability, and internal consistency. On the other hand, validity refers to the accuracy of test scores and the extent to which they measure what they were intended to measure. This can be established through content, criterion-related, and construct validity.

Alternatively, some authors argue that validity should be established first, as a test that is not valid cannot be considered reliable, even if it produces consistent results. APA (2016) supports this view by affirming that the foundation of educational and psychological measurement is validity. The APA further explains that a test's validity must be established before its reliability can be assessed, as a test that is not valid cannot be considered reliable, even if it produces consistent results.

While there is no clear consensus on whether reliability or validity should come first, many experts agree that both are important and should be considered during the entire process of psychometric testing. Both reliability and validity are essential in order to produce trustworthy and useful test results. An unreliable test would produce inconsistent results, making it difficult to determine an individual's true score. On the other hand, a test that is not valid would produce inaccurate results, making it difficult to understand the meaning of the test scores.

It is possible to prioritize reliability or validity, depending on the purpose of the test. For example, if the test is being used for high-stakes decisions, such as employment or admission to a particular program, then it is important to establish the validity of the test. In this case, it is important to ensure that the test measures what it is intended to measure and that the scores accurately reflect the abilities or traits being tested.

In contrast, if the test is being used for research purposes, it may be more important to establish reliability first. In this case, it is important to ensure that the results of the test are consistent and stable over time. This will allow researchers to accurately compare the results of the test across different groups and over time.

In conclusion, both reliability and validity are important in psychometric testing. It is important to consider both during the entire process of test development and administration, and the order of priority may depend on the purpose of the test. In order to produce trustworthy and useful test results, it is important to ensure that tests are both reliable and valid and that they are used in accordance with ethical and professional standards.

Cite this article: Ady Hameme, N. A. (2023, February 5). Which one comes first? Reliability test or validity test?. Retrieved <insert month> <insert date>, <insert year>, from https://www.myadvrc.com/publications/article-16

< previous next >