Home › Guides › Research Design › Validity & reliability

Validity vs reliability in research

Q: What is the difference between validity and reliability?

Validity is whether you are measuring the right thing — does the instrument capture the concept you intend. Reliability is whether you measure it consistently — would you get the same result on repeated measurement. Validity is about accuracy; reliability is about consistency.

Q: Can a measure be reliable but not valid?

Yes. A bathroom scale that is always 3 kg heavy is perfectly reliable (consistent) but not valid (wrong). The reverse is not possible in the same way: a measure cannot be valid if it is not reliable, because inconsistent measurement cannot consistently capture the true value.

Q: What are the types of validity?

Common types are face validity (does it look right), content validity (does it cover the whole concept), construct validity (does it measure the underlying construct), and criterion validity (does it agree with a gold-standard or predict an outcome, i.e., concurrent and predictive validity). In experiments, internal and external validity describe causal confidence and generalizability.

Q: How do I measure reliability?

Depending on the type: test-retest reliability (same test, two time points), inter-rater reliability (agreement between coders, e.g., Cohen's kappa), and internal consistency (do scale items hang together, usually Cronbach's alpha). Which one matters depends on your instrument.

By Dr. Rafiq Muhammad, MD, PhD · Updated June 2026

Validity is whether you’re measuring the right thing — does your instrument actually capture the concept you intend. Reliability is whether you measure it consistently — would you get the same answer on repeat. Validity is accuracy; reliability is consistency. Examiners check both, and confusing them is a classic methods-section slip.

The classic example: a bathroom scale that always reads 3 kg too heavy is perfectly reliable (same answer every time) but not valid (it’s the wrong answer). That asymmetry is the key insight below.

Why a measure can be reliable but not valid (and not the reverse)

A measure can be consistent yet wrong — reliable but invalid (the scale). But a measure cannot be valid without first being reliable: if your instrument gives a different answer each time, it can’t be consistently hitting the true value. So reliability is necessary for validity, but not sufficient. Establish reliability, then validity.

The types you should name

Validity (right thing)	Reliability (consistent)
Face — looks right on the surface	Test–retest — same test, two time points
Content — covers the whole concept	Inter-rater — agreement between coders (Cohen’s κ)
Construct — measures the underlying construct	Internal consistency — items hang together (Cronbach’s α)
Criterion — agrees with a gold standard / predicts (concurrent & predictive)

In experiments you’ll also meet internal validity (confidence the IV — not a confounder — caused the change) and external validity (how far the result generalizes).

For internal-consistency reliability of a scale or questionnaire, the free Cronbach’s α calculator estimates α from the average inter-item correlation or from item and total variances.

Get the free Research Design Decision Tree

A step-by-step decision tree from your research question to the design that fits it — qualitative, quantitative, or mixed methods — with the sampling approach each path implies, five matched question-to-design examples, and a ten-point pre-commitment checklist, from Research Design Simplified. We’ll email you the download link.

Frequently asked questions

Validity vs reliability?

Validity = measuring the right thing (accuracy); reliability = measuring it consistently. Both are needed.

Can a measure be reliable but not valid?

Yes (the always-3kg-heavy scale). But it can’t be valid without being reliable.

What are the types of validity?

Face, content, construct, and criterion (concurrent & predictive); plus internal and external validity in experiments.

How do I measure reliability?

Test–retest, inter-rater (Cohen’s κ), and internal consistency (Cronbach’s α) — pick the type that fits your instrument.

The workbook for this

The Statistical Test Selection Workbook

“Which test do I use?” It is the single most common question asked by graduate students and clinical researchers.

58 fill-in templates. Paperback $31.99 on Amazon

See inside the workbook

Just want the templates? The Statistical Test Selection Kit: every template as a fillable PDF, $19.

← Research design guide Open the Cronbach’s α calculator →