How Accurate Are Personality Tests? Reliability and Validity, Explained

Three kinds of accuracy

Reliability: does it hold still?

A test is reliable when the items measuring one trait agree with each other and when you get a similar score on another day. Lee Cronbach’s coefficient alpha (1951) is the standard check on the first. For the second, Timo Gnambs’s 2014 meta-analysis of Big Five questionnaires found retest correlations over short intervals typically around 0.8, varying by instrument and trait.

Length matters

Each item carries some noise, so more items per trait average it out. Very short scales trade reliability for speed.

Stability over years

Personality does change, slowly. Brent Roberts and Wendy DelVecchio (2000) found rank-order stability rose from about 0.31 in childhood to about 0.74 between ages 50 and 70. A test that gives you a different profile every month is measuring noise. One that shows small shifts over years is measuring you.

Why type tests are less consistent

David Pittenger’s reviews (1993, 2005) reported that as many as half of people got a different MBTI type when retested after about five weeks. The underlying scale scores are fairly stable; the problem is the cut-off. Kevin Bess and Robert Harvey (2002) found MBTI scores pile up in the middle instead of splitting into two groups, so many people sit near the line where a tiny change flips a letter. Enneagram tests that report only your top type have the same weakness when two types score close together. The Big Five vs MBTI guide and Enneagram vs Big Five guide go deeper.

Validity: does it measure something real?

Robert McCrae and Paul Costa (1989) found that four MBTI scales lined up with four Big Five traits, which shows the two systems measure overlapping ground. The stronger evidence is prediction. Daniel Ozer and Verónica Benet-Martínez (2006) reviewed links between the Big Five and outcomes across health, relationships, work and civic life. Brent Roberts and colleagues (2007) found traits predicted mortality, divorce and career attainment about as well as socioeconomic status and cognitive ability. Christopher Soto (2019) retested 78 published findings and replicated 87%.

The self-report problem

Almost every online test is self-report, so it can only be as accurate as your view of yourself. David Funder’s Realistic Accuracy Model (1995) describes what that view depends on: the trait has to show up in behavior, someone has to notice it, and they have to interpret it correctly. You see your inner life clearly and your outward behavior less so.

Simine Vazire’s 2010 study found friends judged highly evaluative traits, such as intellect, better than people judged themselves, while people judged their own anxiety better than friends did. Brian Connelly and Deniz Ones (2010) found that ratings by others predicted job performance better than self-ratings. Other biases to know about:

• Social desirability. Answering as you would like to be. Strongest when stakes are high, as in job applications.
• Acquiescence. Agreeing with statements regardless of content. Good tests mix in reverse-keyed items to cancel it out.
• Reference group. Steven Heine and colleagues (2002) showed people rate themselves against those around them, which makes cross-cultural comparisons tricky.
• Current mood. A bad week pushes Neuroticism answers up. Answer for how you usually are.

The Barnum effect

In 1949 Bertram Forer gave students what he said were individual personality sketches. Everyone got the same text, and they rated its accuracy at about 4.3 out of 5. Descriptions that feel accurate are not evidence that a test is. A quick check: would your description surprise a friend if it were handed to them with someone else’s name on it? If not, it is too general to tell you much.

How to tell if a test is any good

• It names its model (Big Five, HEXACO, RIASEC, ECR-R) and where its items come from.
• It reports continuous scores, not only a type.
• It explains how answers become scores.
• It includes reverse-keyed items and enough items per trait to be reliable.
• Its claims are modest. A test that predicts your ideal partner, salary and life purpose is selling something.
• It tells you what it does not measure.

The comparison of free personality tests applies these checks to the popular options, and the 16Personalities alternatives guide covers type tests specifically.

How accurate is Depth Profile?

Held to the same checklist: the free Big Five test uses 48 IPIP-style items, 9 or 10 per trait, mixing normal and reverse-scored statements with a few scenario and slider questions. Scores run from 0 to 100, and the scoring is explained on the test page. That is more items per trait than the TIPI or BFI-10 and fewer than the IPIP-NEO-120.

The adapted items and formats have not been through the kind of independent validation the original IPIP scales have, so treat gaps of under 5 to 10 points as noise, and take a longer research instrument if you need facet-level precision. The Full Depth Report ($19) gives approximate percentiles and is clear about what it is estimating. Coaches choosing a tool for clients can read the guide for coaches or see the Coach Pack.