The Replication Crisis Is Not One Verdict on Social Psychology

Social psychology has a peculiar museum problem.

Its most famous rooms are full of excellent machinery. Lines of visibly different lengths. A learner behind a wall. A basement prison. A tedious tray of pegs, a dollar, twenty dollars, and a theory of why people rearrange their beliefs rather than sit beside an inconsistency.

The labels are familiar enough that the exhibits can acquire the status of weather. Conformity happens. Obedience happens. Situations eat character for breakfast.

Then the replication crisis arrives with a clipboard and asks whether the exhibits still work.

The tempting conclusion is brisk, satisfying, and wrong. The machines have failed inspection. Social psychology is worthless. Somebody should put a tarp over the whole wing.

It has the emotional efficiency of a stamp. It also turns several distinct problems into one verdict, which is precisely the mental operation the crisis ought to make less fashionable.

Henrich, Heine, and Norenzayan identify the first problem: sampling.

WEIRD means Western, Educated, Industrialized, Rich, Democratic.

A study of Western undergraduates can be well designed and still be a study of a particularly thin slice of humanity. That is a failure of generalization, not proof that its result is imaginary.

Simmons, Nelson, and Simonsohn identify another: analytic flexibility.

Their demonstration has participants listen to a neutral instrumental track or a children’s song and report their age. The children’s-song group comes out older, with a respectable p value attached.

A second study makes Beatles listeners nearly a year and a half younger by their reported birth date.

Science has many powers. Altering the date of somebody’s birth is not among them.

The point is not that the authors cheated.

They disclose the mundane machinery. A covariate needed for significance. A stopping rule checked every ten participants or so. A dozen unreported variables compressed into a cleaner story.

Four ordinary degrees of freedom, combined in simulation, push a nominal five percent false-positive rate past sixty.

P-hacking is not a tiny fraud goblin hiding in the spreadsheet. It is a system of choices that can look respectable one choice at a time.

The Reproducibility Project identifies a third problem: what survives into the record.

It selected a hundred studies from a single year across three journals. Of the ninety-seven with originally significant results, thirty-six percent of direct replications also reached significance. Replicated effects averaged roughly half the original magnitude.

That is the number everyone remembers, because numbers like to dress as verdicts.

But the journal spread is the actual hinge. Around twenty-three percent for the social and personality journal, around thirty-eight for the general one, around forty-eight for the cognitive one.

The result does not say that psychology has one reproduction rate stored somewhere in a vault. It says this selected record is uneven.

Gilbert, King, Pettigrew, and Wilson challenged the interpretation the following year, arguing that power reasoning and methodological mismatches made the gloomy aggregate claim unsound. Anderson and colleagues replied in the same issue that the evidence permits multiple interpretations.

No corrected planetary number emerged.

The dispute is not pedantic interference around the real story. It is the real story. An aggregate needs an argument before it becomes an obituary.

Low power gives the arithmetic a long memory.

Cohen found typical studies poorly equipped to detect a medium effect. Later work places median psychology power around a third, well below the conventional eighty percent recommendation.

A significant result from an underpowered study is unusually likely to be false, or lucky-large. The original clears the bar in a fortunate costume. The replication arrives without the hat.

This is why the famous effects need individual labels, not a group sentence.

Asch’s line judgments are robust, but conditional.

A participant among unanimous confederates conformed on close to a third of critical trials, against an error rate near zero when alone. A single correct dissenter cut that to a small fraction, and private written responses cut it too.

Later meta-analysis found the rate varying with a culture’s individualism and declining in the United States over the following decades. A British replication with engineering students found rates far below the originals.

The effect survives as social influence with levers. It does not survive as a theorem that a crowd turns vision into soup.

Milgram’s obedience result is robust in its core and attenuated in its famous number.

The canonical sixty-five percent is twenty-six of forty men in one condition. Across eighteen variations, compliance runs from nothing at all under contradictory experimenters to well over ninety percent when responsibility is diffused.

Burger’s partial replication found about seventy percent continuing past the point where the original’s key refusals began. Perry’s archival account found extra experimenter pressure beyond the reported script, and a near-unpublished condition in which most participants disobeyed.

Authority matters. The poster did not contain the whole experiment.

Festinger and Carlsmith’s dollar-and-twenty-dollar pattern is attenuated as a unique dissonance mechanism, not a blank cheque for cognitive dissonance.

Students performed an hour of tedious tasks. Those paid a dollar to praise them rated them more positively than those paid twenty.

Festinger reads dissonance reduction. Bem’s self-perception theory gets the same pattern without the inner aversive engine.

A result can outlive a theory’s monopoly on it.

Many Labs makes the alternative to doom almost embarrassingly concrete.

Dozens of samples across a dozen countries tested thirteen effects. Ten replicated consistently across nearly every site.

Imagined intergroup contact reducing prejudice came out weak and inconsistent. Flag priming’s effect on conservatism failed. So did currency priming’s effect on system justification.

The effect is the unit of judgment.

That conclusion leaves a field with fewer sweeping anecdotes and more carefully marked objects.

A direct replication asks whether the original procedure recurs. A conceptual replication asks whether an underlying thought appears by another route. Neither should be allowed to impersonate the other.

Nor does a failure establish fraud, or a success establish an original mechanism unchanged. Those are possible stories. They are not the data’s automatic subtitles.

The repairs are correspondingly unromantic.

Preregistration time-stamps hypotheses, sample size, and analysis plans before data collection, making departures visible.

Registered Reports place the design under review before results exist, and detach publication from significance. Conventional psychology articles report significant hypotheses almost all of the time. Published Registered Reports report them under half.

The reforms change what a record can hide. They do not issue certificates of metaphysical truth.

What the crisis restored is not that the canon was fraudulent, or that replication has arrived as its executioner.

It is that status and merit must eventually meet in the same room.

A textbook can preserve a beautiful result long after its conditions, magnitude, or explanation have become conditional. Method is the institution that asks the plaque to earn its place again.

The museum stays open. The labels get better.