Finding new physics means correctly distinguishing signal from noise in LHC searches. What can go wrong and what research practices could increase the risks of noise being mistaken for a signal? With an eye on the reproducibility crisis and its causes in the social and medical sciences, we review risks and precautions in the search for new phenomena in collider physics.