What is difference between repeatability and reproducibility? Repeatability measures the variation in measurements taken by a single instrument or person under the same conditions, while reproducibility measures whether an entire study or experiment can be reproduced in its entirety.
Every laboratory scientist has a horror story. The five-minute step they didn’t know they needed, which ended up costing them five months — or five years. Maybe it was swirling the plate as crowded cells were split between culture dishes. Or maybe the published protocol said to wash your sample once and heat thrice but meant the opposite, so that following the printed instructions destroyed the sample.
More than 60% of respondents to a 2016 Nature survey said they had tried to repeat other scientists’ experiments and been unable to do so [1]. A poll of members of the American Society for Cell Biologists [2] similarly found that more than 70% had been unable to replicate [3] a published experimental result, with incomplete detail in the original protocol given as the most common explanation.
That’s no surprise to Tim Errington, director of research at the Center for Open Science in Charlottesville, Virginia, who managed the Reproducibility Project: Cancer Biology (RP:CB), which launched in 2013 to replicate results of prominent cancer-biology papers. Errington sees two overarching reasons for poorly explained methods. One is a lack of incentives or training for better descriptions. The other is that researchers assume everyone works the same way and fail to recognize what details are crucial. The word ‘standard’ is a red flag for this type of thinking, he says. “There is no such thing as a standard protocol: it’s only standard in your lab.”
But things are improving, says Elizabeth Iorns, who helped to launch the RP:CB and is chief executive at Science Exchange in Palo Alto, California, which provides research-outsourcing services. “There’s a lot of progress in documentation and tools that are available to people to use.” A move towards automated experiments and data collection is also helping, she says, and people are more aware than ever about the need to describe reagents precisely.
Such tools will move the needle on reproducibility only if they are widely used, Errington warns. “Even though we have them, it’s not systemic yet, or rewarded.”
Communicating methods more accurately, research-improvement advocates say, requires changes to both mindset and workflow. Here are some strategies that can help….
Read the blog in full and access the strategies on the Nature Careers Website – https://www.nature.com/articles/d41586-021-02428-3 [4]