Even for me this one is going to be slightly obscure, full of unnecessary references to books you’ve probably never even heard of and highly opinionated. Today we’re going to talk about reputation in science. Whether it’s possible to build one and how that might affect your career.
We’ll start with Florence Nightingale and her rose diagrams. As the first woman elected to the Royal Statistical Society, Nightingale was a queen of data visualisation. She used elaborate graphs to demonstrate, albeit with slight manipulation, that hygiene was important in hospitals. This was used to secure bills and funding to improve public health. Part of the reason she was so successful was because she was able to effectively convey complex data in a form that would be easily digestible by a general audience. Nightingale was, in essence, one of the earliest demonstrations of the vividness effect in science.
Nisbett and Ross described the vividness effect in the following way: vivid information is “likely to attract and hold our attention and to excite the imagination to the extent that it is emotionally interesting, concrete and imagery-provoking, and proximate in a sensory, temporal, or spatial way”. Eurgh. Writing was terrible in the 1980s. Basically what they’re saying is if it’s loud, colourful and obvious we’re more likely to notice and pay attention to it than if it’s drab and dull.
By this point, I can see you think I’m slightly crazy but stick with me, more references are coming…
In his famous work on baseball, the author of Moneyball – Michael Lewis – talks about how the vividness effect was important in recruiting players. He says that very vivid players, ones that looked good on the field, were recruited preferentially despite the fact that statistics and research showed that the slightly dull players were often better. You can see where I’m going now, can’t you?
In science, vividness seems to be becoming increasingly important.
Parra and colleagues, in their paper investigating the nature of scientific reputation, found that traditional bibliometric indicators of success – things like number of citations – have little to no correlation with a scientists ‘reputation’. But according to a PNAS paper by Peterson and colleagues, ‘a reputation is developed, a standing within a research community, based largely upon the quantity and quality of his/her publications’. So how do we parse these two disparate ideas?
Interestingly the body of work by Peterson suggests we probably don’t have to. We generate papers, bigger and better ones, because we have money and time. And people give us money and time when we already have money. It’s called the Matthew effect and Peterson has written some excellent papers on it. In fact if you have time and you like maths, Peterson’s papers on careers are all worth a read. I, frankly, don’t have time and am terrible at maths but I read them anyway because they’ve actually got some really useful, if obvious, tips for early career researchers.
Impact factor is commonly used to evaluate the relative importance of a journal. However, not the importance of you… there are other ways to make impact.
In his 2015 paper on collaboration, for example, Peterson outlines the importance of good collaborative networks, especially for early career scientists. Developing so-called ‘superties’ allow scientists to reduce workload, spread costs and de-risk projects by sharing them with others. In other papers he goes on to discuss how important it is to develop these ties as a young researcher, largely because ‘young scientists lacking reputation can be negatively affected by social stratification in science’. Find yourself a big name to align with and you’re more likely to do well.
But is the way we get ‘big names’ relevant any more? These days big names are not the same as they were in the ‘80s or even the early 2000s. These days big names have popular Twitter profiles and Instagram feeds. Big names run podcasts and appear on the news. Big names are loud and wear bowties. Big names do not necessarily do good science.
And how would we even tell if they did? Scientific output has increased to a ridiculous degree. In 2014, there were over 7 million academic papers published. A bit of fun with a calculator has revealed that to be 14 new papers every minute. Yes folks, you read that right, 14 new papers every minute. Who on earth can keep up with that?
In a work on the history of the field that I work in (extracellular vesicles) I cite some key original papers by some guys called Chargaff and West. One of the things I enjoy talking about is their series of papers entitled ‘studies on the chemistry of blood coagulation’. It’s a great demonstration of what science used to be. ‘We’ll find a couple of new and interesting things, pop them in a paper, tell you we’re not sure why it all happened but eyes out for the next paper, coming soon to a printer near you!’.
Now I’m not advocating for a relapse to the good old days of strictly white men in white coats smoking in labs with no gloves on, but the way we create scientific acumen and prestige these days has gotten, in my opinion, a little out of hand.
Entirely by accident whilst looking for something else I came across a wonderful paper by Michael Fire and Carlos Guestrin on the current metrics used to analyse the 14 papers a minute we’re churning out. They highlight the importance of Goodhart’s Law – when a measure becomes a target it ceases to be a good measure. From public policy they show how this can go wrong by citing the use of ‘crime metrics’ in the New York police department. By having a target of catching so many criminals it incentivises catching criminals, to the detriment of people who maybe didn’t actually commit a crime. In academic publishing it incentivises publishing lots of data, repeatedly in high impact journals, leading to the development of ‘paper mills’ and predatory journals.
Fire and Guestrin used two sentences in the first couple of paragraphs which I need to quote because they highlight the problem with academic metrics, and academic reputations. Because of Goodhart’s law, and the changes in the numbers of papers we produce and how we produce them they say that ‘Academic publishing has changed considerably; now we need to reconsider how we measure success’. And we need to reconsider this because this way of measuring things is changing the environment we work in. Specifically, they say that it ‘contributes to a hypercompetitive research environment, which is changing academic culture—and not in a positive direction.’
In his discussion with Michael Lewis, Stephen Dubner points out that in using metrics and not vividness to pick baseball players, the game has become less fun to watch. Lewis’ response to this is that he would put a lion on the field. The increased risk of some minor injuries, hopefully not to the lion, would increase the excitement to the point where people couldn’t watch it without the lion. In academia we’ve become more and more reliant on this reputational metric where the constant presence of the lion is the only way that people can be pushed to do good science. From pressure comes diamonds, or whatever the heinous phrase is.
But in doing that we’re also adhering to metrics which were applicable before the age of pre-prints, before the age of social media and blogs and podcasts. I’m not saying these are now the only way to develop a reputation but impact should no longer be considered just output. By adhering to these metrics, we’re adhering to a system where impact is cold and hard. And in doing that we can excuse poor behaviour because of scientific reputation. And that way madness lies.
Ed: We completley agree Yvonne, in-fact, we might argue that writing blogs and recording podcasts with us, could be just as benefical to your reputation… anyone reading want to do that? Drop us a line
Dr Yvonne Couch
Author
Dr Yvonne Couch [1] is an Alzheimer’s Research UK Fellow at the University of Oxford. Yvonne studies the role of extracellular vesicles and their role in changing the function of the vasculature after stroke, aiming to discover why the prevalence of dementia after stroke is three times higher than the average. It is her passion for problem solving and love of science that drives her, in advancing our knowledge of disease. Yvonne shares her opinions, talks about science and explores different careers topics in her monthly blogs – she does a great job of narrating too.