Monday, December 12, 2016

The 7 biggest problems facing science, according to 270 scientists

"Science, I had come to learn, is as political, competitive, and fierce a career as you can find, full of the temptation to find easy paths." — Paul Kalanithi, neurosurgeon and writer (1977–2015)
Science is in big trouble. Or so we’re told.
In the past several years, many scientists have become afflicted with a serious case of doubt — doubt in the very institution of science.
As reporters covering medicine, psychology, climate change, and other areas of research, we wanted to understand this epidemic of doubt. So we sent scientists a survey asking this simple question: If you could change one thing about how science works today, what would it be and why?
We heard back from 270 scientists all over the world, including graduate students, senior professors, laboratory heads, and Fields Medalists. They told us that, in a variety of ways, their careers are being hijacked by perverse incentives. The result is bad science.
The scientific process, in its ideal form, is elegant: Ask a question, set up an objective test, and get an answer. Repeat. Science is rarely practiced to that ideal. But Copernicus believed in that ideal. So did the rocket scientists behind the moon landing.
But nowadays, our respondents told us, the process is riddled with conflict. Scientists say they’re forced to prioritize self-preservation over pursuing the best questions and uncovering meaningful truths.
"I feel torn between asking questions that I know will lead to statistical significance and asking questions that matter," says Kathryn Bradshaw, a 27-year-old graduate student of counseling at the University of North Dakota.
Today, scientists' success often isn't measured by the quality of their questions or the rigor of their methods. It's instead measured by how much grant money they win, the number of studies they publish, and how they spin their findings to appeal to the public.
Scientists often learn more from studies that fail. But failed studies can mean career death. So instead, they’re incentivized to generate positive results they can publish. And the phrase "publish or perish" hangs over nearly every decision. It’s a nagging whisper, like a Jedi’s path to the dark side.
"Over time the most successful people will be those who can best exploit the system," Paul Smaldino, a cognitive science professor at University of California Merced, says.
To Smaldino, the selection pressures in science have favored less-than-ideal research: "As long as things like publication quantity, and publishing flashy results in fancy journals are incentivized, and people who can do that are rewarded … they’ll be successful, and pass on their successful methods to others."
Many scientists have had enough. They want to break this cycle of perverse incentives and rewards. They are going through a period of introspection, hopeful that the end result will yield stronger scientific institutionsIn our survey and interviews, they offered a wide variety of ideas for improving the scientific process and bringing it closer to its ideal form.
Before we jump in, some caveats to keep in mind: Our survey was not a scientific poll. For one, the respondents disproportionately hailed from the biomedical and social sciences and English-speaking communities.
Many of the responses did, however, vividly illustrate the challenges and perverse incentives that scientists across fields face. And they are a valuable starting point for a deeper look at dysfunction in science today.
The place to begin is right where the perverse incentives first start to creep in: the money.


Academia has a huge money problem

To do most any kind of research, scientists need money: to run studies, to subsidize lab equipment, to pay their assistants and even their own salaries. Our respondents told us that getting — and sustaining — that funding is a perennial obstacle.
Their gripe isn’t just with the quantity, which, in many fields, is shrinking. It’s the way money is handed out that puts pressure on labs to publish a lot of papers, breeds conflicts of interest, and encourages scientists to overhype their work.
In the United States, academic researchers in the sciences generally cannot rely on university funding alone to pay for their salaries, assistants, and lab costs. Instead, they have to seek outside grants. "In many cases the expectations were and often still are that faculty should cover at least 75 percent of the salary on grants," writes John Chatham, a professor of medicine studying cardiovascular disease at University of Alabama at Birmingham.
Grants also usually expire after three or so years, which pushes scientists away from long-term projects. Yet as John Pooley, a neurobiology postdoc at the University of Bristol, points out, the biggest discoveries usually take decades to uncover and are unlikely to occur under short-term funding schemes.
Outside grants are also in increasingly short supply. In the US, the largest source of funding is the federal government, and that pool of money has been plateauing for years, while young scientists enter the workforce at a faster rate than older scientists retire.
Take the National Institutes of Health, a major funding source. Its budget rose at a fast clip through the 1990s, stalled in the 2000s, and then dipped with sequestration budget cuts in 2013. All the while, rising costs for conducting science meant that each NIH dollar purchased less and less. Last year, Congress approved the biggest NIHspending hike in a decade. But it won’t erase the shortfall.
The consequences are striking: In 2000, more than 30 percent of NIH grant applications got approved. Today, it’s closer to 17 percent. "It's because of what's happened in the last 12 years that young scientists in particular are feeling such a squeeze," NIH Director Francis Collins said at the Milken Global Conference in May.
Some of our respondents said that this vicious competition for funds can influence their work. Funding "affects what we study, what we publish, the risks we (frequently don't) take," explains Gary Bennett a neuroscientist at Duke University. It "nudges us to emphasize safe, predictable (read: fundable) science."

Truly novel research takes longer to produce, and it doesn’t always pay off. A National Bureau of Economic Research working paper found that, on the whole, truly unconventional papers tend to be less consistently cited in the literature. So scientists and funders increasingly shy away from them, preferring short-turnaround, safer papers. But everyone suffers from that: the NBER report found that novel papers also occasionally lead to big hits that inspire high-impact, follow-up studies.
"I think because you have to publish to keep your job and keep funding agencies happy, there are a lot of (mediocre) scientific papers out there ... with not much new science presented," writes Kaitlyn Suski, a chemistry and atmospheric science postdoc at Colorado State University.
Another worry: When independent, government, or university funding sources dry up, scientists may feel compelled to turn to industry or interest groups eager to generate studies to support their agendas.
Finally, all of this grant writing is a huge time suck, taking resources away from the actual scientific work. Tyler Josephson, an engineering graduate student at the University of Delaware, writes that many professors he knows spend 50 percent of their time writing grant proposals. "Imagine," he asks, "what they could do with more time to devote to teaching and research?"
It’s easy to see how these problems in funding kick off a vicious cycle. To be more competitive for grants, scientists have to have published work. To have published work, they need positive (i.e., statistically significant) results. That puts pressure on scientists to pick "safe" topics that will yield a publishable conclusion — or, worse, may bias their research toward significant results.
"When funding and pay structures are stacked against academic scientists," writes Alison Bernstein, a neuroscience postdoc at Emory University, "these problems are all exacerbated."

Fixes for science's funding woes

Right now there are arguably too many researchers chasing too few grants. Or, as a 2014 piece in the Proceedings of the National Academy of Sciences put it: "The current system is in perpetual disequilibrium, because it will inevitably generate an ever-increasing supply of scientists vying for a finite set of research resources and employment opportunities."
"As it stands, too much of the research funding is going to too few of the researchers," writes Gordon Pennycook, a PhD candidate in cognitive psychology at the University of Waterloo. "This creates a culture that rewards fast, sexy (and probably wrong) results."
One straightforward way to ameliorate these problems would be for governments to simply increase the amount of money available for science. (Or, more controversially, decrease the number of PhDs, but we’ll get to that later.) If Congress boosted funding for the NIH and National Science Foundation, that would take some of the competitive pressure off researchers.
But that only goes so far. Funding will always be finite, and researchers will never get blank checks to fund the risky science projects of their dreams. So other reforms will also prove necessary.
One suggestion: Bring more stability and predictability into the funding process. "The NIH and NSF budgets are subject to changing congressional whims that make it impossible for agencies (and researchers) to make long term plans and commitments," M. Paul Murphy, a neurobiology professor at the University of Kentucky, writes. "The obvious solution is to simply make [scientific funding] a stable program, with an annual rate of increase tied in some manner to inflation."
Another idea would be to change how grants are awarded: Foundations and agencies could fund specific people and labs for a period of time rather than individual project proposals. (The Howard Hughes Medical Institute already does this.) A system like this would give scientists greater freedom to take risks with their work.
Alternatively, researchers in the journal mBio recently called for a lottery-style system. Proposals would be measured on their merits, but then a computer would randomly choose which get funded.
"Although we recognize that some scientists will cringe at the thought of allocating funds by lottery," the authors of the mBio piece write, "the available evidence suggests that the system is already in essence a lottery without the benefits of being random." Pure randomness would at least reduce some of the perverse incentives at play in jockeying for money.
There are also some ideas out there to minimize conflicts of interest from industry funding. Recently, in PLOS Medicine, Stanford epidemiologist John Ioannidis suggested that pharmaceutical companies ought to pool the money they use to fund drug research, to be allocated to scientists who then have no exchange with industry during study design and execution. This way, scientists could still get funding for work crucial for drug approvals — but without the pressures that can skew results.
These solutions are by no means complete, and they may not make sense for every scientific discipline. The daily incentives facing biomedical scientists to bring new drugs to market are different from the incentives facing geologists trying to map out new rock layers. But based on our survey, funding appears to be at the root of many of the problems facing scientists, and it’s one that deserves more careful discussion.

No comments:

Post a Comment