Sept. 14, 2020
BLOOMINGTON, Ind. -- In the world of scientific research today, there’s a revolution going on – over the last decade or so, scientists across many disciplines have been seeking to improve the workings of science and its methods.
To do this, scientists are largely following one of two paths: the movement for reproducibility and the movement for open science. Both movements aim to create centralized archives for data, computer code and other resources, but from there, the paths diverge. The movement for reproducibility calls on scientists to reproduce the results of past experiments to verify earlier results, while open science calls on scientists to share resources so that future research can build on what has been done, ask new questions and advance science.
Now, an international research team led by IU’s Mary Murphy, Amanda Mejia, Jorge Mejia, Yan Xiaoran, Patty Mabry, Susanne Ressl, Amanda Diekman, and Franco Pestilli, finds the two movements do more than diverge. They have very distinct cultures, with two distinct literatures produced by two groups of researchers with little crossover. Their investigation also suggests that one of the movements -- open science -- promotes greater equity, diversity, and inclusivity. Their findings were recently reported in the Proceedings for the National Academy of Sciences.
The team of researchers on the study, whose fields range widely – from social psychology, network science, neuroscience, structural biology, biochemistry, statistics, business, and education, among others – were taken by surprise by the results.
“The two movements have very few crossovers, shared authors or collaborations,” said Murphy. “They operate relatively independently. And this distinction between the two approaches is replicated across all scientific fields we examined.”
In other words, whether in biology, psychology or physics, scientists working in the open science participate in a different scientific culture than those working within the reproducibility culture, even if they work in the same disciplinary field. And which culture a scientist works in determines a lot about access and participation, particularly for women.
IU cognitive scientist Richard Shiffrin, who has previously been involved in efforts to improve science but did not participate in the current study, says the new study by Murphy and her colleagues provides a remarkable look into the way that current science operates. “There are two quite distinct cultures, one more inclusive, that promotes transparency of reporting and open science, and another, less inclusive, that promotes reproducibility as a remedy to the current practice of science,” he said.
A Tale of Two Sciences
To investigate the fault lines between the two movements, the team, led by network scientists Xiaoran Yan and Patricia Mabry, first conducted a network analysis of papers published from 2010-2017 identified with one of the two movements. The analysis showed that even though both movements span widely across STEM fields, the authors within them occupy two largely distinct networks. Authors who publish open science research, in other words, rarely produce research within reproducibility, and very few reproducibility researchers conduct open science research.
Next, information systems analyst Jorge Mejia and statistician Amanda Mejia applied a semantic text analysis to the abstracts of the papers to determine the values implicit in the language used to define the research. Specifically they looked at the degree to which the research was prosocial, that is, oriented toward helping others by seeking to solve large social problems.
“This is significant,” Murphy explained, “insofar as previous studies have shown that women often gravitate toward science that has more socially oriented goals and aims to improve the health and well-being of people and society. We found that open science has more prosocial language in its abstracts than reproducibility does.”
With respect to gender, the team found that “women publish more often in high-status authorship positions in open science, and that participation in high-status authorship positions has been increasing over time in open science, while in reproducibility women’s participation in high-status authorship positions is decreasing over time,” Murphy said.
The researchers are careful to point out that the link they found between women and open science is so far a correlation, not a causal connection.
“It could be that as more women join these movements, the science becomes more prosocial. But women could also be drawn to this prosocial model because that’s what they value in science, which in turn strengthens the prosocial quality of open science,” Murphy noted. “It’s likely to be an iterative cultural cycle, which starts one way, attracts people who are attracted to that culture, and consequently further builds and supports that culture.”
Diekman, a social psychologist and senior author on the paper, noted these patterns might help open more doors to science. “What we know from previous research is that when science conveys a more prosocial culture, it tends to attract not only more women, but also people of color and prosocially oriented men,” she said.
The distinctions traced in the study are also reflected in the scientific processes employed by the research team itself. As one of the most diverse teams to publish in the pages of PNAS, the research team used open science practices.
“The initial intuition, before the project started, was that investigators have come to this debate from very different perspectives and with different intellectual interests. These interests might attract different categories of researchers.” says Pestilli, an IU neuroscientist. “Some of us are working on improving science by providing new technology and opportunities to reduce human mistakes and promote teamwork. Yet we also like to focus on the greater good science does for society, every day. We are perhaps seeing more of this now in the time of the COVID-19 pandemic.”
With a core of eight lead scientists at IU, the team also included 20 more co-authors, mostly women and people of color who are experts on how to increase the participation of underrepresented groups in science; diversity and inclusion; and the movements to improve science.
Research team leader Mary Murphy noted that in this cultural moment of examining inequality throughout our institutions, looking at who gets to participate in science can yield great benefit.
“Trying to understand inequality in science has the potential to benefit society now more than ever. Understanding how the culture of science can compound problems of inequality or mitigate them could be a real advance in this moment when long-standing inequalities are being recognized—and when there is momentum to act and create a more equitable science.”
The study was funded by the National Science Foundation, the Russell Sage Foundation, a Microsoft Investigator Fellowship, the IU Emerging Areas of Research initiative “Learning: Brains, Machines, Children,” and the IU Pervasive Technology Institute.
In addition to the eight IU researchers, additional team members who contributed expertise in diversity science, equity and inclusion; behavioral science; and the movements to improve science include Sapna Cheryan at the University of Washington, Nilanjana Dasgupta at the University of Massachusetts-Amherst, Mesmin Destin at Northwestern University , Stephanie A. Fryberg at the University of Michigan, Julie A. Garcia at California Polytechnic State University, Elizabeth L. Haines at William Paterson University, Judith M. Harackiewicz at the University of Wisconsin-Madison, Alison Ledgerwood at the University of California-Davis, Corinne A. Moss-Racusin at Skidmore College, Lora E. Park at The State University of New York-Buffalo, Sylvia P. Perry at Northwestern University, Kate A. Ratliff at the University of Florida, Aneeta Rattan at the London Business School, Diana T. Sanchez at Rutgers University, Krishna Savani at Nanyang Technological University, Denise Sekaquaptewa at the University of Michigan, Jessi L. Smith at the University of Colorado-Colorado Springs, Valerie Jones Taylor at Lehigh University, Dustin B. Thoman at San Diego State University, and Daryl A. Wout at John Jay College of Criminal Justice.
For more information, contact Liz Rosdeitcher, science writer in the Department of Psychological and Brain Sciences, at rosdeitc@indiana.edu or (812) 855-4507.
