Science | NPR

Who gets to use NASA's James Webb Space Telescope? Astronomers work to fight bias

With the James Webb Space Telescope safely deployed, many scientists want to use it. To minimize the effect of unconscious biases, they go through a process developed for the Hubble Space Telescope.

Listen

To embed this piece of audio in your site, please use this code:

<iframe src="https://embed.hpm.io/1/58036" style="height: 115px; width: 100%;"></iframe>
X
A crowd gathers as Nobel laureate John Mather and Northrop Grumman engineer Scott Willoughby speak in front of a model of NASA's James Webb Space Telescope at South by Southwest on March 9, 2013.
A crowd gathers as Nobel laureate John Mather and Northrop Grumman engineer Scott Willoughby speak in front of a model of NASA's James Webb Space Telescope at South by Southwest on March 9, 2013. Alex Evers

The scientists who eventually get to peer out at the universe with NASA's powerful new James Webb Space Telescope will be the lucky ones whose research proposals made it through a highly competitive selection process.

But those that didn't make the cut this time can at least know that they got a fair shot, thanks to lessons learned from another famous NASA observatory.

Webb's selection process was carefully designed to reduce the effect of unconscious biases or prejudices by forcing decision-makers to focus on the scientific merit of a proposal rather than who submitted it.

"They assess every one of those proposals. They read them. They don't know who wrote them," explains Heidi Hammel, an interdisciplinary scientist with the James Webb Space Telescope. "These proposals are evaluated in a dual-anonymous way, so that all you can see is the science."

This is a recent innovation in doling out observing time on space telescopes. And it's a change that came about only after years of hard work done by astronomers who were concerned that not everyone who wanted to use the Hubble Space Telescope was getting equal consideration.

A bias emerges in who wins telescope time

One of their first clues came when Iain Neill Reid went looking for signs of any possible gender bias in the acceptance rate for Hubble proposals. He's the associate director of science at the Space Telescope Science Institute, the science operations center for both Hubble and now Webb.

His results, published in 2014, were startling. Proposals that were led by women had a lower acceptance rate than proposals led by men. This discrepancy remained constant for more than a dozen years, the entire period of time he analyzed.

"I was surprised at how consistent it was," says Reid. "There was a systematic effect."

To try to fix this, he and his colleagues eventually developed the "blinded" proposal review process that's now being used for Hubble, Webb and NASA's other major space telescopes. So far, the evidence suggests that this is working to level the playing field — even though the measure was initially opposed by a lot of the astronomy community.

Loading...

Since any telescope in space is a rare, precious resource, NASA wants to devote its time to the most-promising science. Anyone in the world can submit a proposal for where to point a space telescope, and there's so much demand that the majority of ideas have to be rejected.

Even before the James Webb Space Telescope was launched, for example, the first call-out for proposals drew 1,173 ideas that would require 24,500 hours of prime observing time. But only 6,000 hours were available.

"It was a cutthroat competition. We rejected three-quarters of all the accepted proposals, and we're taking the top ranked quarter," says Jane Rigby, an astrophysicist with NASA's Goddard Space Flight Center who serves as the operations project scientist for the new telescope.

And even though Hubble launched more than 30 years ago, astronomers still clamor to use it. Every year they submit 1,000 or more proposals.

"Only the top 20% of those proposals will actually make it through to the telescope to get time," says Reid.

Focusing on the science, not the scientists

After his study showing a gender discrepancy in acceptance rates for Hubble proposals, Reid and his colleagues tried different solutions. First, instead of having the lead scientist's name on the front page of a proposal, they tried putting it on the second page. Then, they tried just using initials. Nothing worked.

"Then we got sensible and we said, 'Let's actually talk to some experts in social sciences,' because they can understand this better than we do," says Reid.

They reached out to Stefanie Johnson of the University of Colorado and her then-student, Jessica Kirk, now at the University of Memphis. The pair sat in on the meetings that evaluated and ranked proposals. And they noticed that a lot of the time, the discussion centered on who had submitted the proposal, rather than scientific considerations.

"There might be a question about it, like, 'Oh, you know, this seems really good but can they actually do this?'" recalls Johnson. "A lot of times, there's someone who will speak up in the room and say, 'I know this person ... they will figure it out, because that's who they are.'"

"There is this evaluation not just of the science and the research, but of the researchers," adds Kirk.

This means astronomers who were already established and well-known got an extra leg up.

"They were getting a pass," says Reid. "They had a lower bar, in some ways, to overcome, than the scientists who were coming into the field completely fresh with no track record."

This artist's conception shows what the James Webb Space Telescope looks like in space, now that all of its main components are deployed.
This artist's conception shows what the James Webb Space Telescope looks like in space, now that all of its main components are deployed. Adriana Manrique Gutierrez, NASA animator

Johnson and her colleagues recommended making the review process completely blinded and anonymous. Not only would the evaluation committees not get to see any names, all proposals would be required to be written in a way that made it totally impossible to know who the proposal was from.

Some doubted this new system would work

The institute surveyed the astronomy community to see what it thought of this potential change.

"You can imagine, the knee-jerk reaction was actually pretty polar," says Lou Strolger, deputy head of the instruments division at the Space Telescope Science Institute and chair of its working group on anonymous proposing.

He says about half of those who responded favored the idea — and those tended to be women or people who were relatively young.

"They thought that this would be a good way to make it not only more fair but to encourage new people to participate," he says.

But lots of astronomers had objections.

"They ranged from 'This will totally upset how good science is done' to 'You'll basically fool yourself into giving time to people who don't know what they are doing' — all sorts of things," recalls Strolger.

Still, the institute's director gave the go-ahead, and they plowed forward. In 2018, astronomers did their first truly anonymous review for Hubble proposals. Priya Natarajan, a theoretical astrophysicist at Yale University, was there and chaired the process. She says occasionally someone would try to guess who had submitted a proposal.

"But the buy-in from the community was so tremendous," she says, "that there would be other people on the panels who would say, 'Oh no, no, come on, let's stick to the science.'"

The gender difference flipped: "I was stunned"

And sticking to the science had a real impact. That year, for the first time ever, the acceptance rate for proposals led by women was higher than the acceptance rate for proposals led by men. The gender difference had flipped.

"I was stunned," says Natarajan. "There was an effect right away."

And when members of the selection committees were finally allowed to see who had submitted a proposal that they had just deemed worthy of telescope time, Strolger says that they never objected that the person wasn't up to the job, although they were often surprised.

"There were a lot of 'Oh, that was not at all who I thought it was' kind of reactions," says Strolger.

Data from the last few years suggests that this process continues to help close the gap between men and women in acceptance rates for Hubble proposals, and it may have improved fairness in other ways, too.

There's been a dramatic rise in approvals for astronomers who have never used Hubble before, says Strolger. "It went from something like a dozen per year to 50 per year."

What's more, data from the first round of proposals for Webb shows hints of similar results with "a much closer gap in male and female acceptance rates," says Strolger.

"This seems to be working, and it seems to be working as we anticipated it would."

What other biases could affect telescope users?

Still, anonymizing everything doesn't solve all the problems in making sure everyone has equal access, says Johnson, who notes that unconscious bias can affect who in astronomy gets advantages such as mentors and job opportunities.

"It's not perfect. It doesn't wipe out systemic bias, and I don't know of the impact that the dual-anonymization has in terms of creating greater racial equity," she says. "But it did seem to lift some of the gender bias."

Trying to track equity issues is complicated by the fact that the Space Telescope Science Institute has historically not gathered demographic information about those who submit research proposals.

"Partly by policy and partly by federal law, we're not permitted to collect that information," explains Strolger.

That's why, when Reid did his initial study looking at gender and Hubble, the best he could do was to make assumptions about gender based on the lead scientist's name or his knowledge of people in the field.

The researchers are now looking for ways to learn more about submitters, perhaps by allowing people to voluntarily or anonymously submit information about themselves to a third party.

"We hope that by providing ways in which we can get access to more demographic data," says Strolger, "we can begin to see where other biases may lie."

Copyright 2022 NPR. To see more, visit https://www.npr.org.

Transcript :

A MARTINEZ, HOST:

Now that the James Webb Space Telescope has finished unfolding itself in space, it'll undergo months of fine tuning to get it ready to, you know, do some science. And lots of scientists are hoping to use this powerful $10 billion instrument. NPR's Nell Greenfieldboyce reports on a recent effort to make sure that every researcher gets a fair shot.

NELL GREENFIELDBOYCE, BYLINE: Space telescopes are a rare, precious resource, and NASA wants to get the best science out of them, the best bang for its buck. So if an astronomer has an idea for where to point a telescope to study black holes or planets in other solar systems or whatever, there's a way to submit a proposal.

NEILL REID: Anyone from across the world can lead a proposal, can be on a proposal - Australia, China, Japan, Russia anywhere.

GREENFIELDBOYCE: That's Neill Reid. He's with the Space Telescope Science Institute. It's the science operations center for the iconic Hubble Space Telescope and now the new James Webb Space Telescope. He says the demands to use space telescopes is so high, the majority of proposals get rejected for Hubble every year.

REID: We typically get something like a thousand, 1,100 proposals. And only the top 20% of those proposals will actually make it through to the telescope to get time.

GREENFIELDBOYCE: Reid says a while back, someone asked him, does the institute know if there's any difference in the acceptance rate for Hubble proposals led by women and proposals led by men?

REID: We didn't at that point because we don't actually collect that information.

GREENFIELDBOYCE: So Reid and his colleagues did the best study they could, just using the gender suggested by the name of each proposal's lead scientist. That person is called the principal investigator, or PI.

REID: And we came up with an answer that there was this systematic difference. Proposals led by male PIs do better than proposals led by female PIs.

GREENFIELDBOYCE: The difference wasn't huge, but it was consistent for every year they looked, going back 16 years. Clearly, something was going on. The institute brought in a consultant, Stefanie Johnson of the University of Colorado. She and her research partner sat in on the committees that evaluated and ranked proposals to use Hubble. And what they noticed is that a lot of the discussion centered on who was making a proposal.

STEFANIE JOHNSON: There might be a question about it like, oh, you know, this seems really good, but can they actually do this? Like, are they sure? A lot of times, there's someone who will speak up in the room and say, you know, I know this person. They will figure it out because that's who they are.

GREENFIELDBOYCE: This means certain people got an extra leg up. So Johnson and her colleagues recommended making the review process completely blind. The evaluation committees wouldn't get to see any names, and all proposals would be written in a way that made it impossible to know who the proposal was from. The institute surveyed the astronomy community to see what it thought of this potential change.

LOU STROLGER: And you can imagine the knee-jerk reaction was actually pretty polar.

GREENFIELDBOYCE: Lou Strolger works at the Space Telescope Science Institute. He says about half of the astronomers who responded were in favor of the idea. These tended to be younger people and women. The other half had objections

STROLGER: They ranged from, you know, this will totally upset how good science is done to, you know, you'll basically fool yourself into, you know, giving time to people who don't know what you're doing.

GREENFIELDBOYCE: Still, the institute plowed ahead. In 2018, it did its first truly blinded review for Hubble proposals. Astrophysicist Priya Natarajan of Yale University was there. She says occasionally, someone would try to guess who was behind a proposal.

PRIYA NATARAJAN: But the buy-in from the community was so tremendous that there would be other people on the panels who would say, oh, no, no, come on. Let's stick to the science.

GREENFIELDBOYCE: And sticking to the science had a real impact. That year, for the first time ever, the acceptance rate for proposals led by women was higher than the acceptance rate for proposals led by men.

NATARAJAN: I was stunned that there was an effect right away.

GREENFIELDBOYCE: And when reviewers were finally allowed to see who had submitted a proposal that they had just deemed worthy of telescope time, Lou Strolger says they never objected that the person wasn't up to the job, although they often were surprised.

STROLGER: There were some, oh, that was not at all who I thought it was sort of reactions.

GREENFIELDBOYCE: Data from the last few years suggest that this process continues to help narrow the gap between acceptance rates for men and women. And it may have improved fairness in other ways, too. Strolger says there's been a dramatic rise in approvals for first-time users, astronomers who have never used Hubble before.

STROLGER: It went from something like a dozen per year to, you know, 50 per year.

GREENFIELDBOYCE: All of this convinced NASA officials to adopt this approach for other space telescopes, too. And although the brand-new James Webb Space Telescope has only gone through one round of proposal selection, there's already signs that this anonymous process is working. And that's important because a lot of astronomers are hoping to use Webb. The first call for proposals drew in more than a thousand from 44 countries. Only about 300 made the cut. Nell Greenfieldboyce, NPR News.

(SOUNDBITE OF KARTELL'S "SILVER SCREEN") Transcript provided by NPR, Copyright NPR.