A growing problem in research and publishing involves “paper mills”: organizations that produce and sell fraudulent manuscripts that resemble legitimate research articles. This form of fraud affects the integrity of academic publishing, with repercussions for science as well as the general public. How can fake articles be detected? And how can paper mills be counteracted?
In this episode of Under the Cortex, Dorothy Bishop talks with APS’s Ludmila Nunes about the metascience of fraud detection, industrial-scale fraud and why it is urgent to tackle the fake-article factories known as “paper mills.” Bishop, a professor of neurodevelopmental psychology at Oxford University, is also known for her breakthrough research on developmental disorders affecting language and communication.
Unedited transcript:
[00:00:12.930] – Ludmila Nunes
In research and publishing, “paper mills” are organizations that produce and sell fraudulent manuscripts that resemble legitimate research articles. This form of fraud affects the integrity of academic publishing with repercussions for the science itself, but also for the general public. How can these fake articles be detected? And how can paper mills be counteracted? Today’s episode will explore these questions. This is under the cortex. I am Ludmila Nunes with the association for Psychological Science. Today I have with me Dorothy Bishop, professor of Neurodevelopmental Psychology at Oxford University. Dr. Bishop is known for her breakthrough research on developmental disorders affecting language and communication, which has helped to build and advance the developmental language impairment field. But Dr. Bishop’s interests go well beyond neurodevelopmental psychology, as her blog, Bishop Blog, illustrates. At the APS International Convention of Psychological Science, held last March in Brussels, dr. Bishop presented her work on the meta science of fraud detection, discussing industrial scale fraud and why it is urgent to tackle the so called paper mills, or fake articles factories. I invited Dr. Bishop to join us today and tell us more about the battle against industrialized cheating in academic publications. Dorothy, thank you for joining me today.
[00:01:50.400] – Ludmila Nunes
Welcome to Under the Cortex.
[00:01:52.360] – Dorothy Bishop
Thank you very much for inviting me, Ludmila, it’s lovely to be here.
[00:01:56.870] – Ludmila Nunes
So I was lucky enough to see your presentation at ICPS, and I thought this topic was fascinating because it’s a different type of fraud than what we usually talk about in psychology. We talked a lot about replication crisis. Some researchers faking their data, but this is different. Do you want to explain us what paper mills are?
[00:02:23.950] – Dorothy Bishop
Yes. As you said before, they’re this sort of industrial scale fraud. I think perhaps initially started out as helping people write papers for language reasons, but then sort of veered into actually creating fake papers that they would then sell to people that needed publications and then place them in journals. So you have these strange papers that really you can divide into two types. The first type is like the plausible paper mill paper, which is often made from a template of a genuine paper. So they’ve taken a genuine paper and then just tweaked it a bit so that it looks like a perfectly decent piece of work, but has had some changes. And then there’s another kind which seems to be very weird papers that anybody who reads them will think they’re extremely strange, and they possibly are computer generated, or in some cases, they seem to be plagiarized or possibly just based on very basic things like undergraduate essays. So they’re a bit of a mishmash, but they are things that would not normally pass peer review. And so the question that really started to interest me is, how did these things even get into the literature?
[00:03:39.910] – Dorothy Bishop
The first kind of paper mill paper, which looks plausible, you can imagine, would have just have fooled a regular editor. But this second kind that look very weird and don’t often make much sense and may have all sorts of bizarre features, suggests that we’ve got a situation in some journals where the editorial process has been compromised and where editors may even be complicit with the paper mill.
[00:04:07.150] – Ludmila Nunes
And to make it clear, we are talking about articles that can make their way into more prestigious peer reviewed journals.
[00:04:14.840] – Dorothy Bishop
Yes, I mean, again, there’s a very wide range and it’s an evolving scenario. I sometimes draw parallels with things like COVID because it’s a bit like being suddenly attacked by some virus that mutates. But what people have discovered is that there’s a lot of money in this. And of course, if you can place a paper in a prestigious journal, that’s more valuable. And indeed, if you go online, these tend to not be in English, but there are some of these adverts in English, but mostly in other languages where the papers are actively advertised. So they’ll say authorship for sale and the amount you pay will depend on the caliber of the journal and on your authorship position. So there’s always a pressure from the paper mills. They regard it as a great success if they can get a paper into a prestigious journal, or at least into a journal that’s featured in, say, Web of Science at Clarivate, which means it gets indexed so that your citations would count in an H index. And I should say that you might say, well, who would cite these if they’re really rubbish? But they also do look after the citations.
[00:05:22.990] – Dorothy Bishop
So what will happen if you get a paper mill paper in one of these journals? A whole load of citations will be added, often of very little relevance to the content of the paper, but it’s clearly to boost citations of other paper mill papers. So there’s this entire industry which is rather like a sort of parallel universe where they’re playing at doing science and the outputs have all the hallmarks of regular science, but the content is just rubbish.
[00:05:51.810] – Ludmila Nunes
And so what made you aware and interest in this topic? Was it a specific event or you just started seeing these articles and thinking this seems weird that these even got published?
[00:06:05.970] – Dorothy Bishop
Yes, I’m trying to think back actually to exactly how it happened. But I think I was already interested after I retired. I retired last year, this time last year, and I decided I wanted to look a bit more at fraud because I’d been focusing on the reproducibility crisis that you’d mentioned and on questionable research practices and how to improve science more generally. But like most people, I think I regarded fraud as relatively rare and much more difficult to prove and so on. And I think it was probably the person that a Russian woman who co authored a paper on this with me, Anna Abalkina, who I knew from a Slack channel where we exchanged information, and she mentioned that she was aware of a paper mill and that some of the papers in it were psychology papers. But she is an economist, and she didn’t really feel she could evaluate these. So I offered to look at them. And then we found a little nest of six of these papers in a perfectly reputable psychology journal, which really surprised me that they really raised questions about how did they get in there, because these were of the kind that they were probably based on original projects or something, but they were no way of a caliber that would get into a journal.
[00:07:22.040] – Dorothy Bishop
It was often quite hard to work out what they were actually about. They sort of meandered around and talked in great generalities, and you might say, well, how do you know it was not just the editor having an off day? Well, Anna’s strategy for identifying these paper mill papers and these six came from a much bigger paper mill, but it was that you could find papers where the email address of the authors was a fake email address. So the domain looked like an academic domain, but you can track it back, and it’s a domain that is available for purchase on the web. So you can buy these email domains, and then you can make loads and loads of email addresses. And typically, they were email domain addresses that didn’t match the country of the author. So you’d have somebody with a UK or Irish looking email address, but from Kazakhstan, for example. That can happen if somebody’s moved jobs. But it kept happening in all these papers, and then I looked at them, and they clearly didn’t look as if they should be published anywhere. But you could say, well, I’m just being difficult there.
[00:08:31.910] – Dorothy Bishop
But the real interesting point with these was that in most cases, they had open peer review. And so you could look at the peer review, and the peer review did not look like normal peer review. It was incredibly superficial. So it would say things like, the paragraphs are too long, break them up. Or it would say, the reference list needs refreshing. And the same terminology used across these six papers, very short referee reports asking for very superficial changes, not engaging at all with the content. And the peer reviewers who are in some cases listed were people if you tried to find them online, they didn’t seem to exist, or they had very shady presences. You couldn’t find that they’d ever published anything. So it looked as if the whole business of peer review had been compromised by using fake peer reviewers with fake email addresses and fake authors with these fake email addresses to get these publications accepted in the journal.
[00:09:31.750] – Ludmila Nunes
So you started by looking at this set of six articles, and then you went beyond that.
[00:09:37.910] – Dorothy Bishop
Well, Anna had already gone well beyond that, I have to say. She’s got a spreadsheet with I think now it’s gone to a couple of thousand papers, all with these same sorts of characteristics. And of course the difficulty is you do tend to have to go through carefully. You don’t want to accuse somebody who’s written a perfectly legitimate paper. So it’s quite painstaking work. She’s found, more recently another journal. This one was the Journal of Community Psychology. It’s a Wiley journal. There was another journal, I can’t remember the publisher, but another psychology journal that looked thoroughly reputable where another little cluster occurred. And of course, one of the things we wanted to do was to say, how did this happen? Because it does suggest that the editor was either asleep at the job or was involved in the paper mill. And it was hard to believe this because it looked again like a perfectly reputable editor. So the way we tried to test that was by actually submitting a paper to the Journal in which we described the paper mill to sort of like act as a sort of sting operation, in the sense that we thought if he was really just not reading the papers and delegating to somebody to just sort of go through the motions, it maybe he wouldn’t notice that our paper was highly critical of his journal.
[00:10:49.460] – Dorothy Bishop
In fact, he did notice and he just desk rejected our paper and said it was a very superficial paper, which is hilarious given what he did. So that to us did seem evidence that something was seriously wrong with the way he was treating papers. And we did complain and write to the integrity person in Wiley and eventually those papers were all retracted, I’m pleased to say. But yes, I also then around the same time did start looking at other journals. I mean, there are a number of people doing this and it’s painstaking work. You sort of could trawl through some of these journals. But there’s another way in which things are getting into less reputable journals, perhaps, although still journals that were listed in Clarabay and that’s via special issues. So some journals have realized that a good way to get a lot of submissions and if you’re charging people for submissions, this makes a lot of money, is to have a special issue. And so they advertised in the journal saying, would you like to edit a special issue? In fact, some of your listeners may have had emails from such journals saying, would you like to edit a special issue?
[00:12:00.010] – Dorothy Bishop
We’re very keen on this. And then it seems that they didn’t do sufficient quality control on the people who replied. And there were people in paper mills who clearly realized this was an amazing opportunity to get somebody in as an editor and then they can just accept whatever they like. And this was on a different scale to that previous case I mentioned. So there are a number of hindawi journals in particular, which is again a subgroup of Wiley, unfortunately, but they were really expanding the number of special issues massively. And they had articles in there that were even worse than the ones in the Journal of Community Psychology in the sense that they really made no sense. So I invented a term for some of them which was AI Gobbledygoop, because you would have a few paragraphs that made sort of sense but were just bit boring and not very you didn’t really know where it was going, but they were about something. In some cases, they were about something that had nothing to do with the journal, like Marxism in a Journal of Environmental Health. So you’d have your bit about Marxism, then you’d suddenly get in the middle a whole load of very, very technical artificial intelligence stuff, full of formulae and technical terms which didn’t relate to anything else very much as far as you could see, and then it would revert at the end to the more standard stuff.
[00:13:22.910] – Dorothy Bishop
And it was like a sandwich with this strange stuff in the middle that was you couldn’t necessarily with competence, say it was rubbish. But what you can often do is then track it to Wikipedia. It was often just lifted from Wikipedia. And these things were at scale. I mean, we’re talking about literally sometimes more than 100 such papers in a special issue with a particular editor. And so I started doing big analyses of those by really just combing the website of certain journals to see how many had these special issues with numerous papers who were the editors. And then looking at actually the response times between the submission to the journal.
[00:14:05.810] – Ludmila Nunes
I bet they were much faster than.
[00:14:08.340] – Dorothy Bishop
Usual, they tended to be very fast. And then I could tie that in with whether these papers had had comments on the website pub peer. Now, Pubier is a very useful post publication peer review website where anybody can put in a statement or comment on a paper. And of course there’s not huge quality control, although they do weed out completely crazy things or stuff that’s libel us. But basically a lot of people had started picking up on paper mills and making comments about things like the references bear no relationship to the paper, or the paper doesn’t bear any relationship to the topic of the journal, or picking up on duplication or plagiarized material and things like this. And so I could show that there were massive amounts of these Pub peer comments, particularly for the special issues, which had things accepted very, very fast. And it identified about 30 journals that were, I think, pretty problematic on the there’s no one indicator that something’s a paper mill, but when you get enough of these different indicators, you can start to be and particularly when it’s across a number of papers in the journal, you realize it’s a problem.
[00:15:22.420] – Dorothy Bishop
So rather to my surprise, I mean, I just had no idea how much of this stuff there was out there. But you could have a full time job investigating this because it’s getting worse, I think, and it’s going to get worse as artificial intelligence gets worse. And it’s easier to generate papers that are fake but that don’t look as crazy as some of the ones that I’ve been looking at.
[00:15:43.940] – Ludmila Nunes
That’s exactly what I was going to ask you, if you think that the amount of these fake papers is increasing and what role artificial intelligence might play in masking them better and making them harder to detect.
[00:16:01.000] – Dorothy Bishop
Yeah, I think it’s very frightening because we’ve just recently, I mean, there’s been such a lot of excitement just in the past couple of months about the new AI systems that are coming out that can generate people are worried about student essays and things. Anybody can cheat, and it’s very hard to pick it up. That’s going to apply as well to academic articles. And previously, some of the attempts to use AI have led to very unintentionally hilarious consequences because what they were trying to do, they were plagiarizing stuff and then running it through some AI that put it through a sort of thesaurus to change some of the words because they were trying to avoid plagiarism detectors. And that sometimes led to very strange turns of phrase, particularly in statistical things. So that instead of the sort of standard error of the mean, it would be the standard blunder of the mean. If you know of statistics, this is bad, it’s not what is called. And then breast cancer becomes bosom peril. And there’s a guy and team, in fact a guy running a little team of people in France who have been gathering examples of these and then doing the opposite.
[00:17:06.830] – Dorothy Bishop
They’re actually running articles, picking up these, what they call tortured phrases to identify paper mill products. And that’s a very good way at the moment of identifying them. But of course, I doubt it will have much longevity because the odds are that as soon as the paper mill people realize that you can do that, they’ll move it to something else and they’ll stop using and indeed, that they’ll probably no longer find it necessary to use that system. But I have to say it causes a lot of merriment in looking at some of the things that people put into papers when they don’t know what the words actually mean.
[00:17:41.410] – Ludmila Nunes
Okay, so we’ve been talking about this way of cheating and getting articles published, articles that aren’t real. But who is benefiting from these?
[00:17:52.550] – Dorothy Bishop
Very good question. Lots of people are benefiting, unfortunately. The sad reality is that there are a number of countries where you are required to have publications to progress in your career. And the earlier paper mills were from China, where in the medical profession, if you wanted to be a hospital doctor and you weren’t interested in doing research, you nevertheless had to have a publication to, I think, advance to be the next level consultant or whatever. And so, of course, the motivation for people to buy such a publication was really high and in fact, I think in some cultures people don’t regard it as doing anything wrong to do this. So there’s a sort of whole cultural thing that well, of course you do this because this is something you’re required to do and there’s no way you’re really going to generate a publication. The Chinese academic system, I think, is beginning to realize that they have a massive problem and they’re trying now to take measures to prevent this. But it’s been going on and a lot of these ones in the Hindawi journals that I mentioned do come from China and there are other countries too where there is this sort of system.
[00:19:02.940] – Dorothy Bishop
So unfortunately, Kazakhstan, Ukraine also to some extent, Iran. So they tend to be countries where the pressures on people who want to advance are extreme and the resources are perhaps not so good. And sometimes you can trace the paper mills, unfortunately, to people in senior positions who’ve found this is a good money making enterprise. So once you get corruption in the system, unfortunately it really can encourage this. Now, of course, the people that make huge sums of money are the people who are selling the papers and they are just straightforward crooks, but they are making huge amounts of money. If you’re producing these things and you’re charging people $1,000 per paper published in just one journal you’ve got perhaps 100 special issues each with 50 odd papers in. You can do the sums and start to realize that the income is huge. And then the publishers you see that there’s been a lot of criticism of the idea that the publishers may have been complicit because they’re making so much money from the article processing charges. But for them, the chickens have now come home to roost just recently in a very big way because Clarivate has delisted a whole load of Hindawi journals for exactly this.
[00:20:24.090] – Dorothy Bishop
So I think what they say is they’ve been using their own again, everybody’s using AI, but they’ve been using their own developed AI systems to try and spot paper mill products and have identified patterns that are clearly abnormal in some of these journals and realized that it’s doing a lot of harm to have them in the main scientific literature. And that’s very damaging to the publisher whose profits, in fact, visibly have sunk. People have been documenting how much they’ve lost. So ultimately, I think people say, oh, well, the publishers went into it for the money but I think they must know that it’s not in their interest to get a very bad reputation for publishing rubbish. I think the problem has been not so much that they’ve been doing it on purpose, but that they’ve taken their eye off the ball and they have really not paid sufficient attention to the importance of vetting editors. The editor has a role as a gatekeeper, which is very important. And I think editors have just been assumed that anybody can do it and you don’t really need much in the way of qualifications or background. And in fact you obviously do need people not only with some publications but with integrity and some sort of track record and you know, who are doing it for the right reasons.
[00:21:40.850] – Dorothy Bishop
And I’m afraid that you’ve got the impression that almost anybody would be accepted as an editor for a period during sort of 2021, 2022.
[00:21:51.570] – Ludmila Nunes
So this is a problem, of course, because it’s an ethical problem, but it’s also a scientific problem because having these articles around can impair the development of science.
[00:22:05.040] – Dorothy Bishop
Absolutely.
[00:22:06.330] – Ludmila Nunes
And I’m thinking, for example, undergrad students or very young students who are not so used to read scientific articles and are not used to scrutinize the articles can get the wrong idea and think that this research is actually good research.
[00:22:26.590] – Dorothy Bishop
Another thing to bear in mind is that we’re now in the world of big data and there’s an awful lot of research is now involving metaanalyses or in the biological sphere, massive sort of big data things where they are pulling in from the internet lots. I mean, it feels like cancer biology, which are plagued by paper mills. A lot of people’s research involves automating processes for trawling the literature to find studies that work on a particular gene or a particular compound of some sort and then just sort of putting them all in a big database of associations between genes and phenotypes. Now, if this stuff gets in there, you can imagine it plays havoc with the people who then want to use those databases for, say, drug development. So I don’t think there’s probably quite a parallel for that in psychology. But that’s the kind of worry that one has and even the really rubbishy ones. I mean, if you’re trying to do a metaanalysis and it throws up loads and loads of papers, you have to check each of those papers, you’re just wasting your time. Even if you come to the conclusion that it’s a rubbish paper, you can’t include it’s clogging up people’s attention span and clogging up the literature.
[00:23:36.300] – Dorothy Bishop
And as you say, some people may also not have the ability to tell that it’s rubbish. But even those of us that do, you still have to read it long enough to tell that it’s rubbish.
[00:23:46.010] – Ludmila Nunes
It’s still a waste of our time. And as you mentioned in many cases now if we are doing a meta analysis, we can just scrub the data and pull that from the articles. And if we are trusting that the sources are good sources, normal journals, these articles just make their way into a meta analysis.
[00:24:04.270] – Dorothy Bishop
For example, I mean, the other victims of this I would say, which is rather indirect, but you could argue that the victims are the honest people in the science. And I have had emails from clearly honest people from Iran, from China who are trying to report to me because I’ve somehow now identified as somebody who picks up on this stuff, colleagues who are doing this. And quite often, as I said before, these are quite senior people that might be doing this. And they’re desperate for it to be stopped because from their perspective, it’s awful because A, they can’t get on if they’ve got a boss who’s doing this and B, their entire country and all the research coming out of that country then gets denigrated as being flawed in this way. And it must be awful to be somebody who’s an honest scientist working in an environment where this stuff is happening at scale.
[00:24:57.790] – Ludmila Nunes
Exactly. We mentioned paper mills in China and of course we might start paying more attention to an article that comes from Chinese authors and trying to scrutinize, is this a real paper? Is this a fake paper? And that’s very unfair for the serious researchers, for people who are actually doing their job. But this issue also has repercussions for the general public and can contribute to the mistrust in science, which is already a problem.
[00:25:30.650] – Dorothy Bishop
Yeah. And I mean, some people then say, well, we shouldn’t talk about it because it will impair public trust in science. But I have the opposite view, which is that we should talk about it and show that we’re tackling it because we can’t expect people to trust us. If this sort of stuff is going on. We have to show that we can deal with it and that we take it seriously because unless we show that it really matters to us and we’re going to take it on, we don’t really deserve the trust of the public.
[00:26:00.710] – Ludmila Nunes
Exactly. I completely agree with you. I think hiding the problem is not going to help science and it’s not going to help the public to trust scientists at all.
[00:26:11.350] – Dorothy Bishop
But it’s a real problem that when the trust is eroded, because we’ve seen that very much with COVID that people do reach the point where they don’t know what to believe and they know there’s some misinformation out there and they know there’s some good information out there. If peer review is not, or supposed peer review is not a reliable signal anymore as to what is more trustworthy, then it’s very, very difficult because none of us can be experts in all these areas that are involved in judging whether things like a vaccine is effective or whether a disease is associated with symptoms. And we have to have some mechanism whereby we know that certain types of work are trustworthy. So it’s very, very important to keep this from polluting the journals and the sources that should be trustworthy.
[00:26:57.830] – Ludmila Nunes
So just to summarize to our listeners, are there any and I know you’ve already mentioned some, but which strategies can we use to evaluate an article? And I’m thinking mostly of the general public, if they find this random article online? Can they identify if this is a real article or if it might come from a paper mill?
[00:27:23.870] – Dorothy Bishop
Well, I think it’s very difficult. There are a few things that are obvious, like the tortured phrases. I mean, if you have a paper on Parkinson’s disease and instead of talking about Parkinson’s disease, it talks about Parkinson’s ailment or Parkinson’s malady, there’s quite a lot of literature that does that. It’s important to be aware that’s very unlikely to be just because somebody’s not got English as their first language. Because even if you don’t have that, you’ll be reading the literature. It’s all about Parkinson’s disease. So that sort of substitution suggests that something fishy has gone on and somebody’s deliberately trying to obscure the fact that work is plagiarized. So torture phrases is one good clue. A very rapid turnaround time, which isn’t always reported in a journal, but is sometimes reported in a journal, I regard as suspicious. I mean, you could just get lucky. This is probably it’s not 100% watertime. You could submit your paper and it’s immediately sent out for review and people immediately agree to review it. But anybody who has published papers knows that the usual thing is that it hangs around a bit before they can find reviewers and then the reviewers sit on it for a bit.
[00:28:32.680] – Dorothy Bishop
So we were talking in the Hindawi journals I was looking at, there were journals, special issues that were reliably managing to turn things around in two weeks from receipt to the first response. And that, I think is suspicious if it’s as fast as that or if there’s no requirement for revision in that time, if it goes in fact, not just receipt, but actual publication is really, really fast. So it’s not 100% watertight. But if I already had suspicions and I saw that, I’d think that was the case. The website pub peer is worth. I mean, you can check any article on Pub Peer and see if anybody’s commented on it. But of course it’s again, not 100%. Sometimes comments are not accurate, sometimes bad articles nobody’s picked up on. But you can then check back yourself with the paper and see, oh yes, that’s actually true, that that graph is copied, say, from another paper, or those numbers don’t add up right. Things like that will be reported on Pub here. So that’s another thing that you can check. But I think my general view is that we should go more for prevention than detection.
[00:29:44.080] – Dorothy Bishop
Again, a bit like with a virus. I mean, if you detect it, you want to do something about it. But the way we do science could be modified to make it much less easy for the paper mill people to operate. And here, most of the sort of methods of open science are quite effective in helping defend against it. There’s a very good cancer biologist, Jennifer Byrne, who’s argued for this in her field where. She’s picked up a lot of these really believable paper mill papers which have quite subtle errors in DNA sequences and things that you need to be really expert to pick out. But she said, well, if people had to pre register their papers or their studies and had to put in just as you do with clinical trials, you had to have a registry where before you did the study, you had to have protocol of what you were going to do. It wouldn’t be possible for people to do as much of this because the average study takes about a year to do. So you couldn’t rapidly be churning these things out. And if you have open data, that helps enormously because typically a paper mill paper will say that the data are available, but if you ask for it, it’s not.
[00:30:54.470] – Dorothy Bishop
And it’s often more trouble than it’s worth to generate fake data. So that’s not so common that they’ll do that. And open scripts. How did you analyze the data? An open peer review has proved invaluable. So you don’t necessarily have to have the names of the peer reviewers, but if you can just see the peer review, you can typically tell whether it’s normal peer review of a kind that engages with the content of the paper or whether it’s this very superficial stuff, which is clearly just hand waving.
[00:31:26.570] – Ludmila Nunes
So again, open science practices, creating a more transparent science can help this type of industrial fraud to be counteracted.
[00:31:36.550] – Dorothy Bishop
We can make it harder. I mean, the other way is, of course, as I said before, for the publishers to be very strict and to scrutinize editors better. But I think that you can make it just more trouble than it’s worth to generate a plausible fraudulent paper if you have stricter requirements of what people need to do to get published.
[00:31:57.270] – Ludmila Nunes
This is Ludmila Nunes with APS and I’ve been speaking to Dorothy Bishop from Oxford University. It was great having you and thank you so much for this stimulating conversation and discussing ways to improve our science.
[00:32:13.950] – Dorothy Bishop
Well, thank you very much for having me. It’s been a pleasure.
[00:32:16.990] – Ludmila Nunes
For more interesting research in psychological science, visit psychologicalscience.org.