Saturday, October 5, 2013

Unscientific spoof paper accepted by 157 "black sheep" open access journals - but the Bohannon study has severe flaws itself

This week, the journal Science has published a news article about the apparent lack of proper peer-review at many open access journals.
The author, contributing Science reporter John Bohannon, concocted a spoof paper with scientific problems, which ended up to be accepted in 157 out of 304 open access journals.
As the editor and publisher of a leading open access journal (Journal of Medical Internet Research JMIR - not to be confused with JIMR = Journal of International Medical Research, published by SAGE which accepted the spoof paper), which is ranked #1 by impact factor in its Thomson Reuters category (ahead of journals published by Elsevier, BMJ, etc.), I too was a target of this spoof, having received several emails from this fictitious author trying to bribe get me to accept his article. For the record: I didn't even send it out for peer-review as it was out of scope. Oddly enough, these emails and my journal don't even show up in Bohannon's data appendix and in his denominator. Also, Bohannon writes he excluded journals which charge a submission fee, but yet, his data appendix of 314 lists several journals that do charge submission fees. If they were excluded, why do they show up in his data appendix? Was our journal excluded, even though he did send me the paper by email and tried to convince me to accept it by paying money?
Bohannon's data are inconsistent, and this article would not have survived peer-review, not only because it is scientifically unsound (making sweeping conclusions without a control group, and sampling a non-random sample of questionable publishers) and ethically questionable (he never even got back to me apologizing for the spoof or even disclosing that the paper he sent me was a spoof), but also because of obvious inconsistencies in the data. Ironically, this paper is about lack of / failure of peer-review, yet this "study" itself is not peer-reviewed (and it would have been unpublishable in a serious, peer-reviewed journal). Despite circumventing peer-review, it is widely disseminated through press-releases (bypassing the usual "scientific" process).
While I appreciate that this study generated a list of open access publishers which have low-quality or no peer-review (see below), it should also be said that the overarching implied conclusion - that open access as a business model is flawed, or that OA journals are of generally lower quality than subscription journals, is outrageous.
What bothers me most is that we can't even refute/replicate this study (by also sending the spoof paper to subscription journals) as no ethics board in the world would approve such a blatantly unethical "study" (using deception and wasting the time/resources of hundreds of journals and academics), so it remains what it is - a piece of bad, sensationalist journalism, unfortunately published in a journal called "Science", implying a scientific study.

Among other problems described above, Bohannon fails to point out that ...

1) legitimate Open Access publishers are the main victims of the scammers and fraudsters that call itself OA publishers but are nothing more than criminal organizations.

2) The "journals" tested here are largely criminal organizations from Beall's list (this is a list if questionable OA publishers. Long before Beall, I started my own list in 2008 by blogging about questionable/unethical publishers like Bentham or Dove, which sure enough also ended up accepting the spoof paper). Anybody who wants to have a list of legitimate OA publishers can easily consult the list of OASPA members (Open Access Scholarly Publishers Association), an organization I co-founded out of the exact same concerns around quality. While I think OASPA can and should do more to ensure quality of its members, I do think being an OASPA member is a pretty good predictor for being a legitimate publisher. I hope however that OASPA is taking swift action against some of its members which published the paper, including Dove and Sage (I am not involved in the membership committee, and unfortunately, Dove recently became a member of OASPA, despite my previous concerns. I would advocate a 2 year membership suspension, after which the OASPA membership committee should submit a test paper to see if the peer-review process has improved).

3) well-known problems with peer-review (reviewers and editors missing critical errors in a manuscript) are not a problem of OA journals specifically. Spoof papers, fraudulent or poor papers, or pharma-sponsored papers have all been accepted by subscription-based journals before.

4) the article implies all "author-pays" OA journals have a conflict of interest by making more money if they accept more articles, ignoring the fact that the simple solution would be institutional memberships with OA journals or submission fees (payable regardless of the outcome). At JMIR Publications, we have experimented with these business models and hope that the scientific community will be increasingly open to such innovative models (we wish that more universities and departments would become institutional members supporting the work of JMIR - institutional membership leads to an article fee waiver on a per-paper basis). There are many possible business models of gold OA beyond "pay on acceptance".

It is foolish to extrapolate these findings of a few black sheep publishers and scammers (mainly based in Nigeria and India, whose journals are generally neither in PubMed nor have an impact factor) to an entire industry. This would be as logical as concluding from Nigerian wire fraud emails that all lawyers who take a fee-for-service are scammers!
There are only two well-known publishers on this list - Sage and Elsevier (the largest publisher of subscription based journals), and it is not without irony that these publishers are primarily publishing toll-access journals (see Appendix).

Gunther Eysenbach MD MPH FACMI
Editor/Publisher, JMIR Publications


Screenshot from http://scicomm.scimagdev.org/ showing where most of the bank accounts of the seemingly "International" or "American" journals that accepted the spoof paper lead to - the vast majority being in India and Nigeria.

Update 7-October-2013: 

I made our email exchange with the fake author aka Bohannon available online. JMIR has a submission fee which needs to be paid at the end of the submission process (otherwise the submission is flagged as "incomplete" and the editor doesn't even see it in the submission queue), but we do have a waiver policy for authors from developing countries. I am not arguing here whether or not JMIR or its sister journals like i-JMR (which has a broader scope) should have been included or not in the sample, but the fact that neither JMIR nor the email exchange shows up in the data appendix to the Bohannon study raises several issues and questions:

  1. If JMIR is not listed in the spreadsheet, and the emails are also not included in the data appendix, what else is missing from the data provided by Bohannon? Were there any other journal editors contacted?
  2. If I would have accepted the paper sent to me by email, waiving the submission fee on grounds of being from an author from a developing country, would JMIR have been suddenly included in the list of journals which accepted the paper?
  3. It seems a bit like the inclusion/exclusion criteria were not set a-priori but were made up as he went along (which is a rather unscientific approach). It is clear from our website and submission process that we charge a submission fee, why did he attempt to submit the paper anyways? And why does he later write that these journals were excluded?
  4. There is a bit of an ethical issue here if he submits a flawed paper by email and communicates with me under a false identity, and never debriefs me saying that this was part of an investigation. In fact, any Ethics Board would probably require that a study involving fraud/deception at least debriefs study participants (some may even require retrospective informed consent).   

APPENDIX - Publishers and throw-away/crap journals which have accepted the spoof paper

(Source: Data Appendix from Science 4 October 2013:  vol. 342 no. 6154 pp. 60-65  DOI: 10.1126/science.342.6154.60 http://www.sciencemag.org/content/342/6154/60/suppl/DC1)
AVOID THESE PUBLISHERS/JOURNALS


publisher journal_URL journal_name
Scientific Journals http://www.scientific-journals.co.uk/index.php?p=1_12 Journal of Pharmaceuticals
International Scientific Publications http://www.scientific-publications.net/open-access-journals/ecology-and-safety/ Ecology & Safety
KEJA Publications http://www.kejapub.com/ijpbr/ International Journal of Pharmaceutical and Biological Research
Engineering and Technology Publishing http://www.jomb.org/ Journal of Medical and Bioengineering
International House for Academic Scientific Research http://www.ijamsar.iserp.org/ International Journal of Advanced Medical Sciences and Applied Research
Indian Society for Education and Environment http://iseeadyar.org/ijdad/archives.html Indian Journal of Drugs and Diseases
International Journals of Scientific Research http://ijscience.com/Advances_in_Biological_Sciences/default.htm Advances in Biological Sciences
Electronic Center for International Scientific Information http://ecisi.com/ijarr/ International Journal of Agriculture: Research and Review  
PharmaInterScience Publishers http://www.pharmainterscience.com International Journal of Pharmacy and Biomedical Sciences
Pharmaceutical Research Foundation http://pharmresfoundation.com/japr/ Journal of Advanced Pharmaceutical Research
Macrothink Institute http://www.macrothink.org/journal/index.php/jbls Journal of Biology and Life Science
World Academy of Science, Engineering and Technology http://www.waset.org/publications.php International Journal of Medical, Pharmaceutical, Biological, and Life Sciences
Signpost e Journals http://www.journals.academicpursuits.us/index.php/SOAJ_BPS Signpost Open Access Journal of Biomedical and Pharmaceutical Sciences
Internet Scientific Publications, LLC http://www.ispub.com/journal/the-internet-journal-of-herbal-and-plant-medicine The Internet Journal of Herbal and Plant Medicine
Bentham open http://www.benthamscience.com/open/tobcj/index.htm The Open Bioactive Compounds Journal
Scientific Research Publishing  http://www.scirp.org/journal/ojrad/ Open Journal of Radiology
JK Welfare & Pharmascope Foundation http://www.ijrps.pharmascope.org/ International Journal of Research in Pharmaceutical Sciences
Innovative Scientific Information & Services Network  http://www.isisn.org/Biosciece_Research.htm Bioscience Research
Scienpress Ltd http://www.scienpress.com/journal_focus.asp?Main_Id=73 International Journal of Health Research and Innovation
Academic Research Publishing Agency http://www.arpapress.com/jpcs/index.aspx Journal of Pharmacy and Clinical Sciences
Center for Enhancing Knowledge http://www.cekinfo.org.uk/EIJST European International Journal of Science and Technology
Asian Economic and Social Society http://www.aessweb.com/instructions-to-authors.php?id=5003 Journal of Asian Scientific Research
Segment Journals http://www.segmentjournals.com/ejournal.php?id=3 Biological Segment
International Network for Applied Sciences and Technology http://www.inast.org/jpbr.html Journal of Plant Biology Research
Science and Engineering Publishing Company http://www.seipub.org/acs/ Advances in Chemical Science
British Association of Academic Research http://www.bjmhs.baar.org.uk/index.php British Journal of Medical and Health Sciences
Global Journals, Inc. (US) https://globaljournals.org/GJMR/medical-research/menu-id-1250/ Global Journal of Medical Research
Medwell Online http://medwelljournals.com/journalhome.php?jid=1815-9362 Research Journal of Pharmacology
Electronic Center for International Scientific Information http://ecisi.com/ijarr/ International Journal of Agriculture: Research and Review
GKS Publishers http://www.researchinbiotechnology.com/index.php/rib Research in Biotechnology
eJournals of Academic Research & Reviews http://www.ejarr.com/ejbs/index.aspx eJournal of Biological Sciences
Narain Publishers Pvt. Ltd http://www.npplweb.com/wjsmro/ World Journal of Surgical Medical and Radiation Oncology
Biomedical Informatics Publishing Group http://www.bioinformation.net/journal/aboutus.htm Bioinformation
ScienceHuB http://www.scihub.org/AJMDS/index.html American Journal of Medical and Dental Sciences
International Journal of Advanced Technology and Engineering Research http://www.ijater.com/ International Journal of Advanced Technology and Engineering Research
Maxwell Scientific Organization http://maxwellsci.com/jp/j2p.php?jid=AJMS Asian Journal of Medical Sciences
BioInfo http://www.bioinfo.in/contents.php?id=24&page=aim International Journal of Drug Discovery
African Journals Online http://www.ajol.info/index.php/ajbr African Journal of Biomedical Research
World Scientific and Engineering Academy and Society http://wseas.org/wseas/cms.action?id=4011 WSEAS Transactions on Biology and Biomedicine
Scientific & Academic Publishing http://www.sapub.org/journal/aimsandscope.aspx?journalid=1103 International Journal of Cancer and Tumor
Euresian Publications http://www.environmentaljournal.org Universal Journal of Environmental Research and Technology
Innovare Academic Sciences http://www.ijaponline.org/ International Journal of Applied Pharmaceutics
Radiance Research Academy http://www.ijcrr.com International Journal of Current Research and Review
Research and Review
OMICS Publishing Group http://www.omicsonline.org/medchemhome.php Medicinal Chemistry
Kobe University School of Medicine, Kobe http://www.med.kobe-u.ac.jp/journal/ Kobe Journal of Medical Sciences
Modern Scientific Press http://modernscientificpress.com/Journals/IJBioMed.aspx International Journal of Modern Biology and Medicine
Dove Medical Press Ltd http://www.dovepress.com/drug-design-development-and-therapy-journal Drug Design, Development and Therapy
Science Journal Publication http://www.sjpub.org/sjmct.html Science Journal of Medicine and Clinical Trials
Scientific Research Platform http://europeanjournalofscientificresearch.com/ European Journal of Scientific Research
Basic Research Journals http://basicresearchjournals.org/medicine/mmsh.html Journal of Medicine and Clinical Sciences
Victorquest Publications http://www.ijthmjournal.com International Journal of Traditional and Herbal Medicine
Independent Publishing house http://www.theprofesional.com Professional Medical Journal
Science & Knowledge Publishing Corporation Limited http://scik.org/index.php/ejbls/ European Journal of Biological and Life Sciences
International Research Journal http://www.ssmrae.com/review/hhome.php International Research and Review
Sage http://www.jimronline.net  Journal of International Medical Research
Deccan Pharma Journals http://www.deccanpharmajournals.com/index.php?option=com_content&view=article&id=16&Itemid=11 Deccan Journal of Medicinal Chemistry
World Science Publisher http://worldsciencepublisher.org/journals/index.php/JOS Journal of Science
InTech Open Access Publisher http://www.intechopen.com/journals/international_integrative_medicine International Journal of Integrative Medicine
Sphinx Knowledge House http://www.sphinxsai.com/framesphinxsai.htm International Journal of PharmTech Research
World Academic Publishing http://www.jlsmr.org/ International Journal of Life Science and Medical Research
Society of Education  http://www.soeagra.com/abr/abr.htm Advances in Bioresearch
Global Advanced Research Journals http://garj.org/garjmms/index.htm Global Advanced Research Journal of Medicine and Medical Sciences
Association of Pharm innovators http://www.jbiopharm.com/index.php/ajbps Asian Journal of Biomedical and Pharmaceutical Sciences
ARPN Journals http://www.arpnjournals.com/jabs/index.htm Journal of Agricultural and Biological Science
International Journal of Current Research http://www.journalcra.com/ International Journal of Current Research
Centre For Info Bio Technology http://www.cibtech.org/jms.htm International Journal of Basic and Applied Medical Sciences
e-journals http://www.shankargargh.net/index.php International Research Journal of Medical Sciences
Scholarly Journals International http://scholarly-journals.com/sjm/index.htm Scholarly Journal of Medicine
Science Park Journals http://scienceparkjournals.org/jam/about.htm Journal of Medicine and Radiology
International Journals of Multidisciplinary Research Academy http://www.ijmra.us/about_us.php International Journal of Engineering, Science and Mathematics
PBS Journals http://pbsjournals.com/ajmcs/ Asian Journal of Medical and Clinical Sciences
Trans Stellar http://www.tjprc.org/journals.php?jtype=2&id=51 International Journal of Medicine and Pharmaceutical Sciences
David Publishing http://www.davidpublishing.org/journals_info.asp?jId=1164 Journal of Life Sciences
International Journal of Life Sciences Biotechnology and Pharma Research http://www.ijlbpr.com/submitmanuscript.html International Journal of Life Sciences Biotechnology and Pharma Research
The Sims Institute Press Ltd  http://www.jexpclinassistreprod.com/ Journal of Experimental & Clinical Assisted Reproduction
Lifescience Global http://www.lifescienceglobal.com/journals/journal-of-cancer-research-updates Journal of Cancer Research Updates
International Scholars Journals http://internationalscholarsjournals.org/journal/ijocr International Journal of Oncology and Cancer Research
International Research Journals http://www.interesjournals.org/JMMS/index.htm Journal of Medicine and Medical Sciences
BioMedSciDirect Publications http://www.biomedscidirect.com/aboutjournal.php International Journal of Biological & Medical Research
Pelagia Research Library http://pelagiaresearchlibrary.com/der-pharmacia-sinica/index.html Der Pharmacia Sinica
Discovery Publishing Group http://www.discovery.org.in/md/Aim_Scope.htm Medical Science Journal
Wudpecker Research Journals http://www.wudpeckerresearchjournals.org/WJMP/Index.htm Wudpecker Journal of Medicinal Plants
Herald International Research Journals http://heraldjournals.org/hjmms/index.htm Herald Journal of Medicine and Medical Science
 Industry Research Collaboration Center http://airccse.org/journal/IJBB/ijbb.html International Journal on Bioinformatics & Biosciences
Cosmic Journals http://cosmicjournals.com/ijear.html International journal of Education and Applied Research
Pharmacie Globale http://pharmacie-globale.info/ International Journal of Comprehensive Pharmacy
CSCanada http://www.cscanada.net/index.php/ans Advances in Natural Science
Hikari Ltd. http://www.m-hikari.com/cems/index.html Clinical and Experimental Medical Sciences
International Journal of Pharma and Bio Sciences http://www.ijpbs.net International Journal of Pharma and Bio Sciences
Journal of Chemical, Biological and Physical Sciences http://www.jcbsc.org/ Journal of Chemical, Biological and Physical Sciences
International Digital Organization for Scientific Information http://idosi.org/gjp/gjp.htm Global Journal of Pharmacology
Open Access Science Research Publisher http://www.openaccessscience.com/index.php/journals/ijmap International Journal of Medicinal and Aromatic Plants
Association of Physiologist, Pharmacists and Pharmacologists (APPP) http://www.njppp.com/ National Journal of Physiology, Pharmacy and Pharmacology
Science Instinct Publications http://www.sciipub.com/index.php/forum/2-international-journal-of-pharmacology-and-toxicology-science International Journal of Pharmacology & Toxicology Science
Science Target http://www.sciencetarget.com/site/index.php/about-ijhmp International Journal of Herbs and Medicinal Plants
Society for Science and Nature http://scienceandnature.org/Journal_IJABR.php International Journal of Advanced Biological Research
Society of Applied Sciences http://www.ajebs.com/ Asian Journal of Experimental Biological Sciences
Erudite Journals Limited http://eruditejournals.org/ejmmsr/index.htm Erudite Journal of Medicine and Medical Science Research
Scholar Science Journals http://ijbr.ssjournals.com/index.php/journal International Journal of Biomedical Research
International Research Journals http://resjournals.com/ARJ/Index.htm Agricultural Science Research Journal
Greener Journals http://www.gjournals.org/GJMS/GJMS%20Home.html Greener Journal of Medical Sciences
Global Research Online http://www.globalresearchonline.net/ International Journal of Pharmaceutical Sciences Review and Research
PharmaInfo http://jpsr.pharmainfo.in/issue.php?page=37 Journal of Pharmaceutical Sciences and Research
The Ribeir√£o Preto foundation for Scientific Research (FUNPEC) http://www.geneticsmr.com/ Genetics and Molecular Research
International Association of Journals & Conferences http://www.iajc.org/journals.html International Transaction Journal of Engineering, Management, & Applied Sciences & Technologies
Photon Foundation https://sites.google.com/site/photonfoundationorganization/home/international-journal-of-medicinal-plants International Journal of Medicinal Plants
Praise Worthy Prize http://www.praiseworthyprize.com/IREBIC.htm International Review of Biophysical Chemistry
SAVAP International http://www.journals.savap.org.pk/journal.html Academic Research International
BioIT international Journals http://bipublication.com/IJPA-pharmacy_journals.html International Journal of Pharmaceutical Applications
Interscience Open Access Journals http://www.interscience.in/ijppt.html International Journal of Pharmacology and Pharmaceutical Technology
Advanced Research Journals http://www.arjournals.org/index.php/ijaps International Journal of Advances in Pharmaceutical Sciences
SJournals http://sjournals.com/index.php/SJMS/index Scientific Journal of Medical Science
International Institute for Science, Technology and Education http://www.iiste.org/Journals/index.php/ALST Advances in Life Science and Technology
Silicon Valley Publishers http://www.ijastnet.com/update/ International Journal of Applied Science and Technology
GVGS http://www.rjpbcs.com/ Research Journal of Pharmaceutical, Biological and Chemical Sciences
Online Research Journals http://onlineresearchjournals.org/JMMSR/index.htm Online Journal of Medicine and Medical Science Research
Wilolud Journals http://www.wiloludjournal.com/ojs/index.php/cjpharm Continental Journal of Pharmaceutical Sciences
e3Journals http://www.e3journals.org/journal.php?jid=3 Journal of Biotechnology and Pharmaceutical Research
International Journals of Engineering & Sciences http://www.ijens.org/ijbas.html International Journal of Basic & Applied Sciences
Global Research Journals http://www.globalresearchjournals.org/journal/jppr Journal of Pharmacy and Pharmacological Research
Pharmaceutical and Biological Society http://ajphs.com/ Asian Journal of Pharmaceutical and Health Sciences
World Academy of Research and Publication http://www.warponline.org/viewjc.php?id=j1 International Journal of Chemical and Environmental Engineering
Textroad Journals http://www.textroad.com/Pharmaceutical%20and%20Biomedical%20Sciences-Scope.html Journal of Pharmaceutical and Biomedical Sciences
Research Publisher http://researchpub.org/journal/jbpr/jbpr.html Journal of Biochemical and Pharmacological Research
Journal of Chemical and Pharmaceutical Research http://jocpr.com/ Journal of Chemical and Pharmaceutical Research
Power Control Optimization http://www.pcoglobal.com/gjto.htm Global Journal of Technology and Optimization
Medipoeia Publication http://www.japsonline.com/ Journal of Applied Pharmaceutical Science
Master Publishing Group http://www.ijbs.org/ International Journal of Biomedical Science
Bioflux http://www.aab.bioflux.com.ro/ Advances in Agriculture & Botanics
American Journal of PharmTech Research http://www.ajptr.com American Journal of PharmTech Research
The Faculty of Medicine, University of Nis http://wwwserver.medfak.ni.ac.rs/Acta%20Facultatis/Instructions.htm Acta Facultatis Medicae Naisensis
SERSC http://www.sersc.org/journals/IJBSBT/ International Journal of Bio-Science and Bio-Technology
Xinnovem Publishing Group http://www.bmjournal.in/journal.php Biomirror
Sciensage Publications http://www.sciensage.info/jasr Journal of Advanced Scientific Research
PharmTech Publications http://www.onlinepharmacytech.info/ Journal of Pharmaceutical Science and Technology
INSInet Publications http://www.ajbasweb.com/ Australian Journal of Basic and Applied Sciences
E Business Navigators  http://www.biolmedonline.com/ Biology and Medicine
Global Academic Society http://www.ijsr.in/abour_journal.php Indian journal of Scientific Research
Medknow Publications  http://www.jnatpharm.org/ Journal of Natural Pharmaceuticals
Elsevier http://ditonline.info/index Drug Invention Today
Bio Tech Systems http://www.btsjournals.com/Pages/default.aspx Journal of Biotech Research
OmniScientia http://www.jnrt.info/ Journal of Nephrology and Renal Transplantation
Global Researchers Journals http://www.grjournals.com/Default.aspx?tabid=6537 Journal of Physiology and Pharmacology Advances
Sooraj Publications http://www.ijcsr.co.in/ International Journal of Chemical Sciences and Research
Green Earth Research Foundation http://www.gerfbb.com/ GERF Bulletin of Bioscience
European Journal of Chemistry http://www.eurjchem.com/index.php/eurjchem/index European Journal of Chemistry
Hygeia http://www.hygeiajournal.com Hygeia
Trendz Publications http://www.itpsonline.net/index.php Journal of Innovative trends in Pharmaceutical Sciences
Indo Global Journal of Pharmaceutical Sciences http://www.iglobaljournal.com Indo Global Journal of Pharmaceutical Sciences
International Journal of Research in Pharmacy and Chemistry http://www.ijrpc.com International Journal of Research in Pharmacy and Chemistry
Pharma Science Monitor http://www.pharmasm.com Pharma Science Monitor
IJSAT Publishers http://www.ijsat.com International Journal of Science and Advanced Technology
Sadguru Publications http://www.sadgurupublications.com/Journal.aspx?PageID=19 Journal of Current Chemical & Pharmaceutical Sciences
 International Journal of Chemical and Pharmaceutical Sciences  http://www.ijcps.com/ International Journal of Chemical and Pharmaceutical Sciences
IRPN http://www.ejournalofsciences.org/ International Journal of Science and Technology
University Ss Kiril and Metodij http://www.mjms.ukim.edu.mk/ Macedonian Journal of Medical Sciences
International Journal of Research in Pharmacy and Science  http://www.ijrpsonline.com/ International Journal of Research in Pharmacy and Science
Journal of Medical Research http://valleys.co.in/ Journal of Medical Research

10 comments:

Karen Shashok said...

Thanks for a very illuminating perspective on this notorious incident.

“I too was a victim of this spoof, having received several emails from this fictitious author trying to bribe me to accept his article. For the record: I didn’t even send it out for peer-review as it was out of scope. Oddly enough, these emails and my journal don’t even show up in Bohannon’s data appendix and in his denominator. Also, Bohannon writes he excluded journals which charge a submission fee, but yet, his data appendix of 314 lists several journals that do charge submission fees. If they were excluded, why do they show up in his data appendix? Was our journal excluded, even though he did send me the paper by email and tried to convince me to accept it by paying money?”

This raises serious questions about Bohannon’s data that should be answered. Could you post your correspondence with Bohannon? Is it possible that Journal of Medical Internet Research was included initially but later excluded as a result of the changes Bohannon made along the way in how he selected journals for his final sample? Comparing the dates of your correspondence with him against the dates in the emails made available as Data and Documents (Emails, zip file) would provide the answer.

If the data were reported selectively, or if the inclusion criteria were not applied consistently, this would by an additional and very serious problem with the article.

Bjoern Brembs said...

This is very interesting news!
Would it be possible to publish your exchange with Bohannan such that the community can confront him with it?

Many thanks,

Bjoern Brembs

Gunther Eysenbach MD MPH said...

I made our email exchange with the fake author aka Bohannon available online (see link above). Hopefully Bohannon or Science can answer the following questions:
(1) If JMIR is not listed in the spreadsheet, and the emails are also not included in the data appendix, what else is missing from the data provided by Bohannon? Were there any other journal editors contacted?
(2) If I would have accepted the paper sent to me by email, waiving the submission fee on grounds of being from an author from a developing country, would JMIR have been suddenly included in the list of journals which accepted the paper?
(3) It seems a bit like the inclusion/exclusion criteria were not set a-priori but were made up as he went along (which is a rather unscientific approach). It is clear from our website and submission process that we charge a submission fee, why did he attempt to submit the paper anyways? And why does he later write that these journals were excluded?
(4) Isn't there a bit of an ethical issue here if he submits a flawed paper by email and communicates with me under a false identity, and never debriefs me saying that this was part of an investigation. In fact, any Ethics Board would probably require that a study involving fraud/deception at least debriefs study participants (some may even require retrospective informed consent).

Anonymous said...

Dear Dr. Eysenbach-

The emails are in the data. Look at folder 293 in emails.zip here:

http://www.sciencemag.org/content/342/6154/60/suppl/DC1

You are right that your journal (ID 293) is not listed in the summary data spreadsheet. I added the fee-charging journals into the spreadsheet the data before posting the data. Thank you for pointing that out. I will update the data so that it is clearly listed.

To be clear: There was no attempt to hide the fact that a paper was submitted to your journal. The submission-fee journals were excluded from the analysis, and it was a last-minute decision to include those emails at all.

Sincerely,
John Bohannon

Anonymous said...


Dear Dr. Eysenbach-

Here are my answers to your questions:

1. If JMIR is not listed in the spreadsheet, and the emails are also not included in the data appendix, what else is missing from the data provided by Bohannon? Were there any other journal editors contacted?

Thanks to your help, I have now found several records that were missing from the data. All of them were journals that were excluded from the analysis either because they required a fee for submission or I discovered during the editing process that they were in fact journals with publisher to which I had already submitted a paper. (Both of the 2 duplicates resulted in acceptances, twice, nonetheless.) Here is the change log that I will be including with the updated data:

CHANGE LOG (7 October 2013)

The following records were added:
187 (submission fee required)
254 (duplicate submission, same publisher as 488)
293 (submission fee required)
398 (duplicate submission, same publisher as 358)

The following records were removed:
85 (no paper submitted)

The following records were modified:
89 (correct name and URL of publisher now identified)

NOTE: None of the above changes affect the analysis as they do not change the tally of acceptances or rejections.

2. If I would have accepted the paper sent to me by email, waiving the submission fee on grounds of being from an author from a developing country, would JMIR have been suddenly included in the list of journals which accepted the paper?

Only if you did ultimately accept the paper. If on the other hand you rejected the paper, you would have been included among the rejections. Instead, I was not able to submit the paper because of the submission fee, so it was excluded from analysis.

3. It seems a bit like the inclusion/exclusion criteria were not set a-priori but were made up as he went along (which is a rather unscientific approach). It is clear from our website and submission process that we charge a submission fee, why did he attempt to submit the paper anyways? And why does he later write that these journals were excluded?

It is not true that "the inclusion/exclusion criteria were not set a-priori but were made up as he went along". In the article I spell out my criteria clearly.

4. There is a bit of an ethical issue here if he submits a flawed paper by email and communicates with me under a false identity, and never debriefs me saying that this was part of an investigation. In fact, any Ethics Board would probably require that a study involving fraud/deception at least debriefs study participants (some may even require retrospective informed consent).

This is a legitimate issue for debate and it is addressed by a bioethicist named Dan Wikler in the comments section of Peter Suber's online discussion here:
https://plus.google.com/109377556796183035206/posts/CRHeCAtQqGq

Sincerely,
John Bohannon

Christophe Laurent said...

Dear Gunther,
I'm impressed by the way you handled this. I am very fond of the whole Open Acces movement, and openness in most public matters too, for that matter. I think this (call it the #JMIR angle if you want) should be also opened up to the public. I understand it might not be the galant thing to do from the point of view of an editor per se. But I think there must be a way to dissect this properly, and openly towards the "bigger" public. E.g. Also the casual readers of that came across any rendering of this strange experiment. In short: also this odyssey should be "open access".
Doesn't all this kind of seem you strange? It may be a long shot. But I wonder if there's no one behind that study that has any intrrests in traditional publishing. (I'm trying to establish motive for such an elaborate enterprise: this was a lot of letters and man-hours. Who payed for this??)
But I repeat: this odyssey should be "open access".

Karen Shashok said...

Thanks very much Gunther and John for clarifying a few potential issues with the data set and methods. The study has flaws but at least post-publication peer review seems to be working.

"Was our journal excluded, even though he did send me the paper by email and tried to convince me to accept it by paying money?"

Gunther, in the correspondence you uploaded I see no evidence in the text of the emails that the fake authors tried to convince you to accept the manuscript. This is not to excuse Science and Bohannon for the many worrisome issues that surround their ill-conceived experiment, but simply to say that how readers interpret things can vary depending on a lot of assumptions readers make about the writer.

As a comment by Renee on Retraction Watch noted http://retractionwatch.wordpress.com/2013/10/03/science-reporter-spoofs-hundreds-of-journals-with-a-fake-paper/, the email correspondence between the fake authors and the journals constitute an interesting source of data for an anthropological (or linguistic anthropology) case study.

David Ross said...

I would like to point out that the Journal of International Medical Research (JIMR), published by SAGE, did not accept the article submitted to it for publication as it stands. It was accepted subject to further editorial queries being answered to the satisfaction of the journal. This is clearly laid out in the correspondence with the author.

JIMR is a long-established, respected journal that has been publishing quality research for 41 years. It is JCR-listed and its rejection rate was 62.5% in 2012. Professor Malcolm Lader Ph.D., M.D., F.R.C. Psych., F. Med. Sci., Emeritus Professor, King’s College London is the Editor of the journal and has been for the last 25 years. It has a world class Editorial Board.

JIMR has operated a two-stage review process, specific to this journal. First, the editor performs an initial review of a submission and then, once a paper has passed this initial review stage, it is accepted subject to a detailed technical edit, which is undertaken by at least two experienced medical technical editors. It is not uncommon for articles to be rejected at this stage.

We are extremely concerned that a paper with fundamental errors got through the initial stage, and are taking steps to ensure that it cannot occur again, but are confident that the technical edit would have revealed the errors.

SAGE is committed to ensuring that the peer review and acceptance process for all our journals, whether traditional subscription-based or Open Access, is robust.

A fuller statement is posted on the SAGE website:
http://tinyurl.com/jwldtgm.

David Ross
Executive Publisher, Open Access
SAGE

Gunther Eysenbach MD MPH said...

@Karen: Different people may read the emails differently but as an editor I found it strange to get 3 emails from the authors pushing for a decision (after I already explained that there is a submission fee) and dangling the money in front of my nose ("Our university grant gives us access to the money to pay publication fees only upon proof of acceptance of our work. Once our paper passes peer review and is acceptable for publication, then we get access to the money.") - I felt it it's basically like saying, "come on, accept it already, then we pay you".

@all
In any case, I give credit to John Bohannon for correcting the data, as well as answering my questions, even though some issues remain unanswered - it is still not clear why he tried to submit an article when it was clear we have a submission fee, and the ethical issue is up for debate. I still think editors should at least have received an email from Science with an apology and the disclosure that this paper was a hoax.

Anonymous said...

Thanks for sharing your perspective. This "sting" looks kind of pranky, identifying a problem and then overattributing it. Another thing that should be pointed out is that Bohannon hasn't proven much by simply getting a paper published in a no-review or weak-review bogus journal. It does NOT mean that the OA system allows bad science to permeate. What allows bad science to permeate is when other scientists start citing bad work. So garbage papers in bogus journals don't threaten the system, and Bohannon is wrong to advance this premise. He'd have to see if anyone actually cited his spoof, and built new hypotheses from it. Just getting the spoof paper published is very early the larger scientific process.

Given the huge profits at for-profit "real" publishers (Springer, etc), all built upon publicly-outsourced overhead costs, OA needs to be part of the future of science.