He Main Reason That the Royal Society of London Developed the Modern Form of Peer Review Was to

Peer review, the procedure by which articles and applications are vetted by experts in a researcher's field, is a central function of scholarly publishing and grants in fields from physics to classics. For many scientists, information technology is the crucial hurdle to articulate earlier publishing new research, one that is meant to provide a measure of validation and quality control. Indeed, it is often considered one of the markers of objective scientific inquiry. Peer review is and then central to the fashion science works that many observers presume that, like modernistic science, the system had its origins in the Scientific Revolution of the seventeenth century. That turns out non to be the case. In fact, the "referee system" did not exist earlier the nineteenth century and the term "peer review" was a creation of the late twentieth century. Furthermore, systems of review were not intended to be a mark of scientific legitimacy; that expectation merely arose during the Cold War. Peer review's history, similar its hereafter, is contested equally scientists keep to argue over its place in the scientific enterprise.

The prehistory of peer review

The origins of peer review are usually explained with an highly-seasoned, but mostly incorrect, myth. In 1665, the Royal Guild of London gave Henry Oldenburg (1619–1667) permission to compile the Philosophical Transactions of the Royal Society, a new periodical that would collect of import scientific findings and news. Oldenburg wisely saw that he would demand expert opinions to decide which of the bachelor papers he should publish, and thus the good referee has had a central office in scientific publishing ever since the founding of the first scientific periodical.

Epitome 1: Philosophical Transactions of the Royal Society. Source: Archive of the Royal Society, used under the Creative Eatables Attribution 4.0 International license.

That origin story appears to take arisen from a 1971 sociology paper, "Patterns of Evaluation in Science: Institutionalization, Structure and Functions of the Referee System" past Harriet Zuckerman and Robert Merton.¹ Zuckerman and Merton's commodity about refereeing in the sciences was widely read and extremely influential, and their account of peer review's history has been repeated in many scholarly books and papers since the 1970s.ⁱⁱ When historians such as Adrian Johns and Aileen Fyfe took a closer look at the early on Philosophical Transactions, however, it became articulate that Zuckerman and Merton had misunderstood or misdated some of Oldenburg'due south correspondence, and that Oldenburg did not in fact employ whatsoever system at the Philosophical Transactions that could reasonably be compared to modern refereeing.^three Although the members of the Regal Society were nominally tasked with reviewing the contents of the Transactions, in practice Oldenburg maintained such tight command over the periodical that he was sometimes referred to as its "author" instead of its editor.⁴

In contempo years historians have re-analyzed the historical development of specialist scientific journals, and it has become clear that neither peer review nor the scientific journal sprang total-formed from the seventeenth century, equally the Zuckerman-Merton story would have information technology.⁵ Throughout the eighteenth century and well into the nineteenth, journal articles were just one mode amongst many for researchers to communicate recent scientific findings. Monographs, literary essays, public presentations, and pamphlets were all considered valid methods of disseminating scientific knowledge—and very few works in those formats went through anything like external refereeing.

Pre-publication review was most common at scientific societies, where members were anxious to ensure that anything issued with the society's name on it did credit to the gild as a whole. For case, all publications issued by the Académie des Sciences in Paris had to be vetted by the author's colleagues in the Académie.^{half dozen} In 1731, the Regal Society of Edinburgh informed readers of its Medical Observations and Essays that

Memoirs sent by correspondence are distributed co-ordinate to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author. Nothing is printed in this review which is non stamped with the marking of utility.⁷

Those systems of internal review and critique, nevertheless, exercise non seem to map onto or presage the external refereeing organization that we know today. Aileen Fyfe and Noah Moxham take called this period the "pre-history" of peer review—a useful term that reminds us we shouldn't conflate formal refereeing systems with the more than full general idea of having scholars comment on each others' work.⁸ The purpose of those older systems was to safeguard the reputation of the lodge affiliated with the publication. Internal discussion created a communal sense of investment and responsibility in a project, and there was no expectation of anonymity for anyone involved in the procedure. Many of those older systems of internal critique were as well meant to ensure that cipher seditious or heretical was printed under a order's name. Furthermore, most vetting within a learned gild would have been somewhat informal and haphazard; a member of sufficient standing would expect that his work would be canonical by his colleagues with minimal fuss. (The gendered linguistic communication hither is intentional, equally well-nigh early learned societies refused to accept women as members.)

The concept of having peers evaluate each others' work, in other words, is far from new. But it would be a mistake to await back at informal, unsystematic critiques within elite learned societies and equate them with the modern system of peer review. In purpose and in class, that kind of internal vetting was very different. For systematic review by anonymous experts who are supposed to human activity equally gatekeepers, we need to look to the nineteenth century.

Referees: From adviser to gatekeeper

The origins of systemic external reviewing are tightly linked to the nineteenth century transformation of the academic periodical into the dominant form of scientific communication.⁹ Around 1800, a physicist or naturalist could choose to announce research results through talks at scientific meetings, academy lectures, newspaper articles, pamphlets, essays for full general-involvement literary magazines, and monographs—and all of those methods would be considered valid ways to both communicate their findings and establish priority for their ideas. By 1900, that was no longer the case; researchers were expected to publish their findings in a reputable scientific journal in order to receive credit for their work. Simply "reputable" in this period did not necessarily hateful "refereed." Many influential journals—and arguably the well-nigh influential ones of all—did non use refereeing at whatsoever point during the nineteenth century.

In the early nineteenth century, French journals such as the Annals de Chimie sometimes included essays evaluating recent papers released by the Academie des Sciences. In 1831, inspired by those French essays, the English polymath William Whewell (1794–1866) made a proposal to his fellow members of the Royal Society of London: that papers submitted to the Philosophical Transactions should be sent to 2 Fellows, who would then write a written report about the paper. The signed report would be published in the Royal Club'due south new periodical, the Proceedings of the Majestic Society of London. The reports would non only generate scientific debates but too material for the Royal Lodge to publish in its new periodical.¹⁰

Whewell's scheme quickly ran into trouble when the first two referees—Whewell himself and the mathematician John Lubbock (1803–1865)—disagreed vehemently nearly the claims of the paper they'd been sent and had trouble drafting a report they were both willing to sign. Just a few referee reports were ever printed in the Proceedings. Simply the exercise of soliciting written reports about Transactions submissions endured, with a twist: instead of asking referees to publish their comments and make their identities public, referee reports became confidential documents that helped the Society determine whether or non to print a newspaper. The writer of the original paper would never know who his referees were or what they had said virtually his work. It was not until the early twentieth century that journals started sending their authors either paraphrased or verbatim copies of referee reports.

By the mid-nineteenth century, 1 of the major responsibilities of the Secretaries of the Royal Society—in that location were ii, one in the physical and one in the biological sciences—was to find people to referee submissions to the Transactions. If one of the Regal Society Secretaries wished to communicate concerns about the paper to the author, he would write the writer a personal letter and advise some changes that the author might desire to brand before the paper went to impress.^xi Other learned societies followed the Royal Society's case and began consulting anonymous referees well-nigh papers submitted to their journals. Great Britain's Geological Gild and Royal Society of Chemical science, for case, both adopted refereeing systems for their publications in the nineteenth century. In the US, the American Physical Social club and the American Sociological Clan adopted refereeing at their journals in the early on twentieth century, only non for every newspaper. Papers the editor liked were often non sent out for external opinions; referees were almost ofttimes employed when a paper seemed questionable and the editor wanted help justifying its rejection.¹²

In the late nineteenth and early twentieth centuries, ideas about referees and their purpose started to undergo some important changes. In the 1840s and 1850s, the referee'southward job was to advise the editor. They were supposed to give a paper a off-white hearing and tell the journal's editor if annihilation near it was obviously embarrassing and might discredit the publication; referees were not necessarily expected to make judgments about the truth of the findings. As the number of journals expanded, however, some observers started to complain that there were as well many scientific articles, and too many of them were terrible—"veritable sewage thrown into the pure stream of scientific discipline," as physiologist Michael Foster put it. Those concerns about the quality of the scientific literature led members of the scientific customs to view the referee not as a mere advisor, but as a gatekeeper—someone who was responsible for the quality of the scientific literature equally a whole. In practise, that vision of the referee equally a guardian of the scientific literature led many journals to rely on a minor number of prominent scientists for their refereeing in the late nineteenth century, meaning that an elite inner circumvolve had enormous ability to shape a journal'southward contents.¹³

Though it is tempting to have this as the model for the modern system of peer review, there were many of import journals in the nineteenth century that didn't use referees at all. For-turn a profit scientific journals like the Philosophical Magazine and Nature relied entirely on their editorial staffs to make decisions well into the twentieth century.^xiv Many of those commercial journals were run past dynamic editors who strongly preferred to evaluate submissions themselves and accept or turn down them based on the editors' own authority. Furthermore, learned societies had the luxury of issuing their publications whenever they had enough content. Commercial journals, on the other hand, were expected to appear on a regular weekly or monthly schedule. Editors were normally trying to recruit content, not reject information technology, and they did not desire to take to wait for referees to render their comments earlier moving forward with a promising article. But since refereeing at a journal was not seen as a sign of scientific rigor or respectability during this menstruum, the lack of referees did non forestall commercial journals for existence venues for many of import scientific papers. The choice to use referees or not was essentially a logistical 1, non an epistemological one.

Image 2: Cover of Nature from first outcome in 1869. Source: Nature (https://www.nature.com/articles/d41586-018-07844-6)

In the nineteenth century, refereeing was almost entirely a British and American practice. Exterior the English-speaking scientific world, very few periodicals consulted referees nearly their contents. In 1835, when the French Académie des Sciences founded the Comptes rendus hebdomadaires des séances de l'Académie des sciences, they made the deliberate conclusion non to solicit reports almost papers before moving frontwards with publication. The Académie wanted to make quick publication a priority and idea consulting outside opinions would only gum up the works.¹⁵ In Germany, prominent journals like the Annalen der Physik und Chemie were usually run past powerful editors, along similar lines to the British and American commercial journals.^xvi Scientists who weren't used to outside refereeing did not necessarily embrace the practise with enthusiasm when they encountered information technology for the start time. Famously, in 1936 Albert Einstein was incensed when John Tate, the editor of the American journal Concrete Review, sent him a referee report about a paper he had recently submitted. Einstein informed Tate that he and his coauthor Nathan Rosen

had non authorized you to prove [our manuscript] to specialists earlier information technology is printed. I see no reason to address the—in whatever case erroneous—comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere.¹⁷

Journals were not the simply institutions meant to evaluate the quality of science. At funding bodies, external refereeing of grant proposals was quite rare prior to Globe War 2. Private grant organizations such as the Rockefeller Foundation, for example, mostly left funding decisions in the hands of their employees, who were trusted to evaluate a scientist's worthiness for a grant internally whether or not they understood the scientific discipline at stake.¹⁸ Fifty-fifty authorities funding bodies tended to let their employees or a pocket-size number of trusted advisors make funding decisions with little or no input from experts who were non affiliated with the grant organization. When the Emergency Clan for German Science (later the German language Research Foundation) was created in 1920, its founders asked a small number of elite scientists to evaluate proposals, and the process judged the applicants' personal qualities as much as their science.¹⁹

A few grant organizations did rely on referees to provide input on proposals, mostly ones affiliated with learned societies or governments. For example, the American National Research Quango, the private inquiry arm of the National Academy of Sciences, developed increasingly formalized refereeing procedures during the 1920s and 1930s, in part to ensure fairness to researchers without established national reputations.²⁰ Just many regime funding bodies, including the American National Institutes of Health and the British Medical Inquiry Council, preferred to create an in-house panel of researchers to evaluate grants instead of reaching out to experts outside the funding body.²¹

At both journals and funding bodies the utilize of external referees was entirely optional prior to the Second Earth War. If a journal prized fast publication, or if a funding body trusted its employees to make decisions, the organization could eschew systematic refereeing with no damage to its reputation. There was fiddling to suggest that refereeing would become an essential office of scientific knowledge-making.

Peer review in the Cold War

Journals and funding bodies both began placing more than emphasis on external refereeing following the Second World War. One reason for the shift towards refereeing at journals was the increasing burden on editors as the number of scientists—and scientific papers—expanded due to generous Cold War funding. At the American weekly Scientific discipline, for case, the Editorial Board handled all refereeing in-house for the beginning half of the twentieth century. In the 1950s, however, members of the editorial board complained that "the job of refereeing and suggesting revisions for hundreds of technical papers is neither the best use of their fourth dimension nor pleasant, satisfying work," and agreed to brainstorm sending papers to outside experts.²² Similarly, when the American Journal of Medicine was founded in 1946, its editor Alexander Gutman wanted to offer his authors fast publication and decided to handle acceptances and rejections almost entirely on his own. However, equally the journal became more popular, Gutman was unable to keep up with the number of submissions, and by the 1960s he too had begun sending papers out for external opinions.²³

Just the growing workload at scientific journals does non explicate how refereeing became crucial to the very idea of scientific rigor. The link between refereeing and scientific legitimacy seems to accept followed the dramatic rise in authorities-sponsored inquiry in the postwar United States. Between 1948 and 1953, federal spending on scientific research in the United states increased by a factor of 25.²⁴ The massive expansion of government funding led to more than public attention on scientists—and to suggestions that science should be more accountable to the public and to members of the US Congress. Scientists, on the other manus, were less than enthusiastic about the idea that their grant proposals should be evaluated by laymen with no scientific training.

The tension between accountability to the public and scientific autonomy came to a caput in the mid-1970s during a controversy over inquiry grants awarded past the National Science Foundation (NSF). The early 1970s were a time of growing economic crisis for the US, and 3 legislators—Republican Congressmen John Conlan (1930–) and Robert Bauman (1937–), and Democratic Senator William Proxmire (1915–2005)—launched a series of very public attacks on specific grants the NSF had awarded. All three men defendant the NSF of awarding frivolous grants—largely in the social sciences—and wasting taxpayer money on projects such as a middle-school sociology curriculum and a study of stress in rats and monkeys. Bauman and Proxmire argued that the NSF's poor controlling justified far more Congressional control over the grants they awarded.²⁵

Conlan took his criticisms a step farther: he defendant the NSF of employing a shoddy, secretive review process. Unusually for a grant organization, the NSF had employed external referees to evaluate proposals since its foundation in 1950. According to Conlan, however, its refereeing procedure was a sham. He claimed that NSF employees had the power to brand whatever conclusion they wanted, and that they frequently ignored referee opinions. Conlan argued that the full text of all referee reports, plus the names of the authors, needed to be available to both the grant applicants and the public at big. The NSF, however, insisted that referees would not be able to offer aboveboard feedback unless their identities were kept confidential. By this fourth dimension, referees for journals and for the few funding bodies who used them systematically had grown to expect that their identities would remain a cloak-and-dagger from the authors, and NSF Director H. Guyford Stever bandage Conlan's request as a major break with scientific protocol.

Prototype 3: Rep. John Conlan (Arizona). Source: Congressional Pictorial Directory (1973), 93rd US Congress, p. 12

The controversy eventually became and then heated that Congress convened a hearing on the NSF's review procedures, held in July 1975. NSF leaders, including manager Stever and deputy director Richard Atkinson, argued that their reviewer system was basically sound, that NSF employees did not have undue power, and that grant review had to exist left in scientific hands in club to exist valid. They agreed to identify more weight on referee opinions and created a new audit office to confirm that both positive and negative referee reports were taken into account when making a final decision near a grant.

Meanwhile, almost everyone chosen to testify at the hearings, from scientists to public servants, agreed that review by colleagues was a crucial part of science. Witnesses called information technology "a fair and effective system,"²⁶ "an indispensable component of the decisionmaking process for allocating funds,"²⁷ and "an integral characteristic of 'the scientific method.'"²⁸ In the face of a challenge to their autonomy, scientists now insisted refereeing was more than an optional bureaucratic function: information technology had become "indispensable" for the practice of science. That shift helped the NSF resist major changes to its grant-awarding powers, and also created the expectation that scientific quality would be assessed by scientists—creating a sense of intellectual autonomy for the scientific community fifty-fifty as scientists were ever more than dependent on support from the federal government.

Interestingly, the mid-1970s were also the moment when Americans increasingly chosen refereeing by a new term: "peer review." The term offset came into utilise in the U.s. to describe the ways physicians evaluated each other nether the Professional person Standards Review System (PSRO) associated with Medicaid. Past the mid-1960s the term "peer review" was used to draw grant review procedures at the National Institutes of Health, where small-scale study sections of in-business firm experts evaluated proposals; by the 1970s "peer review" was synonymous with all funding body and periodical refereeing in the US.²⁹ The shift from "refereeing" to "peer review" is not just a linguistic oddity. Renaming the refereeing process "peer review" implies that simply a very narrow range of people—the expert peers of the person who wrote the paper or grant proposal—can be permitted to weigh in on the paper or proposal's worthiness. It limits the range of acceptable reviewers. The term "peer review" itself helped connect claims nearly the quality of scientific enquiry to the people immune to be involved in its publication.

Peer review in the 21st century

The new emphasis on peer review in the Usa was soon reflected in journals as well; American scientists increasingly regarded journals with unsystematic peer review systems as shoddy or unreliable. Relying on editorial judgment or the opinions of funding body employees was no longer acceptable; a respectable scientific institution needed to listen to its referees in lodge to make decisions about which science deserved publication or funding. The judgment of the peer reviewers—experts offer their time anonymously in order to serve the cause of improving science—came to represent the judgment of the scientific community equally a whole. Peer review was thus elevated from an optional bureaucratic process to a system that was supposed to ensure the quality and trustworthiness of scientific discipline.

Countries and organizations outside the purview of American governmental oversight tended to embrace systematic external refereeing afterward, and did not necessarily see it as a exercise key to science until later the 1970s. Equally late as 1989, the editors of the prominent British medical journal The Lancet complained that "in the United states of america far as well much is being asked of peer review" and proudly assured readers that at The Lancet, "reviewers are advisers not decision makers."^xxx The influence and sheer size of the American scientific community, however, ensured that the vision of peer review equally the key to scientific respectability spread across U.s.a. borders. Even the Lancet began using more than and more referees out of business organisation that without formal peer review, they could no longer attract American authors or readers.³¹ The experience of The Lancet was typical: though possibly initially lamenting the emphasis American journals and funding put upon peer review, its increasing axis in the US combined with the scientific importance of American authors, universities, and funding bodies ensured the do spread quickly. The impact spread across the sciences; disciplines in the humanities also came to rely more on peer review to signal scholarly legitimacy in the late 1970s and early 1980s. By the 1990s, few journals or funding bodies in any nation were peer review holdouts.

More than recently, peer review seems have faced its ain moment of crisis. Critics have argued that the peer review process is not doing a proficient job of distinguishing good science from bad. Several high-profile papers have been published in top journals after having passed through peer review, only to be heavily criticized after publication or retracted amid allegations of fraud.³² Some studies have indicated that women and underrepresented minorities are more than likely to receive unfavorable referee reports than their colleagues.³³ Other observers accept argued that peer review suppresses innovative inquiry and rewards more familiar, safer projects.³⁴ In 2011, Great Britain's House of Commons deputed a report on the country of peer review, and ended that while peer review "is crucial to the reputation and reliability of scientific research," many scientists believe the system stifles progress, is often biased, and that "there is little solid evidence on its efficacy."³⁵ In the 1970s, peer review was recast as the organisation that rewarded adept scientific discipline and corrected bad scientific discipline; in the 2010s, scientists are now grappling with the fact that information technology doesn't seem to practice either of those things particularly well.

Members of the scientific customs take offered several different visions for how peer review might change in the future. Ane popular suggestion is that referees should sign their reports—a proposition that unconsciously echoes both Whewell's initial vision for referee reports in the 1830s and Conlan's program for the NSF in the 1970s. Proponents of that change argue that if reviewers accept to sign their comments, those comments will be more thoughtful and more than effective than if the reviewers are allowed to remain anonymous. Other observers argue that scientists should eschew peer review entirely and allow researchers to merely post their papers for feedback from the whole community, as physicists do on www.arxiv.org. And so far, none of these suggestions has gained much traction, or had much of an influence over publishing or grant refereeing practices. Some journals, notwithstanding, take begun experimenting with smaller-scale changes to the peer review system. The periodical eLife, for example, solicits opinions from multiple referees, just does not send their reports straight to the author. Instead, the referees and editor discuss the newspaper with each other to get in at one unanimous conclusion virtually whether the paper should be accepted, revised, or rejected.³⁶

Ane of the reasons that it has been difficult to imagine substantial change to the peer review system may exist the persistence of the myth claiming Henry Oldenburg invented peer review during the Scientific Revolution. If peer review is framed every bit an unchanging and fourth dimension-honored organisation, intimately connected to the origins of modern science itself, reform seems impossible. But peer review has a far messier and more complicated history than its origin myth suggests, and its role in the scientific community has never been static. Its course and purpose accept been shaped and reshaped according to what scientists needed from the practice—brownie for a new scientific society, improvement in the level of scientific literature, or assurance to patrons that money was being spent responsibly. Refereeing first emerged during a major shift in scientific publishing—the rise of a journal-centric system—and peer review became elevated to a cardinal place in scientific practice equally one resolution to a shifting relationship between scientists and funding sources. As scientific publishing grapples with the rise of online publishing, and every bit scientists attempt to navigate a political moment that has seen deep skepticism of fields like vaccine enquiry and climate science, we may be continuing on the brink of large changes for peer review as we know it.

Bibliography

National Scientific discipline Foundation Peer Review Special Oversight Hearings: Hearings before the Subcommittee on Science, Research, and Applied science, 1995. Washington, D.C.: U.S. Government Printing Office.

"Peers Reviewed," 1989. The Lancet 333: 1115–1116.

Philip Abelson Papers, American Association for the Advancement of Science Archives, Washington, DC.

"Science and Engineering Committee, Eighth Report: Peer Review in Scientific Publications," 2011. www.Parliament.united kingdom, United Kingdom Parliament, London, accessed 29 July 2016, <http://world wide web.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/ 856/856.pdf >.

Bakanic, Von, Clark McPhail, and Rita Simon, 1987. "The Manuscript Review and Decision-Making Process." American Sociological Review, 52: 631–642.

Baldwin, Melinda, 2014. "Tyndall and Stokes: Correspondence, Referee Reports, and the Physical Sciences in Victorian United kingdom." In Bernard Lightman and Michael Reidy, eds., The Age of Scientific Naturalism: John Tyndall and his Contemporaries, 171–186. London: Pickering and Chatto.

Baldwin, Melinda, 2015. Making Nature: The History of a Scientific Periodical. Chicago: University of Chicago Press.

Baldwin, Melinda, 2018. "Scientific autonomy, public accountability, and the rise of 'peer review' in the Cold War U.s.a.," Isis 109: 538–558.

Barany, Michael, 2018. "A Postwar Guide to Winning a Science Grant." Physics Today Online, doi:10.1063/PT.vi.4.20180320a.

Boas Hall, Marie, 2002. Henry Oldenburg: Shaping the Royal Society. Oxford: Oxford Academy Press.

Bornmann, Lutz, Rüdiger Mutz, and Hans-Dieter Daniel, 2007. "Gender differences in grant peer review: A meta-analysis," Journal of Infometrics i: 226–238.

Burnham, John, 1990. "The Evolution of Editorial Peer Review." Periodical of the American Medical Association 263: 1323–1329

Burnham, John, 1992. "How Journal Editors Came to Develop and Critique Peer Review Procedures," in Research Ethics, Manuscript Review, and Journal Quality: Proceedings of a Symposium on the Peer Review-Editing Process, ed. H. F. Maryland and R. E. Sojka, 55–62. Madison, Wis.: ACS Miscellaneous Publications.

Chubin, Daryl and Edward Hackett, 1990. Peerless Science: Peer Review and U.S. Scientific discipline Policy. Albany: Country Univ. New York Press.

Clarke, Imogen, 2016. "The Gatekeepers of Modernistic Physics: Periodicals and Peer Review in 1920s Uk." Isis 106: lxx–93.

Clarke, Imogen and Jim Mussell, 2015. "Conservative attitudes to old-established organs: Oliver Lodge and Philosophical Magazine." Notes and Records: The Royal Social club Periodical of the History of Science 69: 321–336.

Csiszar, Alex, 2018. The Scientific Periodical: Authorship and the Politics of Knowledge in the Nineteenth Century. Chicago: University of Chicago Printing.

Csiszar, Alex, 2016. "Peer Review: Troubled from the Showtime." Nature 532: 306–308.

Frodeman, Robert and Adam Briggle, 2012. "The Dedisciplining of Peer Review." Minerva 50: 3–19.

Fyfe, Aileen, Julie McDougall-Waters and Noah Moxham, 2015. "350 Years of Scientific Periodicals." Notes and Records: The Royal Society Journal of the History of Science 69 no. three: 227–239.

Ginther, Donna, Walter Schaffer, Joshua Schnell, Beth Masimore, Faye Liu, Laurel Haak, and Raynard Kington, 2011. "Race, Ethnicity, and NIH Research Awards," Science 333: 1015–1019.

Goldstein, Raymond E, 2016. "Commentary: A biology journal provides a lesson in peer review." Physics Today 69 no. 12: ten-11.

Goodrich, Dorris West, 1945. "An Analysis of Manuscripts Received past Editors of the American Sociological Review from May 1, 1944 to September 1, 1945." American Sociological Review 10: 716–725.

Gross, Alan G., Joseph East. Harmon, and Michael Reidy, 2002. Communicating Scientific discipline: The Scientific Commodity from the 17^th Century to the Present. Oxford: Oxford University Press.

Ingelfinger, Frank, 1974. "Peer Review in Biomedical Publication." The American Journal of Medicine 56: 686–692.

Johns, Adrian, 1998. The Nature of the Book: Print and Noesis in the Making. Chicago: University of Chicago Printing.

Kaiser, David, 2014. "Shut upwards and calculate!" Nature 505: 153–155.

Kennefick, Daniel, 1999. "Controversies in the History of the Radiation Reaction Problem in General Relativity." In Hubert Goenner, Jürgen Renn, Jim Ritter, and Tilman Sauer, eds., The Expanding Worlds of General Relativity, 207–234. New York: Springer-Verlag.

Kohler, Robert, 1991. Partners in Scientific discipline: Foundations and Natural Scientists, 1900–1945. Chicago: Academy of Chicago Press.

Kronick, David, 1990. "Peer Review in 18th-Century Scientific Journalism." Journal of the American Medical Association 263 no. x: 1321-1322.

Lacour, Michael and Donald Green, 2014. "When contact changes minds: An experiment on transmission of support for gay equality," Science 346: 1366–1369.

Lalli, Roberto, 2016. "'Dirty work,' only someone has to do it: Howard P. Robertson and the refereeing practices of Concrete Review in the 1930s." Notes and Records: The Royal Social club Journal of the History of Science 70: 151–174.

Lock, Stephen, 1994. "Does Editorial Peer Review Work?" Annals of Internal Medicine 121: 60–61.

Mandel, Richard, 1996. A One-half Century of Peer Review, 1946–1996. Bethesda, Md: National Institutes of Health.

McClellan Three, J.E., 2003. Specialist Control: The Publications Committee of the Académie Royale des Sciences (Paris), 1700–1793. New York: American Philosophical Society.

Moxham, Noah and Aileen Fyfe, 2017. "The Royal Guild and the Prehistory of Peer Review, 1665–1965." Historical Periodical: doi:10.1017/S0018246X17000334.

Nightingale, Paul and Alister Scott, 2007. "Peer Review and the Relevance Gap: 10 Suggestions for Policy-Makers." Science and Public Policy 34: 543–553.

Pyenson, Lewis, 1985. "Concrete Sense in Relativity: Max Planck Edits the Annalen der Physik, 1906–1918." In Lewis Pyenson, ed., The Young Einstein: The Appearance of Relativity, 194–214. Bristol: Adam Hilger.

Rennie, Drummond, 1999. "Editorial Peer Review: Its Development and Rationale." In Fiona Godlee and Tom Jefferson, eds., Peer Review in Wellness Sciences, ed. Fiona Godlee and Tom Jefferson, 1–13. London: BMJ.

Siler, Kyle, Kirby Lee, and Lisa Bero, 2015. "Measuring the effectiveness of scientific gatekeeping," Proceedings of the National Academy of Sciences, 112: 360–365.

Thomson, A. Landsborough, 1973. Half a Century of Medical Enquiry. London: Her Majesty'south Jotter Part.

Wagner, Patrick, 2013. "'Preserve of Full Professors'—On the History of the German Enquiry Foundation between 1920 and 1970." In Mark Walker, Karin Orth, Ulrich Herbert and Rüdiger vom Bruch, eds., The High german Inquiry Foundation 1920-1970: Funding Posed betwixt Science and Politics, 23–40. Berlin: Franz Steiner Verlag.

Wakefield, Andrew et al., 1998. "Ileal-Lymphoid-Nodular Hyperplasia, Non-Specific Colitis, and Pervasive Developmental Disorder in Children." The Lancet 351: 637–641.

Weller, Anne, 2001. Editorial Peer Review: Its Strengths and Weaknesses. Medford, North.J.: Information Today.

Wennerås, Christine and Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature 387: 341–343.

Wolfe-Simon, Felisa et al., 2011 "A Bacterium That Can Abound by Using Arsenic Instead of Phosphorus," Science 332: 1163–1166.

Zuckerman, Harriet and Robert Merton, 1971. "Patterns of Evaluation in Science: Institutionalization, Structure, and Functions of the Referee System." Minerva ix: 66–100.

---

1. Zuckerman and Merton 1971.↩

2. Due east.k. Bakanic et al. 1987; Chubin and Hackett 1990; Frodeman and Briggle, "The Dedisciplining of Peer Review"; Ingelfinger 1971; Lock 1994.↩

3. See, due east.thou., Fyfe et al. 2015; Moxham and Fyfe 2017; Johns 1998, chapter 7.↩

4. Boas Hall 2002.↩

5. See, e.grand., Baldwin 2015; Baldwin 2018; Burnham 1990; Burnham 1992; Clarke 2016; Clarke and Mussell 2015; Csiszar 2018; Fyfe et al. 2015; Gross et al. 2002; Lalli 2016; Moxham and Fyfe 2017.↩

6. McClellan 2003.↩

7. Kronick 1990.↩

8. Moxham and Fyfe 2017.↩

9. Csiszar 2018.↩

10. Csiszar 2016; Csiszar 2018, pp. 142–148.↩

11. Baldwin 2014.↩

12. Lalli 2016; Goodrich 1945.↩

13. Foster quoted in Csiszar 2016; see likewise Clarke 2016.↩

14. On refereeing systems at these periodicals, see Baldwin 2015; Clarke and Mussell 2015.↩

15. Csiszar 2018, 110-117.↩

16. Pyenson 1985.↩

17. Albert Einstein, quoted in Kennefick 1999, p. 208.↩

18. Kohler 1991, 68–70; Barany 2018.↩

19. Wagner 2013.↩

20. Mandel 1996, pp. 4–5.↩

21. Thomson 1973, pp. 147–161.↩

22. "Minutes of the Joint Meeting of AAAS Editorial Board and Publications Commission, 10 June 1955," Abelson Papers, Box 1.↩

23. Ingelfinger 1974.↩

24. Kaiser 2014.↩

25. Baldwin 2018.↩

26. National Science Foundation Peer Review Special Oversight Hearings 1975, p. 1092.↩

27. National Scientific discipline Foundation Peer Review Special Oversight Hearings 1975, p. 460.↩

28. National Science Foundation Peer Review Special Oversight Hearings 1975, p. 390↩

29. Baldwin 2018.↩

30. The Lancet 1989.↩

31. Rennie 1999.↩

32. Encounter, e.g., Wolfe-Simon 2011, which faced vigorous criticism of its conclusions, and Lacour and Green 2014 and Wakefield 1998, which were retracted by their publishing journals after allegations of fraud.↩

33. Bornmann et al. 2007; Ginther et al. 2011; Wennerås and Wold 1997.↩

34. Nightingale and Scott 2007; Siler, Lee, and Bero 2015.↩

35. "Scientific discipline and Technology Committee, 8th Study: Peer Review in Scientific Publications," 2011.↩

36. Goldstein 2016.↩

KEYWORDS

1. scientific practice
2. scientific publishing

ibarraarks1973.blogspot.com

Source: https://lps.library.cmu.edu/ETHOS/article/id/19/

Share this post