By Gwendolynne Reid, Cherice Escobar Jones, and Mya Poe
Biases in scholarly citations against scholars of color promote racial inequality, stifle intellectual analysis, and can harm patients and communities.
While the lack of citations to scholars of color in medical journals may be due to carelessness, ignorance, or structural impediments, in some cases it is due to reckless neglect.
Our study demonstrates that the American Medical Association (AMA) has failed to promote greater racial inclusion in its flagship publication, the Journal of the American Medical Association (JAMA), despite an explicit pledge to do so.
A “peculiar duplication”
The AMA’s pledge is best understood in the context of the organization’s history of racism. And this history is intertwined with that of a lesser-known professional medical association, the National Medical Association (NMA).
After its founding in 1847, the AMA notably sidestepped the issue of integration by making membership for Black physicians contingent on state medical society membership.
Like other compromises in American history that have elevated state rights to preserve federal unity, this compromise allowed states that enforced racial segregation—primarily Southern states, where most Black people lived at the time—to exclude Black physicians from the AMA.
After decades of exclusion, Black physicians founded the National Medical Association (NMA) in Atlanta, Georgia in 1895. The resulting “peculiar duplication” of professional organizations created “unity through exclusion.”
Working within this racially exclusionary system, the NMA devoted itself to the health of Black Americans and the professional needs of the Black physicians serving Black patients, a mission that continues today. And for more than 100 years, the peer-reviewed Journal of the National Medical Association (JNMA), has dedicated itself to research “address[ing] medical care disparities of persons of African descent.”
A partial reckoning
In 2005, the AMA Institute for Ethics, in collaboration with both AMA and NMA leaders, convened a panel of experts to examine the origins of the racial divide in American medicine.
One outcome of this work was the 2008 apology by Ronald Davis, the AMA’s immediate past president, for the AMA’s history of racial exclusion at the NMA’s annual meeting in Atlanta. In his address, Davis pledged the AMA to doing “everything in our power to right the wrongs that were done by our organization to African-American physicians and their families and their patients,” going on to express hope “that we can continue down a path toward stronger and stronger collaboration and partnership.”
The AMA apology and commitment to collaborate with the NMA, along with its recent progress on addressing racism in its publication process, are promising steps towards addressing the racial bias baked into medical research and the evidence-based standards of care built on that research. So are the slightly increasing number of research studies published in international “gatekeeper” medical journals like JAMA that explicitly address the role of racism in health. A gap so far overlooked in these measures and analyses, however, is the role of citation in including or excluding medical knowledge produced by Black physicians and researchers.
Studies of citation underscore that citations serve complex purposes in knowledge production, including reward, affiliation, evidence, support, and community building. Those who are not cited are isolated and their work cannot attain the status of accepted scientific knowledge: they cannot contribute to the collective work of the scientific community. Such citational racism is a form of epistemological racism that compromises the validity of the knowledge produced by the scientific community and can hinder innovation.
Including Black scientists on medical research teams and including their work in international gatekeeper medical journals, then, is about more than professional inclusivity. It is an important ingredient towards reducing bias in medical knowledge and the racial inequities in health care this bias helps produce.
What we did
As a first step towards addressing citational racism, we traced the citational relationships between the AMA’s and NMA’s flagship journals — JAMA and JNMA — focusing on their opinion pieces as useful indicators of current and future research as well as priorities and agendas. We selected opinion pieces precisely because these are agenda setting and because we expected that JAMA editors might draw on insights from JNMA editors subsequent to the AMA’s 2008 apology and since the recently renewed racial justice movement.
We conducted a literature search in the SCOPUS database for articles labeled “editorials” in JAMA and JNMA since January 1, 2008. The articles with the “editorial” label in SCOPUS included editorials, opinion pieces, and JAMA Viewpoints.
Before conducting a citation analysis, we cleaned the resulting two datasets exported from SCOPUS using Microsoft Excel and two custom JavaScript scripts run via Node. Specifically, this process corrected variant spellings of select journals that would result in inaccurate counts of citation.
We then used VOSViewer to analyze the two cleaned datasets, which together totaled 1,542 articles, for journal citation frequency and reciprocity, which allowed us to identify which journals JAMA and JNMA opinion articles cite most frequently, and how often the two journals’ opinion pieces cite each other. VOSViewer’s citation counts by publication source provided the citation frequency figures employed below.
As a final step, we exported the indexed keywords for all the opinion pieces from JNMA and JAMA from 2008 to 2021. We searched the keyword index for the following terms–race, racism, justice, and white supremacy–which reflect the presence of racial justice discourse in our sample. This search allowed us to see how frequently such topics were addressed in the opinion pieces.
What we found
Between 2008 and 2021, JNMA published 117 opinion pieces, and JAMA published 1,425 opinion pieces. When we analyzed the top citations from each journal, we found that self-citation was common in both journals, with 8% (JNMA) and 16% (JAMA) of self-citation respectively. Because opinion pieces often address the content of the current journal issue, we were not surprised that self-citation was the most common citation in JNMA and JAMA (Figure 1).
Looking at the other most commonly-cited sources from 2008 to 2021, we found that both journals referenced high-impact scientific publications, such as NEJM, the Lancet, and Cancer. But when we looked more closely at the citational relationship between JNMA and JAMA, we discovered a pattern of citational exclusion. Notably, while JNMA opinion pieces from 2008 to 2021 cited JAMA 21 times (2.8% of JNMA opinion pieces), JAMA opinion pieces cited JNMA only twice during that same time period (.01% of JAMA opinion pieces). In other words, in more than a decade since the AMA apology, JAMA opinion authors have only cited two articles from JNMA–one article on initiatives at U.S. medical schools to recruit underrepresented minorities and a second article on the impact of direct-to-consumer advertisements on African American patients.[1]
Figure 1: Most frequently cited sources in JNMA and JAMA, 2008-2021
When we examined the use of the key terms race, racism, inequity, and white supremacy in JNMA and JAMA opinion pieces indexing, we found that, since 2008, both journals have published opinion pieces that use many of these terms as indexing keywords (Figure 2), though JAMA authors wrote about race and racism at a substantially lower rate than JNMA authors.
Figure 2: Focal Index Keywords, JNMA and JAMA, 2008-2021
An unmet promise
Our results are clear: despite the American Medical Association’s promise of greater inclusion in its 2008 apology to the National Medical Association and public declarations since, the AMA’s flagship journal has yet to reflect that promise in its citational practices. For more than 13 years since the 2008 apology, JAMA authors have continued to largely ignore contributions of JNMA researchers in their opinion pieces.
Our results are not encouraging and underscore the need for more intentional and explicit attention to citation as a means to address racial bias in medical knowledge.
Notably, our analysis shows that JAMA authors rarely cite publications from JNMA, even when writing about issues of racial injustice in health care, a subject well-suited for the flagship American medical journal on Black health. Even when writing about issues of racism in health care, a subject JNMA has explored extensively, JAMA authors rarely cite publications from JNMA. This citational exclusion means that over a century of research on the health and well-being of African Americans, often at intersections of care, disparities, and racism, are omitted from JAMA, one of the highest impact publications in the medical profession (impact factor: 56.27). As a result, this knowledge becomes systemically segregated and ghettoized, reducing its ability to influence ongoing medical research and improve care. In other words, epistemological racism has real implications for the patients and communities that knowledge serves.
A path forward
Because citational practices are influenced by both individuals and systems, dismantling citational racism requires a multi-pronged approach. As individual authors, we can be intentional to avoid perpetuating racist citational systems. And journals can put processes in place to support editors in evaluating the sources authors draw upon to make their scientific claims. JAMA, in particular, due to its unique role in the segregation of medicine in America and in the NMA’s genesis, should play a leadership role in mitigating the harm it has created and in citing research published in JNMA.
But citational inclusion is not merely referencing the work of Black researchers within research studies; “diversity must exceed mere tokenism.” Addressing citational racism means addressing both where and how Black researchers are cited, as well as understanding the flow of knowledge between journals like JAMA and JNMA.
And mitigating citational racism goes beyond individual authors and editors, as individual choices are built on technical systems researchers use to research and write. Technical systems like impact factors and algorithms, for example, reify decades of epistemic erasure, making it easy to find highly-cited sources from gatekeeper journals while burying sources from lesser known journals like JNMA that have been isolated over time.
Impact factors, for example, render invisible the racism that has shaped that influence. Writing this paper, we witnessed this erasure as we struggled to find papers published in JNMA amid the overwhelming search results for gatekeeper journals. We also discovered that JNMA was not included in the standard Elsevier ScienceDirect package, restricting our access to full texts from issues after the 2007 archive created by the National Library of Medicine. Reshaping how health science discourse values publications like JNMA will require deliberate, collaborative efforts across systems and institutions like libraries, databases, search engines, scientific publishers, and other stakeholders.
JNMA has devoted itself to the health and well-being of Black people for more than a century. Contributions from journals like JNMA should be front and center in scientific conversations about Black knowledge contributions and Black health. Whether oversights born of naive, unexamined habits or deliberate citational ostracism, exclusionary practices and the technical systems that support and reinforce citational racism can do real harm to real patients and communities.
Black Lives Matter, and Black knowledge production matters, too.
[1] JAMA opinion pieces did cite the 2017 American College of Cardiology/American Heart Association joint report on guidelines regarding the prevention, detection, evaluation, and management of high blood pressure in adults on five occasions. That report was co-signed by 11 professional organizations including the AMA and the NMA.
Gwendolynne Reid is Director of the Writing Program and Assistant Professor of English at Oxford College of Emory University.
Cherice Escobar Jones is a PhD student at Northeastern University studying rhetorics of race at the intersection of language, writing, and health.
Mya Poe is an associate professor of English at Northeastern University.
