Analysis of ISCB honorees and keynotes reveals disparities

Trang T. Le; Daniel S. Himmelstein; Ariel A. Hippen Anderson; Matthew R. Gazzara; Casey S. Greene

Professional societies and the conferences that they manage provide an important venue for the dissemination of scientific knowledge. Being invited to deliver a keynote at an international society meeting or named a fellow of such a society is a major recognition. We sought to understand the extent to which such recognitions reflected the composition of their corresponding field. We collected keynote speaker invitations for the international meetings held by the International Society for Computational Biology as well as the names of Fellows, an honorary group within the society. We compared these honorees with last and corresponding author contributions in field-specific journals. We used multiple methods to estimate the race, ethnicity, gender, and nationality of authors and the recipients of these honors. To address weaknesses in existing approaches, we built a new dataset of more than 700,000 people-nationality pairs from Wikipedia and trained long short-term memory neural networks to make predictions. Every approach consistently shows that white scientists are overrepresented among speakers and honorees, while scientists of color are underrepresented.

Introduction

Scientists’ roles in society include identifying important topics of study, undertaking an investigation of those topics, and disseminating their findings broadly. The scientific enterprise is largely self-governing: scientists act as peer reviewers on papers and grants, comprise hiring committees in academia, make tenure decisions, and select which applicants will be admitted to doctoral programs. A lack of diversity in science could lead to pernicious biases that hamper the extent to which scientific findings are relevant to minority communities. For example, finding that minority scientists tend to apply for awards on topics with lower success rates [1] could be interpreted either as minority scientists select topics in more poorly funded areas or that majority scientists consider topics of particular interest to minority scientists as less worthy of funding. Consequently, it is important to examine peer recognition in different scientific fields.

Gender bias among conference speakers has been recognized as an area that can be improved with targeted interventions [2,3,4,5]. Having more female organizers on conference committees is associated with having more female speakers [6]. At medical conferences in the US and Canada, the proportion of female speakers is increasing at a modest rate [7]. Gender bias appears to also influence funding decisions: an examination of scoring of proposals in Canada found that reviewers asked to assess the science produced a smaller gender gap in scoring than reviewers asked to assess the applicant [8]. Challenges extend beyond gender: an analysis of awards at the NIH found that proposals by Asian, black or African-American applicants were less likely to be funded than those by white applicants [9]. There are also potential interaction effects between gender and race or ethnicity that may particularly affect women of color’s efforts to gain NIH funding [10].

We used multiple methods to predict the gender, race/ethnicity, and nationality of honorees. Existing methods were relatively US-centric because most of the data was derived in whole or in part from the US Census. We scraped more than 700,000 entries from English-language Wikipedia that contained nationality information to complement these existing methods and built multiple machine learning classifiers to predict nationality. We also examined the last and corresponding authors for publications in ISCB partner journals to establish a field-specific baseline using the same metrics. The results were consistent across all approaches: we found a dearth of non-white speakers and honorees. The lack of Asian scientists among keynote speakers and Fellows was particularly pronounced when compared against the field-specific background.

Materials and Methods

Honoree Curation

From ISCB’s webpage listing ISCB Distinguished Fellows, we found recipients listed by their full names for the years 2009-2019. We gleaned the full name of the Fellow as well as the year in which they received the honor. To identify ISMB Keynote Speakers, we examined the webpage for each ISMB meeting. We found webpages with full names for keynote speakers for the years 2002-2019. On the PSB conference webpages, we found PSB Keynote Speakers for the years 1999-2020.

For the RECOMB meeting, we found conference webpages with keynote speakers for 1999, 2000, 2001, 2004, 2007, 2008, and 2010-2019. We were able to fill in the missing years using information from the RECOMB 2016 proceedings, which summarizes the first 20 years of the RECOMB conference [11]. This volume has two tables of keynote speakers from 1997-2006 (Table 14, page XXVII) and 2007-2016 (Table 4, page 8). Using these tables to verify the conference speaker lists, we arrived at two special instances of inclusion/exclusion. Although Jun Wang was not included in these tables, we were able to confirm that he was a keynote speaker in 2011 with the RECOMB 2011 proceedings [12], and thus we included this speaker in the dataset. Marian Walhout was invited as a keynote speaker but had to cancel the talk due to other obligations. Because her name was neither mentioned in the 2015 proceedings [13] nor in the above-mentioned tables, we excluded this speaker from our dataset.

Name processing

When extracting honoree names, we began with the full name as provided on the site. Because our prediction methods required separated first and last names, we chose the first non-initial name as the first name and the final name as the last name. We did not consider a hyphen to be a name separator: for hyphenated names, all components were included. For metadata from PubMed and PMC where first (fore) and last names are coded separately, we applied the same cleaning steps. We created functions to simplify names in the pubmedpy Python package to support standardized fore and last name processing.

Corresponding author extraction

We assumed that research advisors in the field would be those most likely to be invited for keynotes or to be honored as Fellows. Therefore, we collected corresponding author names to assess the composition of the field, weighted by the number of corresponding authors per publication.

We evaluated two resources for extracting corresponding authors from papers: PubMed and PubMed Central (PMC). Both resources are provided by the US National Library of Medicine and index scholarly articles. PubMed contains a record for every article published in journals it indexes (30 million records total circa 2020) and provides abstracts but not fulltext. PMC, which provides fulltext access, does not contain every article from every journal (5.9 million records total circa 2020). In general, open access journals will deposit their entire catalog to PMC (e.g., BMC Bioinformatics & PLOS Computational Biology), while toll access journals (e.g., Bioinformatics) will only deposit articles when funders require it. Since PMC requires publishers to submit fulltext articles in a structured XML format, the machine-readability and breadth of metadata in PMC is often superior to PubMed.

Of PMC’s 5.9 million fulltext articles, only 2.7 million are part of the “Open Access Subset” which allows for downloading the structured fulltext as opposed to just viewing the article online. However, authorship information does not require full text records. We were able to download structured frontmatter (rather than fulltext) records from PMC’s OAI-PMH service, so we were not limited to just the Open Access Subset. For PubMed, we used the E-Utilities APIs. For PubMed records, we were able to extract author first and last names and their order within a record. For PMC, we were able to extract these fields as well as whether each author was a corresponding author. To automate and generalize these tasks, we created the pubmedpy Python package.

We selected three journals to represent the field of bioinformatics and computational biology, including two ISCB Partner Journals (PLOS Computational Biology and Bioinformatics) and one field-specific journal that is not a partner (BMC Bioinformatics). From PubMed, we compiled a catalog of 29,755 journal articles published from when each journal was established through 2019. We were able to retrieve authorship information for all but 6 of these articles using PubMed or PubMed Central.

To determine corresponding authors for an article, we relied on PMC data if available (20,696 articles) and otherwise fell back to PubMed data (9,053 articles). Almost all articles without PMC data were from Bioinformatics because it is a “selective deposit” rather than “full paricipation” journal in PMC.

We performed further analysis on PMC authors to learn more about corresponding author practices. First, we developed and evaluated a method to infer a corresponding author when the coded corresponding status was not available. For papers with multiple authors and at least one corresponding author, the first author was corresponding 43% of the time, whearas the last author was corresponding 62% of the time. Therefore, we assumed the last author was corresponding when coded corresponding author status was not available (120 articles from PMC and all articles from PubMed).

Second, we investigated the number of corresponding authors for PMC articles. 81% of these articles had a single corresponding author. 1.7% had no corresponding authors. Of these, many were editorials (e.g., PMC1183510, the announcement of PLOS Computational Biology). A very small number of papers had over 10 corresponding authors. Some of these instances were true outliers, like PMC5001208 with 21 corresponding authors. Others like PMC3509495 were incorrect, due to upstream errors. To not give undue influence to papers with multiple corresponding authors, subsequent analyses on corresponding authors are inversely weighted by the number of corresponding authors per paper.

Estimation of Gender

We predicted the gender of honorees and authors using the https://genderize.io API, which produces predictions trained on over 100 million name-gender pairings collected from the web. We used author and honoree first names to retrieve predictions from genderize.io. The predictions represent the probability of an honoree or author being male or female. We used the estimated probabilities and did not convert to a hard group assignment. For example, a query to https://genderize.io on January 26, 2020 for “Casey” returns a probability of male of 0.74 and a probability of female of 0.26, which we would add for an author with this first name. Because of the limitations of considering gender as a binary trait and inferring it from first names, we only consider predictions in aggregate and not as individual values for specific scientists.

Of 411 ISCB honorees, genderize.io fails to provide gender predictions for two names. Of 34,005 corresponding authors, 45 were missing a fore name altogether in the raw paper metadata and 1,466 had fore names consisting only initials. Of the remaining authors, genderize.io failed to predict gender for 1,578 of these fore names. We note that approximately 52% of these NA predictions are hyphenated names, which is likely because they are more unique and thus are more difficult to find predictions for. 87% of these names were predicted to be of Asian origin by last name (see the race/ethnicity prediction model below).

Estimation of Race and Ethnicity

We predicted the race and ethnicity of honorees and authors using the R package wru. wru implements methods described in Imai and Khanna [14] to predict race and ethnicity using surname and location information. The underlying data used for prediction are derived from the US Census. We used only the surname of author or honoree to make predictions via the predict_race() function. However, in the case of names that were not observed in the census, the function’s behavior was to use the average demographic distribution from the census. We modified the function to return a status denoting that results were inconclusive instead. This prediction represents the probability of an honoree or author selecting a certain race or ethnicity on a census form if they lived within the US.

Of 411 ISCB honorees, wru fails to provide race/ethnicity predictions for 98 names. Of 34,050 corresponding authors, 40 were missing a last name in the paper metadata, and 8,770 had a last name for which wru did not provide predictions. One limitation of wru and other methods that infer race, ethnicity, or nationality from last names is the potentially inaccurate prediction for scientists who changed their last name during marriage, a practice more common among women than men.

Estimation of Nationality

To complement wru’s race and ethnicity estimation, we developed a model to predict geographical origins of names. The existing Python package ethnicolr [15] produces reasonable predictions, but its international representation in the data curated from Wikipedia in 2009 [16] is still limited. For instance, 76% of the names in ethnicolr’s Wikipedia dataset are European in origin, and the dataset contains remarkably fewer Asian, African, and Middle Eastern names compared to that of wru.

To address the limitations of ethnicolr, we built a similar classifier, a Long Short-term Memory (LSTM) neural network, to infer the region of origin from patterns in the sequences of letters in full names. We applied this model on an updated, approximately 4.5 times larger training dataset called Wiki2019 (described below). We tested multiple character sequence lengths and, based on this comparison, selected tri-characters for the primary results described in this work. We trained our prediction model on 80% of the Wiki2019 dataset and evaluated its performance using the remaining 20%. This model, which we term Wiki2019-LSTM, is available in the online file LSTM.h5.

To generate a training dataset for nationality prediction, we scraped the English Wikipedia’s category of Living People, which contained approximately 930,000 pages at the time of processing in November 2019. This category reflects a modern naming landscape. It is regularly curated and allowed us to avoid pages related to non-persons. For each Wikipedia page, we used two strategies to find a full birth name and nationality for that person. First, we used information from the personal details sidebar; the information in this sidebar varied widely but often contained a full name and a place of birth. Second, in the body of the text of most English-language biographical Wikipedia pages, the first sentence usually begins with, for example, “John Edward Smith (born 1 January 1970) is an American novelist known for …” We used regular expressions to parse out the person’s name from this structure and checked that the expression after “is a” matched a list of possible nationalities. We were able to define a name and nationality for 708,493 people by using the union of these strategies. Our Wikipedia-based process returned a nationality or country of origin, which was more fine-grained than the broader regional patterns that we sought to examine among honorees and authors. We initially grouped names by continent, but later decided to model our categorization after the hierarchical nationality taxonomy used by NamePrism [17]. Consequently, we used the following categories: Hispanic (including Latin America and Iberia), African, Israeli, Muslim, South Asian, East Asian, European (non-British, non-Iberian), and Celtic English (including US, Canada, and Australia). Table 1 shows the size of the training set for each of these regions as well as a few examples of PubMed author names that had at least 90% prediction probability in that region. We refer to this dataset as Wiki2019 (available online in annotated_names.tsv).

Results

Curated Honorees and Literature-derived Potential Honorees

We curated a dataset of ISCB honorees that included 411 honorees who were keynote speakers at international ISCB-associated conferences (ISMB, RECOMB, and PSB) as well as ISCB Fellows. The ISCB Fellows set contained the complete set of Fellows named (2009-2019). Keynote speakers were available for ISMB for all years from 2002-2019. Keynote speakers from PSB were available for all years from 1999-2020. Keynote speakers for RECOMB were available for all years from 1997-2019. We included individuals who were honored multiple times as separate entries. For example, Christine Orengo was a keynote speaker at RECOMB 2004 and became an ISCB Fellow in 2016, and thus was counted twice in this list.

We sought to compare this dataset with a background distribution of potential speakers, which we considered to be last or senior authors of bioinformatics and computational biology manuscripts. We used those published in Bioinformatics, BMC Bioinformatics, and PLOS Computational Biology as a set of bioinformatics and computational biology manuscripts. We downloaded the metadata of manuscripts published in these journals from PubMed, which provided almost 30,000 articles for evaluation. However, although PubMed provides author order, it does not provide corresponding author information. To determine corresponding authors for an article, we used the PMC corresponding author information when it was available (20,696 articles) and the PubMed last author as a fallback when corresponding author information was missing (9,053 articles).

Assessing Gender Diversity of Authors and Honorees

Although Bioinformatics was established in 1998 and BMC Bioinformatics in 2000, the metadata for these journal papers before 2002 only have initials for first and/or middle author names. Therefore, without first and middle names, we do not have author gender predictions before this year.

We observed a slow increase of the proportion of predicted female authors, arriving at just over 20% in 2019 (Fig. 1, left). We observe very similar trend within each journal, but estimated female proportion has increased the least in PLOS Computational Biology (see notebook). ISCB Fellows and keynote speakers appear to be more evenly split between men and women compared to the population of authors published in computational biology and bioinformatics journals (Fig. 1, right); however, it has not yet reached parity. We observed an increasing trend of honorees who were women in each honor category, especially in the group of ISCB Fellows (see notebook), which markedly increased after 2015. Through 2019, there were a number of examples of meetings or ISCB Fellow classes with a high probability of recognizing only male honorees and none that appeared to have exclusively female honorees. However, the 2020 PSB keynotes, though outside of the primary range of our analyses, had nearly all the probability ascribed to female speakers.

Assessing the Racial and Ethnic Diversity of Authors and Honorees

We predicted the race and ethnicity of authors and honorees using wru, which is based on US census data. We found that an increasing proportion of authors in computational biology and bioinformatics journals had last names associated with selecting Asian as a race/ethnicity category in the US census (Fig. 2A). This was primarily driven by publications in Bioinformatics and BMC Bioinformatics (Fig. 2B, top). We did not observe a corresponding increase at PLOS Computational Biology (Fig. 2B, bottom). Compared to Pubmed authors, ISCB honorees have a higher proportion of individuals whose last names we associated with selecting white as a race/ethnicity category in the US census (Fig. 2C vs. A). Separating honoree results by honor category did not reveal any clear differences (Fig. 2D).

We directly compared honoree and author results from 1997 to 2020 for the predicted proportion of white, Asian, and other categories (Fig. 2E). We found that white honorees have been significantly overrepresented and Asian honorees have been significantly underrepresented in most years.

Predicting Nationality with LSTM Neural Networks and Wikipedia

We next aimed to predict the nationality of honorees and authors. We constructed a training dataset with more than 700,000 name-nationality pairs by parsing the English-language Wikipedia. We trained a LSTM neural network on n-grams to predict nationality. We found similar performance across 1, 2, and 3-grams; however, the classifier required fewer epochs to train with 3-grams so we used this length in the model that we term Wiki2019-LSTM. Our Wiki2019-LSTM returns, for each given name, a probability of that name originating from each of the specified eight regions. We observed a multiclass area under the receiver operating characteristic curve (AUC) score of 95.4% for the classifier, indicating that the classifier can recapitulate name origins with high sensitivity and specificity. For each individual region, the high AUC (above 94%, Fig. 3A) suggests that our classifier was sufficient for use in a broad-scale examination of disparities. We also observed that the model was well calibrated (Fig. 3B). We also examined potential systematic errors between pairs of nationality groupings with a confusion heatmap and did not find off-diagonal enrichment for any pairing (Fig. 3C).

Assessing the Nationality Diversity of Authors and Honorees

We applied our Wiki2019-LSTM model to both our computational biology honorees dataset and our dataset of corresponding authors. We found that the proportion of authors in the Celtic English categories had decreased (Fig. 4A, left), particularly for papers published in Bioinformatics and BMC Bioinformatics (see notebook). Among keynote speakers and fellows we found that the majority of honorees are predicted to be from Celtic English countries (4A, right). Though sample sizes were small, we did observe some differences in the composition of minority groups between meetings. ISMB keynotes had more probability attributable to Israel, while RECOMB had more attributable to East Asian countries (see notebook). When we directly compared honoree composition with PubMed, we observed discrepancies between the two groups, namely a large overrepresentation of CelticEnglish (including American) keynote speakers and a substantial underrepresentation of East Asian keynote speakers (4B). Outside of the primary range of our analyses, the two names of 2020 PSB keynote speakers were predicted to be of Celtic English origin (65% probability) and African origin (99% probability), respectively.

Conclusions

A major challenge that we faced in carrying out this work is to predict precise geographic origins for some groups of names. For example, we were unable to construct a classifier that could distinguish between names from Iberia (Spain and Portugal) and names from Spanish and Portuguese-speaking countries in Latin America. Discrepancies in representation between these groups are thus undetectable by our classifier. Hispanic honoree counts are influenced from Spain as well as Latin America. In these cases, our analyses may substantially understate the extent to which minoritized scientists are underrepresented among honorees and authors.

Biases in authorship practices may also result in our underestimation of the composition of minoritized scientists within the field. We estimate the composition of the field using corresponding author status, but in neuroscience [18] and other disciplines [19] women are underrepresented among such authors. Such an effect would cause us to underestimate the number of women in the field. Though this effect has been studied with respect to gender, we are not aware of similar work examining race, ethnicity, or nationality.

An important questions to ask when measuring representation is what the right level of representation is. We suggest that considering equity may be more appropriate than strictly diversity. In addition to holding fewer corresponding authorship positions, on average, female scientists of different disciplines are cited less often [20], invited by journals to submit papers less often [19], suggested as reviewers less often [22], and receive significantly worse review scores [21]. Societies, both through their honorees and the individuals who deliver keynotes at their meetings, can play a positive role in improving the presence of female STEM role models, which, for example, may lead to higher persistence for undergraduate women in geoscience [23]. Efforts are underway to create Wikipedia entries for more female [24] and black, Asian, and minority scientists [25], which can help early-career scientists identify role models. We find that ISCB’s honorees and keynote speakers, though not yet reaching gender parity, appear to be more evenly split between men and women than the field as a whole. On the other hand, honorees include significantly fewer people of color than the field as a whole, and Asian scientists are dramatically underrepresented among honorees. Although we estimate the fraction of non-white and non-Asian authors to be relatively similar to the estimated honoree rate, we note that both are represented at levels substantially lower than in the US population. Societies can play a positive role in enhancing equity if they design policies to honor scientists in ways that counter these biases.

The central role that scientists play in evaluating each other and each other’s findings makes equity critical. Even many nominally objective methods of assessing excellence (e.g., h-index, grant funding obtained, number of high-impact peer-reviewed publications, and total number of peer-reviewed publications) are subject to the bias of peers during review. These could be affected by explicit biases, implicit biases, or pernicious biases in which a reviewer might consider a path of inquiry, as opposed to an individual, to be more or less meritorious based on the reviewer’s own background [1]. Our efforts to measure the diversity of honorees in an international society suggests that, while a focus on gender parity may be improving some aspects of diversity among honorees, contributions from scientists of color are underrecognized.

Data and Resource Availability

References

1. Topic choice contributes to the lower rate of NIH awards to African-American/black scientists
Travis A. Hoppe, Aviva Litovitz, Kristine A. Willis, Rebecca A. Meseroll, Matthew J. Perkins, B. Ian Hutchins, Alison F. Davis, Michael S. Lauer, Hannah A. Valantine, James M. Anderson, George M. Santangelo
Science Advances (2019-10) https://doi.org/gghp8t
DOI: 10.1126/sciadv.aaw7238 · PMID: 31633016 · PMCID: PMC6785250

2. Ten Simple Rules to Achieve Conference Speaker Gender Balance
Jennifer L. Martin
PLoS Computational Biology (2014-11-20) https://doi.org/gf853n
DOI: 10.1371/journal.pcbi.1003903 · PMID: 25411977 · PMCID: PMC4238945

3. How scientists are fighting against gender bias in conference speaker lineups
Katie Langin
Science (2019-02-11) https://doi.org/gghp8v
DOI: 10.1126/science.caredit.aaw9742

4. Speaking out about gender imbalance in invited speakers improves diversity
Robyn S Klein, Rhonda Voskuhl, Benjamin M Segal, Bonnie N Dittel, Thomas E Lane, John R Bethea, Monica J Carson, Carol Colton, Susanna Rosi, Aileen Anderson, … Anne H Cross
Nature Immunology (2017-05) https://doi.org/gghp8s
DOI: 10.1038/ni.3707 · PMID: 28418385 · PMCID: PMC5775963

5. Addressing the underrepresentation of women in mathematics conferences
Greg Martin
arXiv (2015-02-24) https://arxiv.org/abs/1502.06326

6. The Presence of Female Conveners Correlates with a Higher Proportion of Female Speakers at Scientific Symposia
Arturo Casadevall, Jo Handelsman
mBio (2014-01-07) https://doi.org/qsh
DOI: 10.1128/mbio.00846-13 · PMID: 24399856 · PMCID: PMC3884059

7. Trends in the Proportion of Female Speakers at Medical Conferences in the United States and in Canada, 2007 to 2017
Shannon M. Ruzycki, Sarah Fletcher, Madalene Earp, Aleem Bharwani, Kirstie C. Lithgow
JAMA Network Open (2019-04-12) https://doi.org/gghp8r
DOI: 10.1001/jamanetworkopen.2019.2103 · PMID: 30977853 · PMCID: PMC6481599

8. Are gender gaps due to evaluations of the applicant or the science? A natural experiment at a national funding agency
Holly O Witteman, Michael Hendricks, Sharon Straus, Cara Tannenbaum
The Lancet (2019-02) https://doi.org/djc5
DOI: 10.1016/s0140-6736(18)32611-4

9. Race, Ethnicity, and NIH Research Awards
D. K. Ginther, W. T. Schaffer, J. Schnell, B. Masimore, F. Liu, L. L. Haak, R. Kington
Science (2011-08-18) https://doi.org/csf8j8
DOI: 10.1126/science.1196783 · PMID: 21852498 · PMCID: PMC3412416

10. Gender, Race/Ethnicity, and National Institutes of Health R01 Research Awards
Donna K. Ginther, Shulamit Kahn, Walter T. Schaffer
Academic Medicine (2016-08) https://doi.org/f8zzw4
DOI: 10.1097/acm.0000000000001278 · PMID: 27306969 · PMCID: PMC4965301

11. Research in Computational Molecular Biology
Mona Singh (editor)
Lecture Notes in Computer Science (2016) https://doi.org/gghp87
DOI: 10.1007/978-3-319-31957-5

12. Research in Computational Molecular Biology
Vineet Bafna, S. Cenk Sahinalp (editors)
Lecture Notes in Computer Science (2011) https://doi.org/dvwjgg
DOI: 10.1007/978-3-642-20036-6

13. Research in Computational Molecular Biology
Teresa M. Przytycka (editor)
Lecture Notes in Computer Science (2015) https://doi.org/gghwjh
DOI: 10.1007/978-3-319-16706-0

14. Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records
Kosuke Imai, Kabir Khanna
Political Analysis (2016) https://doi.org/f8ntmv
DOI: 10.1093/pan/mpw001

15. Predicting Race and Ethnicity From the Sequence of Characters in a Name
Gaurav Sood, Suriyan Laohaprapanon
arXiv (2018-07-31) https://arxiv.org/abs/1805.02109

16. Name-ethnicity classification from open sources
Anurag Ambekar, Charles Ward, Jahangir Mohammed, Swapna Male, Steven Skiena
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’09 (2009) https://doi.org/fs3pr8
DOI: 10.1145/1557019.1557032

17. Nationality Classification Using Name Embeddings
Junting Ye, Shuchu Han, Yifan Hu, Baris Coskun, Meizhu Liu, Hong Qin, Steven Skiena
Proceedings of the 2017 ACM on Conference on Information and Knowledge Management - CIKM ’17 (2017) https://doi.org/ggjc78
DOI: 10.1145/3132847.3133008

18. Persistent Underrepresentation of Women’s Science in High Profile Journals
Yiqin Alicia Shen, Jason M. Webster, Yuichi Shoda, Ione Fine
Cold Spring Harbor Laboratory (2018-03-02) https://doi.org/cmh5
DOI: 10.1101/275362

19. The gender gap in science: How long until women are equally represented?
Luke Holman, Devi Stuart-Fox, Cindy E. Hauser
PLOS Biology (2018-04-19) https://doi.org/gdb9db
DOI: 10.1371/journal.pbio.2004956 · PMID: 29672508 · PMCID: PMC5908072

20. The extent and drivers of gender imbalance in neuroscience reference lists
Jordan D. Dworkin, Kristin A. Linn, Erin G. Teich, Perry Zurn, Russell T. Shinohara, Danielle S. Bassett
arXiv (2020-01-07) https://arxiv.org/abs/2001.01002

21. Gender differences in peer review outcomes and manuscript impact at six journals of ecology and evolution
Charles W. Fox, C. E. Timothy Paine
Ecology and Evolution (2019-03) https://doi.org/gfwjjb
DOI: 10.1002/ece3.4993 · PMID: 30962913 · PMCID: PMC6434606

22. Journals invite too few women to referee
Jory Lerback, Brooks Hanson
Nature (2017-01) https://doi.org/gf4jjz
DOI: 10.1038/541455a · PMID: 28128272

23. Role modeling is a viable retention strategy for undergraduate women in the geosciences
Paul R. Hernandez, Brittany Bloodhart, Amanda S. Adams, Rebecca T. Barnes, Melissa Burt, Sandra M. Clinton, Wenyi Du, Elaine Godfrey, Heather Henderson, Ilana B. Pollack, Emily V. Fischer
Geosphere (2018-10-31) https://doi.org/gghp9d
DOI: 10.1130/ges01659.1

24. Why we’re editing women scientists onto Wikipedia
Jess Wade, Maryam Zaringhalam
Nature (2018-08-14) https://doi.org/gdz52z
DOI: 10.1038/d41586-018-05947-8

25. Why we’re creating Wikipedia profiles for BAME scientists
Nicola O’Reilly
Nature (2019-03-07) https://doi.org/gfwhcr
DOI: 10.1038/d41586-019-00812-8

26. Open collaborative writing with Manubot
Daniel S. Himmelstein, Vincent Rubinetti, David R. Slochower, Dongbo Hu, Venkat S. Malladi, Casey S. Greene, Anthony Gitter
PLOS Computational Biology (2019-06-24) https://doi.org/c7np
DOI: 10.1371/journal.pcbi.1007128 · PMID: 31233491 · PMCID: PMC6611653

Region	Training Size	Example Names
Celtic English	280,644	Julie S. Miller, Jesse A. Livezey, Jeremy C Simpson, Chris Smith, Thomas M Drudge
European	188,918	Sven Poths, Céline Feillet, Frederik Otzen Bagger, Lars I. Leichert, Sebastian MB Nijman
Hispanic	66,391	Beatriz Peñalver Bernabé, Diego Miranda-Saavedra, Marcelo Lobosco, Euler Guimarães Horta, Edgar E Vallejo-Clemente
East Asian	54,197	Jee-Hyub Kim, Yoriko Takahashi, Xiaohua Xu, Xuehai Zhang, Yoshihiro Noguchi
Muslim	30,703	Mohammad R. K. Mofrad, Fikret Ercal, Mehdi Yousfi Monod, Ghazaleh Taherzadeh, Noora Al Muftah
South Asian	20,025	Mahender Kumar Singh, Vidhu Choudhary, Suraj Pradhan, Ramakant Sharma, Vinod Menon
African	16,105	Samuel A Assefa, Nyaradzo M. Mgodi, Stanley Kimbung Mbandi, Oyebode J Oyeyemi, Ezekiel Adebiyi
Israeli	4,549	Tal Vider-Shalit, Itsik Pe’er, Michal Lavidor, Yoav Gothilf, Dvir Netanely

Authors

Abstract