A guest post from @iGrrrl
Like winter, changes to the biosketch are coming
Dr. Rockey spoke about the changes to the biographical sketch at the NORDP meeting this week, and I think I can at least offer a bit more depth about the thinking behind this, both from her comments and from what I've seen over the last few years. Certainly my knee-jerk negative opinion about this change has evolved upon reflection and listening to her presentation. This may not be as bad as it sounds. Maybe.
In her talk, of which this question of biosketches was one very small part, her short-hand way of referring to the reasoning behind this was to "reduce the reliance on publishing in one-word-named journals" as a shorthand to judging the quality of an investigator's productivity. When the biosketch was changed in 2010, shortening the publication list to 15 seemed to me to be designed to reduce a senior investigator's advantage of sheer numbers of publications. The rise of metrics and h-factor means that the impact factor of the journal in which the work was published now substitutes, in many a reviewer's mind, as the quick heuristic for assessing the Investigator criterion.
The move to the shorter publication list was also borrowed from NSF's limit of 10 products for the biosketch. This sounds good on paper, but didn't account for the differences in culture. Researchers in NSF-type fields are just as conscious of h-index, but you don't find the same reliance on "glamour magazines" that cut across all NSF research. The result seems to be that many young investigators in biomedicine feel they have to wait to publish until they have a story worthy of C/N/S. I hear sometimes about young researchers failing to make tenure in part because they did not publish enough, not because they didn't have data, but because they were trying for the high-level journal, didn't simply get it out in field- or sub-field-specific journals.
And work that appears in those so-called lower-tier journals shouldn't be dismissed, but it often is effectively ignored when a reviewer's eyes are looking for the titles of the high-impact journals. If a young faculty member's list maxes to 15 and they are all solid papers in reasonable journals, that's usually fine. But sometimes they have fewer than 15, so the reviewer relies more on the impact factor of the journals in which the work appears, and that in turn leads to reliance on C/N/S (JAMA, NEJM, etc). But for the applicant, sometimes the work reflected in the papers is based on a study that simply takes a long time to run, so that one paper in that year might represent a great deal of effort and time with results highly relevant within the context of the subfield. Or a series of papers may have methods published in one journal, the study in two more, and none of them are top-tier, but the entire story is important. This new narrative gives the opportunity to give that context.
This appears to be the point of the change to the biosketch: the impact factor of the journal(s) in which the work appeared may not reflect the impact of the results. Some applicants were including a sentence after every paper on the biosketch to try to give the context and impact--the contribution to the field--but in my experience, reviewers did not like and did not read these sentences. Yet, when reviewers come from a diversity of backgrounds, they may not be able to appreciate the impact of a result on the sub-field. Many of these concerns have been vociferously expressed to Dr. Rockey through various social media, primarily comments here at Our Host's blot, but also on the RockTalking blog.
The idea behind this new approach to discussing an applicant's contributions has some reasonable foundations, but I don't expect it will work. In the short term, applicants will likely struggle to assemble a response to this new requirement. I can't imagine reviewers will enjoy reading the resulting narratives. It may be that a common rubric approach to writing these sections as a clear story will make them uniform enough for reviewers to quickly judge, but I fully expect they will still be looking for Cell, Nature, and Science.