Repost: Researching your CV

Oct 14 2008 Published by under Science Publication

A spate of hits on the old WordPress version of DrugMonkey directed me to a recent post on In the Pipeline by Derek Lowe. In this piece Lowe is discussing the relationship of Impact Factor and GlamourMag publication to the conduct of science. Topics in which you know I have a passing interest, DearReaders! Lowe ends with an interesting gentleman-scholar observation:

someone who runs every single experiment to slot into the next manuscript had better also be running the ones that they'd set up even if journals didn't exist, and we all still communicated by handwritten letters. Good science is still good science, whether it's published (or even if it's published!) in Science or not.

Consequently I thought I'd repost some thoughts of mine on researching your CV which originally appeared July 31, 2007.


Dr. Shellie discusses the semi self-destructive habit of analyzing your publication numbers, types, citation hits, etc in a recent post entitled "Citation Envy". Key point is:

A public service message, from me: checking your citations is not a good way to determine if your life has meaning. Neither is comparing your publication list to that of other people you know. However tempting, it only ends in distress.
You can, however, benefit from checking your citations (as well as other people's publication lists) IF AND ONLY IF you view it as an educational exercise. Try to see how other people have built upon and developed their early work in order to make progress in their field. How can you do it too?

I want to underline the part about viewing this as an educational exercise and point out that this is a critical step in career success. How so?


Despite what we would like to be the case, despite what should be the case, despite what is still the case in some cozy corners of a biomedical science career....let us face some facts.

  • The essential currency for determining your worth and status as a scientist is your list of published, peer reviewed contributions to the scientific literature.
  • The argument over your qualities between advocates and detractors in your job search, promotions, grant review, etc is going to boil down to pseudo quantification of your CV at some point
  • Quantification means analyzing your first author / senior author /contributing author pub numbers. Determining the impact factor of the journals in which you publish. Examining the consistency of your output and looking for (bad) trends. Viewing the citation numbers for your papers.
  • You can argue to some extent for extenuating circumstances, the difficulty of the model, the bad PI, etc but it comes down to this: Nobody Cares.

My suggestion is, if you expect to have a career you had better have a good idea of what the standards are. So do the research. Do compare your CV with those of other scientists. What are the minimum criteria for getting a job / grant / promotion / tenure in your area? What are you going to do about it? What can you do about it? Don't misunderstand me- nobody is going to hand you a job / grant / etc just because you hit the modal publication numbers. But it will be very easy for you to be pushed out of the running if you do NOT hit the expected values. So do what you can to keep your CV as competitive as possible.
Publish or Perish: Everyone knows the mantra because it is true. It is an essential component of just about any biomedical science career that you will go about publishing original observations on a consistent basis. So the first key is, at each and every stage, to do what you can to assure a bread-and-butter stream of data that is going to allow you to publish something on a regular basis. What does this mean? Varies by stage and subfield of course. But at least one publication each and every year is a decent starting point. Grad students should be discussing very frankly with the mentoring investigator how and when they are going to get on publications. Your CV research is used here to convince the well meaning but perhaps old fashioned PI that times are a changin' and you need pubs. Post docs, even more so. The point is not to be obnoxious or demanding just to be fully knowledgeable of the current standards and expectations of your field. Newly minted Independent Investigators? This analysis shapes your choices and balance of collaborations, pedestrian work and risky ventures.
Journal Impact Factor: The higher impact the better. Yeah, yeah. This is a stupid system of quantification, the actual structural science of the highest impact journals is bad, journal factors are driven by highly skewed citations rates, number depend on field size, etc, etc. But it is a reality. So know a little something about the impact "level" that is expected. Is the 2-4 range for your Society journal enough? Do you need to hit one in the 6-9 range consistently or occasionally? How about the tradeoff with the above yearly rate? Does a Science or Nature paper make up for a 4 year drought? This will shape your decisions about where to send papers. Is it worth the risk to shoot high with every manuscript and then step down the impact factor food chain? Are you at best going to go from a 2 to a 4 and nobody considers this relevant? Does your field "expect" you to publish regularly in the society's journal even if the impact factor is less than exciting? Do your promotions committees think that because a journal is tops in the ISI category in which it resides this is better than a much higher impact factor that is way down the, say, general Neuroscience list? You need to know this. And, especially for trainees, you need to understand what it takes, scientifically, to get into a given level of journal. [As a sidebar, you want to understand the typical timeline from submission to acceptance to in-print publication for your usual journals. This helps with a consistent output. Which journals do you have a good chance of getting published with a given calendar year pub date if you submit it in Sept? Remember, you can't go back and fill in publication year holes, better to plan a bit in advance.]
Cites: The ISI is a wealth of data on your citation performance and you need to keep an eye to actual citations of your papers. [Of course you are doing this anyway because it is a good way to ensure you look at papers of interest to you that might be outside your usual PubMed searches, right?] How many cites do your top papers have? What is your h-index (h papers with at least h cites is the h-index)? Total cites? "Yes but surely there is nothing to be done about this, right" you say. Wrong. To quote a mentor, "If you don't cite your papers why would anyone else?". So first and foremost cite yourself liberally. Is this cheating? Perhaps. But ISI has some ways to parse this if someone is really interested. And most people won't be going this far. Less sleazily this is part of advertising your science- if someone is reading one paper, might they not be interested in another one? Well, make it easy for them! Other direct marketing techniques vary in effect and usage- it is worth mentioning that every so often I get an unsolicited reprint or email with "Hey did you know we're working on this" which I interpret as "Gee, would you cite us already?". I don't think I would ever do this, not my type of style. The other technique is to use the manuscript review process to insist that an author cite your work. Everyone has stories of the review that comes back with "The authors should cite X, Y and Z publications from CompetingLab". There are, of course, subtler ways to do this. It is fairly common and occasionally it works. Me, I tend to reserve this for only those cases in which the scholarship of the manuscript really demands it. I suppose the take-home here is simply that you shouldn't avoid suggestions that your work be cited when reviewing, given that such citations are appropriate.
Papers "in preparation" or "submitted": Don't do this if you can possibly avoid it is my advice. I don't like this type of vapor-ware. I mean, crikey, don't most investigators have a half dozen or so things that are almost papers that they just haven't gotten to yet? ...perhaps this is just me. But let's be honest, eh, Drugmonkey? People do this. Why? It must have currency somewhere. All I can say is you better have a half-decent manuscript ready to show someone if they ask for it...
A final note on motivation. Maybe not all of you are chronic procrastinators, chronic ball jugglers or in terminal need of deadlines. My experience suggests many, if not most, scientists express these traits. Think of some of the above considerations to be giving you the extra deadlines that you need. As in "Oh crap, I'm not going to have a pub in 2007. Okay, I better spend the next couple of days on that almost-paper that I've been neglecting and just grind it out". Or, "Okay, I really have to initiate that collaboration so that I can upgrade this story to a 8-10 impact factor journal".

27 responses so far

  • NM says:

    "H Indexers anonymous"
    I'm trying to get a handle on what a good H index might be given that the whole thing is hopelessly confounded by seniority (the older your papers the higher your H factor).
    I'm 3 years postdoc and my H index is 5. Is this a good or a bad thing?

  • whimple says:

    I like impact factors. Here's my classification:
    super-premium: anything with an IF 20+. the usual pan-field single word titles and one or two more recent two-word pretenders. These puppies get post-docs jobs and get junior faculty tenure.
    A-class: Anything from PNAS on up (IF 10+). Excellent. Nobody can ever criticize you meaningfully for publishing here. These can also get people jobs and/or tenure. You don't just need the crazy-high journals.
    B-class: Anything JBC or higher (IF 6+). Very respectable. Totally rock-solid. Good bread-and-butter entries. People can have careers publishing here.
    C-class: Anything IF 4.0 up to but not including JBC. It's ok. If you have good work that just isn't sexy or exciting enough to make it to B-class, put it here. Ok once in a while to clean up your scientific story-board, but you shouldn't make a regular habit of it.
    D-class: (unacceptable). IF less than 4.0. If no one is going to cite it, why do you bother? Not helpful to a career. I know a lot of society journals are in this range. Not an excuse unless you're a high up tenured mucky-muck in the society and publish here as charity to the society. If it's here, or stuck in a dissertation only, put it here.
    I mostly consider all the journals in a given class as essentially the same. I won't fight to "upgrade" a paper within a class. I will fight like crazy to not have to drop down into the next lower class.

  • chall says:

    well, I am not sure that IF is all that depending on what your field is.... my old field was veterinarian.... microbiology... let's just say that it is very hard to go above 10. And most journals would be between 2.5 and 6... Infection and Immunity for example, or vaccine - none of them are IF 6.
    But sure, I agree in general. It's just all about trying to find that "mechanism" that can be adapted into several fields maybe?!

  • Alex says:

    Papers in preparation are important for a postdoc who's been there a little over a year, has a bunch of results in the pipeline, and wants to hit the job market.

  • Beaker says:

    Papers in preparation are important for a postdoc who's been there a little over a year, has a bunch of results in the pipeline, and wants to hit the job market.
    I'm looking at a CV from a job applicant posdoc right now. There's 5 manuscripts listed that are "in preparation for Journal X." This is more than the total number of papers on the CV. Not helpful. In this case, stay off the job market and get those papers published. If you do list papers which are in preparation or submitted, mentioning the journal appears pathetic.

  • cookingwithsolvents says:

    My CV for job applications has my most recent paper as "submitted" (and it was, several months ago...I hope I can change the status soon). I felt it was appropriate given the title of the paper mirrors my graduate research.
    I did not list anything from my postdoc, though I [i]do[/i] have things in preparation (I know, I know....so does everybody else). I just don't feel comfortable listing postdoc papers that aren't submitted yet. However, if a paper is submitted SOMEBODY is reviewing it somewhere so I feel like you should include it. Thus I'll be submitting at least one paper in the next few weeks.

  • crystaldoc says:

    Question for whimple, DM, and others who use IF to decide where to submit: to what extent do year-to-year changes and trends in IF influence your decisions? PNAS is down to 9.64 in 2007; does it remain the benchmark for A-class? JBC is down to 5.58; does it remain the benchmark for B-class? My favorite society journal was 4-ish a few years ago, but has had a big downhill slide to 3.37. At what point do I decide a journal isn't classy enough for my work anymore? Part B of the question is why are these journals slipping in the rankings? The papers they publish do not seem to slipping any in quality or interest. Is it due to competition from new journals? Or are there other established journals that are climbing in the rankings? I haven't noticed this to be the case.

  • anon says:

    Hmm. I have just done a job application (I am also a postdoc) where I listed one paper in prep. to highlight the fact that I have done work in that area (it ties in very neatly with their interests) - and if they want to discuss or see the ms they can. But too many in preps seems just too try hard.
    Also re IF: I think this varies alot with discipline. Very few people in my field would score in a journal over 10. To do so would put you totally in the front running for a job. Anything over 2 is highly respectable!!

  • pinus says:

    I think that putting something 'in prep' that is a new direction for you that relates to some of the ongoing work at the institution you are applying to makes sense. In fact, I did the same thing when I was on the job market, and I think it actually helped..in the sense that folks saw that I was branching out. sure, one could wait until something is in press, but the early bird gets the worm.
    Also:
    I wonder if PNAS's IF is dropping because of all the subpar 'contributed' papers from members?

  • Alex says:

    I erred in my earlier post. When I was on the job market I only listed papers "under review", not "in preparation." After a year of postdoc I had 2 grad school papers published, 1 grad school paper under review, 1 postdoc paper published, and a few postdoc papers under review. I hit the job market at that point, included the "under review" papers on the CV (listed as such), and I'm an assistant professor now.

  • crystaldoc says:

    "I wonder if PNAS's IF is dropping because of all the subpar 'contributed' papers from members?"
    Them's fightin' words! My most awesomest ever papers were 'contributed' to PNAS by my NAS-member postdoc advisor. Besides, anecdotes aside, hasn't the proportion of "track 2" papers grown substantially in recent years? If anything, a larger proportion of the papers are being rigorously peer reviewed, which (one would think) should increase the overall impact.

  • CC says:

    D-class: (unacceptable). IF less than 4.0. If no one is going to cite it, why do you bother? Not helpful to a career.
    Nowadays, going to PLoS One with something like that is the most flattering option. I'd never suggest passing up a Nature Family publication over Open Access principle, but when you get into whimple's C and D territory, Open Access is pragmatism, pure and simple.
    As to why: the obvious reason is to allow that student to graduate with a masters, right?
    Question for whimple, DM, and others who use IF to decide where to submit: to what extent do year-to-year changes and trends in IF influence your decisions?...My favorite society journal was 4-ish a few years ago, but has had a big downhill slide to 3.37.
    I wouldn't worry about that kind of resolution at all. At that point, just rely on your own sense of snobbery to guide you. The issue is 12 versus 5, and the like.

  • ScienceWoman says:

    I find this IF discussion all very interesting and very arcane...the top journals in my field have an IF of

  • ScienceWoman says:

    Oops. Used a less than sign. Comment should say "the top journals in my field have an IF of less than 2.5"

  • River Tam says:

    ScienceWoman - I hear you. However, their meetings are also an order of magnitude bigger. More people, more citations.

  • DrugMonkey says:

    no worries River Tam and SW, whimple @#2 simply reveals the unfortunately common (hi PP!) inability to understand that science, even biomedical science, is much broader than one's own perspective. This was one of the main points of my post, that you have to understand what the standards are in your own field. There are tons of people having continual NIH funded careers who publish in the 2-4 IF range with the occasional 6-10 now and again. Some very established types in drug abuse, for example, appear not to give a hoot about IF and just put everything they publish in one or two less-than-4.0 IF journals.
    Times change, however, and the younger folks need to keep a sharp eye on the places they publish.

  • DrugMonkey says:

    More people, more citations.
    oh yes, and I forgot. This is only the most obvious flaw in the IF but it is well recognized and a huge contributor if I remember the analyses aright. People often make a half-way acknowledgment of this fact....and then go right on using IF. It can be maddening.

  • I share SciWo's and DM's caveats about IF. When one is in a small field and your society journal has an IF

  • whimple says:

    The IF is a convenient shortcut, which is why it's used. Look at how IF is calculated: number of citations in the 2 years following publication. That's fine to publish in your society's 2.0 IF journal, and sure journals can manipulate the IF and it's not perfect by any means, but if your papers really are actually only going to be cited a total of two times, then for all practical purposes nobody cares about your work, including those in your field, and you should go home and release your research money to someone doing work people actually care about.

  • no worries River Tam and SW, whimple @#2 simply reveals the unfortunately common (hi PP!) inability to understand that science, even biomedical science, is much broader than one's own perspective.

    Where do you get the cockamamie idea that I have an "inability to understand" the breadth of biomedical science? I write about my own experiences, and I wear my perspective on my sleeve.
    You wanna know the gory truth about what it takes to succeed in the traditional post-doc-to-tenure-track-to-tenure progression in basic biomedical science at a top-ten medical school? Then listen to Comrade PhysioProf.
    You wanna know how to write a fundable R01? Then listen to Comrade PhysioProf.
    You wanna know how to manage a basic science laboratory? Then listen to Comrade PhysioProf.
    You wanna know how to publish in high-IF biomedical journals? Then listen to Comrade PhysioProf.
    I don't talk about other shit, because I don't know anything about it. Sheesh, holmes! Between you and motherfucking dumbshit fuckwad Sol, I got one hook in my mouth and one in my ass!
    HAHAHAHAHAHAHAH!!!!!!!!!!!!!!!!!!!!!!! Trolled on my own motherfucking blog!!!!!

  • BiophysicsMonkey says:

    I would be more understanding of the reliance on journal IFs if there were no other metric for paper quality available, but there is a very obvious one available: the # of citations for individual papers. This information is readily accessible and is clearly a more accurate indicator of a paper's impact than the journal it's published in.
    If someone publishes a string of JBC papers that are each cited hundreds of times, they are clearly making a bigger impact on their field than someone who publishes in Nature and gets ignored (and yes, plenty of Science and Nature papers are sparsely cited).
    I don't see the need to use journal IF as a proxy when the real data for individual papers is so easily available.

  • S. Rivlin says:

    A recent Editorial in Science actually criticizes the whole IF crap. I agree with BiophysicsMonkey that the number of citations of individual papers is a much better metric of impact.
    The only "how" I learned from the Comrade is how a foul mouth and pomposity impact negatively on the impression his readers have about a scientist who's funded by NIH, serves in a top 10 medical school, manages a basic science lab and publishes in high IF journals.

  • River Tam says:

    My comment to ScienceWoman was meant in the context of different fields having different impact factors and not to get hung up on the fact that your impact factors are shockingly large (no jokes please). I see using impact factors, h-indexes, and other metrics as useful guides to inform decision making processes about our careers - assuming your goal is to work in a research-focus environment. But they are kinda like the "Pirate Code" - more like guidelines than fixed rules. Impact factor is roughly related to the "prestige" of a journal within a field - I'll let others argue the chicken/egg of impact vs prestige. In my field, at least, an article in our top tier of journal (i.e. tier below S/N/PNAS) indicates that the topic is probably dealing with the leading edge of research, the methods were sound, and the results strong). Next tier - perhaps less ideal methods/results, still cutting edge questions, etc etc etc. Does a paper in a higher-tier journal insure that it will have a huge impact on the field? Obviously not. Does it indicate something about how other scientists viewed the type of question being asked and the interesting nature/solidity of the results? I think yes. But again, guidelines, not rules.

  • whimple says:

    I would be more understanding of the reliance on journal IFs if there were no other metric for paper quality available, but there is a very obvious one available: the # of citations for individual papers. This information is readily accessible and is clearly a more accurate indicator of a paper's impact than the journal it's published in.
    That's great if you don't mind waiting two or three years to see how many times your papers get cited. Some people (post-docs looking for jobs, assistant professors trying to get tenure) are on the clock and can't wait that long. Publishing in a high IF journal is a nice, immediately rewarding, substitute.

  • NM says:

    Sol
    Whilst I agree that IF is kind of crappy relying on individual citations to papers has a more serious problem.
    Much like the H index it's hopelessly confounded by the age of the paper/investigator. This is great for you as you can (hypothetically) claim lots of papers with 50+ citation counts because your seniority allows this to have happened.
    I (postdoc) on the other hand might have published the most important paper in my field this year but it's cited only twice so far. Ten years from now we might know the value of a paper published this year via the use of individual paper citation counts. Until then we can only rely on the IF to give us a rough idea of how it will probably fare.

  • [...] I have recommended shooting for consistent output, being concerned with whether a manuscript submitted to Journal X at this point in the year will [...]

Leave a Reply