Archive for the 'Ask DrugMonkey' category

Why aren't they citing my papers?

As the Impact Factor discussion has been percolating along (Stephen Curry, Björn Brembs, YHN) it has touched briefly on the core valuation of a scientific paper: Citations!

Coincidentally, a couple of twitter remarks today also reinforced the idea that what we are all really after is other people who cite our work.
Dr24hrs:

More people should cite my papers.

I totally agree. More people should cite my papers. Often.

AmasianV:

was a bit discouraged when a few papers were pub'ed recently that conceivably could have cited mine

Yep. I've had that feeling on occasion and it stings. Especially early in the career when you have relatively few publications to your name, it can feel like you haven't really arrived yet until people are citing your work.

Before we get too far into this discussion, let us all pause and remember that all of the specifics of citation numbers, citation speed and citation practices are going to be very subfield dependent. Sometimes our best discussions are enhanced by dissecting these differences but let's try not to act like nobody recognizes this, even though I'm going to do so for the balance of the post....

So, why might you not be getting cited and what can you do about it? (in no particular order)

1) Time. I dealt with this in a prior post on gaming the impact factor by having a lengthy pre-publication queue. The fact of the matter is that it takes a long time for a study that is primarily motivated by your paper to reach publication. As in, several years of time. So be patient.

2) Time (b). As pointed out by Odyssey, sometimes a paper that just appeared reached final draft status 1, 2 or more years ago and the authors have been fighting the publication process ever since. Sure, occasionally they'll slip in a few new references when revising for yet the umpteenth time but this is limited.

3) Your paper doesn't hit the sweet spot. Speaking for myself, my citation practices lean this way for any given point I'm trying to make. The first, best and most recent. Rationale's vary and I would assume most of us can agree that the best, most comprehensive, most elegant and all around most scientifically awesome study is the primary citation. Opinions might vary on primacy but there is a profound sub-current that we must respect the first person to publish something. The most-recent is a nebulous concept because it is a moving target and might have little to do with scientific quality. But all else equal, the more recent citations should give the reader access to the front of the citation thread for the whole body of work. These three concerns are not etched in stone but they inform my citation practices substantially.

4) Journal identity. I don't need to belabor this but suffice it to say some people cite based on the journal identity. This includes Impact Factor, citing papers on the journal to which one is submitting, citing journals thought important to the field, etc. If you didn't happen to publish there but someone else did, you might be passed over.

5) Your paper actually sucks. Look, if you continually fail to get cited when you think you should have been mentioned, maybe your paper(s) just sucks. It is worth considering this. Not to contribute to Imposter Syndrome but if the field is telling you to up your game...up your game.

6) The other authors think your paper sucks (but it doesn't). Water off a duck's back, my friends. We all have our opinions about what makes for a good paper. What is interesting and what is not. That's just the way it goes sometimes. Keep publishing.

7) Nobody knows you, your lab, etc. I know I talk about how anyone can find any paper in PubMed but we all need to remember this is a social business. Scientists cite people they know well, people they've just been chatting with at a poster session and people who have just visited for Departmental seminar. Your work is going to be cited more by people for whom you/it/your lab are most salient. Obviously, you can do something about this factor...get more visible!

8) Shenanigans (a): Sometimes the findings in your paper are, shall we say, inconvenient to the story the authors wish to tell about their data. Either they find it hard to fit it in (even though it is obvious to you) or they realize it compromises the story they wish to advance. Obviously this spans the spectrum from essentially benign to active misrepresentation. Can you really tell which it is? Worth getting angsty about? Rarely.....

9) Shenanigans (b): Sometimes people are motivated to screw you or your lab in some way. They may feel in competition with you and, nothing personal but they don't want to extend any more credit to you than they have to. It happens, it is real. If you cite someone, then the person reading your paper might cite them. If you don't, hey, maybe that person will miss it. Over time, this all contributes to reputation. Other times, you may be on the butt end of disagreements that took place years before. Maybe two people trained in a lab together 30 years ago and still hate each other. Maybe someone scooped someone back in the 80s. Maybe they perceived that a recent paper from your laboratory should have cited them and this is payback time.

10) Nobody knows you, your lab, etc II, electric boogaloo. Cite your own papers. Liberally. The natural way papers come to the attention of the right people is by pulling the threads. Read one paper and then collect all the cited works of interest. Read them and collect the works cited in that paper. Repeat. This is the essence of graduate school if you ask me. And it is a staple behavior of any decent scientist. You pull the threads. So consequently, you need to include all the thread-ends in as many of your own papers as possible. If you don't, why should anyone else? Who else is most motivated to cite your work? Who is most likely to be working on related studies? And if you can't find a place for a citation....

16 responses so far

More data on historical success rates for NIH grants

Jul 11 2012 Published by under Ask DrugMonkey, NIH, NIH Careerism, NIH funding

Thanks to a query from a reader off the blog and a resulting request from me, our blog-friend microfool pointed us to some data. Since I don't like Tables, and the figure on the excel file stinks, here is a different graphical depiction:

The red trace depicts success rates from 1962 to 2008 for R01 equivalents (R01, R23, R29, R37). Note that they are not broken down by experienced/new investigators status, nor are new applications distinguished from competing continuation applications. The blue line shows total number of applications reviewed...which may or may not be of interest to you. [update 7/12/12: I forgot to mention that the data in the 60s are listed as "estimated" success rates.]

The bottom line here is that looking at the actual numbers can be handy when playing the latest round of "We had it tougher than you did" at the w(h)ine and cheese hour after departmental seminar. Success rates end at an unusually low point...and these numbers stop in 2008. We're seeing 15% for R01s (only) in FY2011.

Things are worse than they've ever been and these dismal patterns have bee sustained for much longer. If we look at the ~30% success rates that ruled the day from 1980-2003, the divergence from the trend from about 1989 to 1996 was interrupted in the middle and, of course, saw steady improvement in the latter half. The badness that started in FY2004 has been 8 unrelieved Fiscal Years and shows no sign of abatement. Plus, the nadir (to date) is much lower.

Anyone who tries to tell you they had it as hard or harder at any time in the past versus now is high as a kite. Period.

Now, of course, it IS true that someone may have had it more difficult in the past than they do now, simply because it has always been harder for the inexperienced PIs to win their funding.

RPGsuccessbyYear.png
source
As we know from prior posts, career-stage differences matter a LOT. In the 80s when the overall success rate was 30%, you can see that newcomers were at about 20% and established investigators were enjoying at least a 17%age point advantage (I think these data also conflate competing continuation with new applications so there's another important factor buried in the "Experienced" trace.) Nevertheless, since the Experienced/New gap was similar from 1980 to 2006, we can probably assume it held true prior to that interval as well.

12 responses so far

NIDA and NIAAA Kill the K05 Senior Scientist Research Award

Jun 26 2012 Published by under Ask DrugMonkey, Careerism, NIH, NIH Careerism, NIH funding

Per NOT-DA-12-016,

NIDA and NIAAA will not accept any K05 applications beyond the November 2012/ January 2013 receipt dates.

For those unfamiliar, the latest PA-12-148 FOA for the K05 is here and the purpose is listed as:

The purpose of the Senior Scientist Research (K05) is intended to provide protected time for outstanding senior scientists who have demonstrated a sustained high level of productivity conducting biomedical research relevant to the scientific mission of the appropriate institute to focus on their research and to provide mentoring of new investigators.

The DM Executive Summary is, "Salary support for BigSchwangingTypes to relieve the burden on their research awards, covered by a faked-up need for buyout from local Institutional responsibilities with regard to teaching and service".

The junior faculty summary is, "What another total scam by which the GoodOldBoyes/Girlz extract yet MORE money out of the NIH grant system!"

Note: If it were ever possible, you are damn right I would have applied for one of these in a heartbeat and I'm bitter that it was never possible for me to get one of these schweeeeet deals :-).

So when I first saw this notice I just thought "Good!" and moved on.

However.

A reader of the blog wrote to ask for my take on this and further observed:

I am curious how this will impact these senior "NIDA" investigators and their current/future grant applications? If these K05 scientist now have to put more of their effort on their NIDA grants, what impact will that have on their grants, their trainees who are funded off their grants, etc.

I guess I am a little shocked, and happy, that they are finally sticking it to the senior scientists a little. This may indirectly help out us younger scientists?

First of all, I should make clear I have no insight into why NIDA and NIAAA have chosen to dismantle this program at this time. Haven't heard any rumours about it at all.

Charging on over to RePORTER I find that there are only 33 K05s funded between the two institutes (13 NIAAA) at present. This is not a tremendously large number. The total costs seem to run from about $120K per year up to about $250K...over a R03 equivalent on the low end and we're talking R21 territory for the bigger ones. I guess 30-50 more of the smaller R grants funded every year would be a good improvement.

Alternately, they could use these to offset one-module reductions for a larger number of R01 apps, or get perhaps 15 more R01s funded entirely. NIDA has about 90 new R01s funded in FY2012 (to date) and they funded about 122 new R01s in FY2011. Therefore, adding 10-15 more is a significant improvement. This could very well be the only reason. They are searching under the couch cushions for a way, any way, to keep the R-mech success rates up.

Getting back to the emailer's suggestion that this is a way to stick it to the senior investigators....well, that isn't quite clear. The senior investigator salaries have to come from somewhere. And as I alluded to, I think the whole idea of buying out local institutional service time was a bit of a sham. The K05 serves, in part, to relieve the burden on the PIs' R grants. If they didn't have the K05 buyout, they'd be looking to land another R grant to get the same percentage of their salary covered.

They will return to doing so (and here we have a nebulous, fluid population "they" which is made up of the current awardees and the next set of potential future K05 awardees). Putting in more R or P mechanism proposals that will compete with, you guessed it, everyone else. Including the junior investigators.

So it isn't the case that this magically frees up money for the younger set to obtain. It throws the money, yes, but also the Greybearded and Bluehaired Professors back into the R-mech pool.

10 responses so far

Protected Pockets of Time

In yesterday's discussion, I finally got a partial glimpse of the issue when NatC observed:

Discussions about how to manage and plan protected pockets of time OUTSIDE work to do whatever - walk the bulldog, play music, train for a triathlon, watch baseball, play with your kids or nieces/nephews ir travel - would be extremely valuable work/life balance discussions to have early in this sometimes crazy career.

In full disclosure this has rarely been a problem for me. I've managed to get to where I am today (such as it is) with what I think is a healthy balance of work-to-life. Obviously some, including my spouse, might disagree but the important thing is that I think this is the case. We're talking personal, subjective "balance" here and nobody can define it for you. If you have reached it, you are going to be relatively happier and if you feel imbalanced you are going to feel sad* about it.

Yes, I for damn sure wish for more hours in the day. Yes. Of course. And at each and every major stage there were things being neglected so that I could pursue some other thing. Either in the proximal, days to weeks, or in the long-haul, years to decades(!), perspective. But I have never been an obsessive and any fair read would fail to find any major imbalance.

How did I do it?

I think the most useful and general approach is that you have to be willing to fail.

Let me say it again: YOU HAVE TO BE WILLING TO FAIL!!!!!

I was not, I think, willing to fail at getting the PhD. This was a defined, obtainable target for which the steps were mostly clear to me. Do the research, write that shit up into a dissertation and bob's your uncle.

After that? Well, yes, of course I wanted to succeed career-wise. In one of the professorial paths preferably. But I was willing to...not. To fail.

There have been several defined choice points at which I did the considerably sub-optimal career move for the sake of issues that we shall encompass under "life". (Also career moves which might have in the long run been suboptimal but looked great** at the time. Some of this initial appearance was influenced by "life".) Sometimes I did this out of unthinking ignorance, I will admit. I didn't perhaps realize the magnitude of the risk I was running. But I for damn sure knew there was risk. Risk of not making it in some way. Of not getting on the independent research track. Of not getting funding...or not keeping it. Of letting the lab and research program crash down to nonviability.

This hasn't stopped and it continues to this day.

Is my virtue untested? Some might observe that. From the perspective of some it looks like I have a pretty schweet gig***. From above the waterline it looks okay. Something a disgruntled postdoc or Year 3 faculty member might think is pretty much IT. As in "career accomplished"...all it takes now is running it out like you always wanted to. No risk.

I don't see it that way. I still risk failures of various sorts. Mostly the big axe is the grant funding....and it is a big one, hanging over my head more often than it is not.

So much like the disgruntled postdoc and the terrified junior faculty member...I could always work harder. More. Put in more grants. Squeeze out more papers. Refine my lab efficiency to maximize the data. Chase small project funds. Woo more trainees. Hit the seminar circuit harder. Go to more meetings.

All of this would probably benefit my career. It would make things go better professionally. We'd be more productive, no doubt.

I choose not to. That's it. There's no secret. There's no special case of insulation from the risks of choosing not to work harder than the next person. You risk paying a price.

Balance implies tradeoffs. I've certainly found it to be so. There are costs to go with every benefit. Costs that may be "just" stress, may be health issues (mental or otherwise), may be definable career failures. Having "life" balance makes this inevitable. There will be tradeoffs****, people.

This is my answer to NatC's question. Choose. Choose to take the time. Make room for what is important to you. Realize that by doing so you might fail. You might.

But you know what? These St Kern and Poo types?

I know for damn sure they've failed at life.

And that I was never willing to risk.

__
*don't get a puppy to cheer yourself up.
**so we won't count these, at the time they seemed really pro-career.
***and I do, I do.
****of course it goes both ways. you may be choosing a career path that really isn't compatible with your desire to tour Europe with an opera group every summer. You may have to give up some of the "life" stuff

23 responses so far

The worst drivers on US roads are

The demonstrably worst drivers are 1) Beemer pilots, 2) Volvo drivers of the mom demographic, 3) jacked up pickemup trucks with trucknutz idiots and 4) minivan drivers. Lately the ecophreaks driving Prii are making a *strong* showing. But I suspect that is either the dope or the conversion of #2s to Prii.

Discuss.

42 responses so far

Idiot runners

Christ.

The notion that I have to be all #getoffylawn about the concept of fartlek pains me.

It is not intervals, you do not pluralize the word and you most certainly should not be throwing up at the end of the workout.

9 responses so far

On providing feedback: A simple poll

Apr 13 2012 Published by under #FWDAOTI, Ask DrugMonkey, Poll

Multiple choice, select as many as apply.....

8 responses so far

When should new PIs submit their second major grant?

Dec 20 2011 Published by under Ask DrugMonkey, Careerism, NIH, NIH Careerism

This question arose on the Twitts and the person wanted to know if waiting for the first renewal was a good idea.

In short, no.

First off, the days of counting on the NIH renewal application being funded, even given great productivity, are over. Tactically you need to shoot for overlapping awards just to ensure continuity of a one-R01 sized lab.

Second, it is my continued view that it is really hard to make it as a single grant lab in the long term. You need diversity of projects because one might run into doldrums now and again for reasons outside of your control.

The third point addresses a followup concern- can newb profs handle the load of two major projects? I say yes. I can't think of a single scientist who struggled and/or failed because they had two R01 support in the early years. I'm sure they exist but I can't think of any examples from my experiences. I do know a few folks lucky to get two major awards early (first 2-3 years) and they did (and are doing) just fine.

OTOH I can think of several examples of folks who struggled with just one award and for whom I think things would have been vastly better with two grants.

18 responses so far

PA! Huh. What's it good for? ....absolutely nothin! Say it again, PA!

A query to the blog is a very typical reaction of those new (and not so new) to the NIH Grant game. As you will see in my answer. First, the question:

I'm now confused about this whole Program Announcement thing. The PO said that my application would be judged normally, just as part of whatever else the study section was reviewing, and that there was no special money set aside for the PA. If that's the case, what's the point of the PA in the first place? I had been under the impression it would be judged with other grants responding to the PA, but apparently that's not true.

Help?

It is the RFA that generally routes applications into a dedicated, special emphasis panel type study section for review. For those Funding Opportunity Announcements (FOA) there is indeed a set aside pool of money and generally just a single receipt date. The PAS (Program Announcement with Set Aside Funds) also has dedicated funding, generally for the first round of submissions, then it converts to a regular old Program Announcement (PA) type FOA. Applications submitted for a PAR (Program Announcement with special receipt, referral and/or review considerations) is reviewed by a special panel, generally within an IC.

Regular old PAs are open for 3 years and generally use the standard receipt dates. In the ICs of my greatest experience they tend to be renewed and thus may represent essentially permanent PAs for much of your grant writing life. As per the reader query, the applications are reviewed in standard, CSR study sections with the appropriate domains of coverage and expertise. Alongside those applications that use the generic, mechanism based FOA. I would argue that you would only use the latter if you had to. Again, in the ICs of my greatest experience the PAs can be incredibly broad. Take "PA-10-268 Neuroscience Research on Drug Abuse (R01)" as an example. If your IC of interest has such broad topic PAs...you might as well use them.

Now as the reader question intimates, there is no overtly special benefit to your chances of getting funded. And there may be no benefit at all. Hard to tell. Because of course this sort of business only matters* when Program is considering the grey zone pickup funding. Is there a slant or a formula for how many approximately equivalently scored grants they will select under one of their PAs versus the generic parent FOA? I would suspect so, else why have such things? But I can't say for sure. Maybe it is just make work for Program staff....to lay out their priorities. Or maybe it is a defensive excuse for those rare cases when they decide to stiff a grant that came in under the payline "Sorry PI Squirrel, it didn't fit any of our Programmatic Interests...don't you read the PAs?".

The bottom line here for those new to the system is not to get all that excited when language in a PA seems directed at your research program. It isn't *that* good of a bennie. But you might as well have some idea what is in the PAs and respond to them when you can. Because you just never know when it might help.
__
*assuming you have a modicum of sense and are not submitting stuff that is clearly not going to be of interest under the generic R01 parent FOA.

11 responses so far

The reason I urge newbie profs to keep up with NIH funding solicitations is quite simple

Sep 15 2011 Published by under Ask DrugMonkey, NIH Careerism

No matter what esoteric small town grocer science you conduct in your laboratory, there will eventually be a FOA just for you.

13 responses so far

« Newer posts Older posts »