Query of the Day: When did you realize publication was key?

(by drugmonkey) Jan 12 2017

This question is for those who have ever entered a doctoral program in the sciences.

When did you realize that it was really, really important for you to publish first-author papers as a graduate student?

I recall that I really thought that the requirements and goals of grad school were to pass the first year exam (which was a research project presentation), pass the qualifying exam and write and defend a monolithic thesis describing a body of independently dreamed-up and designed research that I conducted myself.

I became aware of a bit of a debate about monolithic theses versus publications in the opinion of various faculty somewhere in my 2nd or 3rd years. So I knew of the idea that some Professors thought that three first-author publications stapled together with a cursory introduction and summary material was superior to the monolithic thesis.

I sided with the monolithic-thesis types and this, I think, let me continue to mislead myself about the importance of publications for my career. I also had career aspirations (right up until about six months before my first faculty appointment started to crystallize as reality) that did not necessarily require a strong publication record from graduate studies. Finally, I had the not-uncommon realization that I was going to have to do some postdoc work after graduate school, and the accompanying notion that postdoc work was when you really got steaming on publications, that let me off the hook.

So my answer would have to be that I didn't really grasp how important first-author pubs in grad school would be until I was late-postdoc and looking to land a faculty gig (and grants). I had probably the first dawning realization midway through my first postdoc. I would have to say that I had no serious understanding of this throughout most of grad school. I had ZERO concept of this as a graduate school applicant and graduate school interviewee.

46 responses so far

Ask DrugMonkey: Should I go over my NIH Program Officer's head?

(by drugmonkey) Jan 12 2017

I get this question from grant applicants now and again so I thought maybe it was time to answer it on the blog. The latest version was from the Twitts:

Ok, first of all "escalate" is the wrong way to think about it. So don't do that. But you should absolutely explore the opinions and input of other Program Officers if you are unhappy with the responses (or lack thereof) that you are getting from your assigned PO.

As a brief reminder, many if not most of the NIH IC's have their POs arranged in a hierarchical structure. The smallest unit is typically a Branch, inhabited by ~2-5 POs, one of whom is the Branch Chief. The grant applications assigned to those POs will all share certain scientific properties, depending on how the Branch is designed. The individual POs in the Branch may have primary distinct roles and expertises in terms of their portfolios but there will be substantial overlap. The Branch Chief is responsible for all of the grants and applications in her Branch, obviously. These are small groups of people so, also obviously, they are closely interacting colleagues. They talk to each other a LOT about the business of the Branch. This is one practical reason you don't want to think about "escalating" and you want to approach matters carefully. The whole Branch may actually share your assigned PO's low opinion of your work. The Chief may be totally buddies with your assigned PO and not really appreciate you screaming about how she or he is incompetent, biased and shouldn't be working for the NIH at all.

Branches are collected into Divisions. I'm a little less certain about the universality of how ICs are organized on this but sometimes the Division director also functions, in essence, as a Branch Chief. She just also has the responsibility for overseeing the entire Division of related Branches.

Still with me? Take a stroll on the Organization page of your favorite IC to see what I mean if this is confusing.

Division directors are allowed to talk to God, aka the IC Director. What I mean by this is that when it comes to the hammer and tongs discussion of what is to be funded, what can possibly be picked up with exception funding, etc, it is the Division director level that is making the case. To all the other Division Directors and to the IC Director. I think they are the ones called upon in Council meetings, generally, if a specific question arises.

The point here is that the Division Director needs to know your applications too. They have a direct chain-of-command responsibility for them. And ultimately they have a responsibility for the performance of the entire Division portfolio of funded grants. They are involved.

Another thing to remember. POs get promoted up the ranks. The Branch Chief of today might be the Division Director of tomorrow. Your PO may become Branch Chief. Also, there can be some shuffling of individual POs across Branches (and even ICs as it happens).

This is why I continue to bang on about how it is in your best interest to meet POs, many of them, and to continue your relationship with them when opportunities arise (annual scientific meetings, for example).

So, back to the question. This usually arises because the applicant feels like their assigned PO is just not interested in their work. The PO may never return their calls. The PO may actively criticize their Specific Aims and tell them not to apply. The PO may be giving all sorts of unhelpful advice or just sticks to the mantra (I advise you to revise and resubmit). The PO may be refusing to push for a pickup for a grey zone score.

An obvious thing to do is to appeal. To try to get someone else.

This is a reasonably good idea. You just need to approach it judiciously. POs can be biased or they can just not "get" your work or proposal. They may have applications on their list that are higher priority to them. They may still be bitter about something that happened with your grad student advisor*!

If your PO is not your Branch Chief, that is probably your first stop. As I say above, it is possible that she knows all about your situation but perhaps she does not. So give it a try. It is also not impossible that she knows all about the limitations of PO X under her Branch but can only really act when someone complains.

When you take it up the chain, I always think the best approach is to be in a stance of seeking advice, rather than complaining about your rights.

"I don't understand...there is a lack of [feedback, enthusiasm, explanation]...perhaps my applications are being assigned to the wrong PO, would another one be better?"

That sort of thing. You can take this same approach with the Division Director. If you do this, however, you need to express doubt that the original Branch is the right one and find some key words in the description of another Branch to suggest perhaps that is a better fit.

Ultimately, sure, you can take this straight to the IC Director. Even the NIH Director, I suppose.

Your ability to get them to take your call or pay any attention to your concerns whatever will depend on your status in the world. I've definitely had senior colleagues who are in continual contact with IC directors and would for sure talk to them directly about grant matters. Things as specific as picking up a near-miss grant application for funding. If you happen to know an IC director well, sure, go for it when the situation is really critical. Other people are sure as heck doing it so why shouldn't you?

I'll close by reiterating that you need to be judicious about this. Keep entitled demanding far away from your thoughts. Keep angry complains about the bias and incompetence of the PO that is frustrating you out of your mind. Take the position of seeking information. Strike an attitude of not understanding why your experience is different from the advice you are getting to contact POs.


15 responses so far

Predictors of Grad School Publications

(by drugmonkey) Jan 11 2017

A new paper in PLoS ONE purports to report on the relationship between traditional graduate school selection factors and graduate school success.

Joshua D. Hall, Anna B. O’Connell, Jeanette G. Cook. Predictors of Student Productivity in Biomedical Graduate School Applications. 2017, PLoS ONE, Published: January 11, 2017
http://dx.doi.org/10.1371/journal.pone.0169121 [Publisher Link]

The setup:

The cohort studied comprised 280 graduate students who entered the BBSP at UNC from 2008-2010; 195 had graduated with a PhD at the time of this study (July 2016), 45 were still enrolled, and 40 graduated with a Master's degree or withdrew. The cohort included all of the BBSP students who matriculated from 2008-2010.

The major outcome measure:

Publications by each student during graduate school were quantified with a custom Python script that queried Pubmed (http://www.ncbi.nlm.nih.gov/pubmed) using author searches for each student's name paired with their research advisor's name. The script returned XML attributes (https://www.nlm.nih.gov/bsd/licensee/elements_alphabetical.html) for all publications and generated the number of first-author publications and the total number of publications (including middle authorship) for each student/advisor pair.

For analysis they grouped the students into bins of 3+, 1-2 or 0 first author pubs with a '0+' category for zero first-author pubs but at least one middle-author publication.

OMG! Nothing predicts graduate school performance (especially those evil, evil, biased - I mentioned evil, right? - standardized scores).

Yes, even people who score below the 50th percentile on quantitative or verbal GRE land first-author publications! (Apple-polishing GPA kids don't seem to fare particularly well, either, plenty of first author publications earned by the 3.0-3.5 riff-raff.)

Oh bai the wai...prior research experience doesn't predict anything either.

Guess what did predict first author publications? Recommendation scores. That's right. The Good Old Boys/Girls Club of biased recommendations from undergraduate professors is predictive of the higher producing graduate students.

As the authors note in the Discussion, this analysis focused only on student characteristics. It could not account for the mentor lab, interaction of student characteristics with the mentor lab characteristics and the like.

I'll let you Readers mull this one over for a bit but I was struck by one thing.

We may be talking at cross purposes when we discuss how application metrics are used to predict graduate student success because we do not have the same idea of success in mind.

This analysis suggests the primary measure of success of a graduate student is the degree to which they succeeded in being a good data-monkey who produces a lot of publishable stuff within the context of their given research laboratory. And by this measure, nothing is very predictive, going by the Hall et al analysis, except the recommendation letter of those who are trying to assess the whole package from their varied perspectives of "I know it when I see it*".

Grad student publication number is, of course, related to who will go on to be a success as a creative independent scientist because of the very common belief that past performance predicts future performance. Those who exit grad school with zero pubs are facing an uphill battle to attain a faculty position. Those with 3+ first author pubs will generally be assumed to be more in the hunt as a potential future faculty member all along the postdoctoral arc.

Assuming all else equal.

This is another way we talk past each other about standardized scores, etc.

The choice of the PI who is trying to select a graduate student for their lab can assume "all else equal". Approximately. Same lab, same basic environment. We don't have this information from Hall et al. and I think it would be pretty difficult to do the study in a way that used same-lab as a covariate. Not impossible...you just are going to need a very large boat.

I think of it this way. Maybe there are some labs where everyone gets 3 or more first-author papers? Maybe there are some where it takes a very special individual indeed to get more than one in the course of graduate school? And without knowing if the student characteristics determine the host lab, we have to assume random (ish) assignment. Thus it could be the case that the better GREquant, for example, gives a slight advantage within lab but this is wiped out by the variability between-labs.

The choice of a selection committee for graduate programs can be less confident about all else being equal. They have to ask what sort of student can be successful across all of the lab environments in the program. Or successful in the majority of them. The Hall et al. data say that many types can be. But we are still asking a question of whether the training environment is such an overwhelming factor that almost nothing about the individual matters. This seems to be the message

If so, why are we bothering to select students at all? Why have them apply with any details other than the recommendation letters?

Maybe this is another place we are speaking at cross purposes. Some of us may still believe that the point of graduate school selection is to train the faculty (or insert any other specific career outcome if relevant) of tomorrow. Part of the goal, therefore, may be to select people on the basis of who we think would be best at that future role**, regardless of the variation in papers generated with first-author credit as a graduate student.

Is the Hall et al. paper based on a straw notion of "success"?

I think you've probably noticed, Dear Reader, that my opinion is that the career of grant-funded PI takes some personality characteristics that are not easily captured by the number of first-author pubs as a graduate student. Grit and resilience. Intrinsic motivation to git-er-done. Awareness of the structural, career-type aspects. At least a minimal amount of interpersonal skills.

What I am not often on about is the fact that I think that given approximately equal conditions, smarts matters. This is not saying that smarts is the only thing. If you are smart as all heck and you don't have what it takes to be productive or to take a hit, you aren't going to do well. It's the flip side. If two people do have grit and resilience and motivation...the smarter person is going to have an easier time of it or achieve more for the same effort**. On average.

And this is a test that is not performed in the new paper. Figuring out how to compare outcomes within laboratory groups might be an advance on this question.

*When I write recommendation letters for undergrads who have worked with me I do not have access to their standardized scores or grades. I have my subjective impressions of their smarts and industry from their work in my lab to go by. That's it. Maybe other people formally review a transcript and scores before writing a letter? I doubt it but I guess that is possible.

**Regarding that future role, again it may be a question of what is most important for success. Within our own lab, we are assuming that differential opportunity to get publications is not a thing. So since this part of the environment is fixed, we should be thinking about what is going to lead to enhanced success down the road, given conceivable other environments. From the standpoint of a Program, the same? or do we just feel as though the best success in our Program is enough to ensure the best success in any subsequent environment? The way we look at this may be part of what keeps us talking past each other about what graduate selection is for.

17 responses so far

Advice for Prospective Graduate Students

(by drugmonkey) Jan 06 2017

There is a lot of great advice of the usual sort floating around - talk to current grad students and postdocs about Department, Program and Lab culture. Median time to completion*. So I won't repeat that.

But here's one thing you may not hear about.

Ask the Program Director for the past two 5-year reviews of the Program. Yes, graduate training programs get peer reviewed on a periodic basis. Every 5 years in my limited experience.

Ask to see the review. Absent that ask for the top five most serious criticisms. In fact you should ask this latter question if anyone who interviews you to get a sense of how much the Program is integrated vs ad hoc.

Here's another important question to ask the interviewing faculty: "Who are the most recent 5-10 faculty appointments to come from your Program alumni?" The key here is to ask it on the spot so they can't look it up.

The most important thing here will not be the actual-factual answers. It will be how the faculty respond to your inquiries.

Good luck.

*Please tell me every prospect asks about the median time to PhD?

UPDATE: I meant this as a step to take after you are invited to interview or offered admission. A step for you to take to help decide which program to attend. Although I suppose even if you only get one offer it is helpful to know what to expect or watch out for.

73 responses so far

Thought of the Day

(by drugmonkey) Jan 04 2017

21 responses so far

Tenured profs should pick up the check?

(by drugmonkey) Jan 03 2017

While I think generosity on the part of more senior scientists is a good thing, and should be encouraged, making this an obligation is flawed. How do you know what that person's obligations are?

I post this in case any PI types out there don't know this is a thing. If you can pick up a check or pay more than your share, hey great. Good for you.

But nobody should expect it of you.

26 responses so far

Happy 2017!

(by drugmonkey) Jan 01 2017

Happy New Year everyone!

Generate knowledge.

Act like a decent person.

Oppose and illuminate the indecent behavior that crosses your path.

3 responses so far

Cannabidiol is still Schedule I, where it has been for some time

(by drugmonkey) Dec 31 2016

The DEA has created a new drug
Code for cannabis extracts, leading to some feather fluffing in the advocacy press.

The Federal Register notice explaining this is pretty clear so I'm not seeing where the alleged confusion lies.

The part responding to prior comment makes the situation with cannabidiol (CBD) very explicit.

One comment requested clarification of whether the new drug code will be applicable to cannabidiol (CBD), if it is not combined with cannabinols.

DEA response: For practical purposes, all extracts that contain CBD will also contain at least small amounts of other cannabinoids.1 However, if it were possible to produce from the cannabis plant an extract that contained only CBD and no other cannabinoids, such an extract would fall within the new drug code 7350. In view of this comment, the regulatory text accompanying new drug code 7350 has been modified slightly to make clear that it includes cannabis extracts that contain only one cannabinoid.

CBD has been on the Schedule for quite some time as far as I know. It is listed specifically on the application for a researcher license. You won't be able to buy it from a legitimate scientific reagent company such as Sigma without a DEA license. Very hard to miss.

I am aware of some very dodgy stuff going on with CBD for the quack supplement industry. From what I can tell, some of these companies are importing pure CBD under cover of "industrial hemp". Hemp is defined by lack of delta9-THC content, of course. Making "hemp" that contains high levels of the clearly Scheduled CBD a very gray area. It will be interesting to see if part of the outcome of this new extracts code will be invigorated prosecution of these CBD supplement companies.

2 responses so far

Ethics reminder for scientists

(by drugmonkey) Dec 31 2016

If the lab head tells the trainees or techs that a specific experimental outcome* must be generated by them, this is scientific misconduct.

If the lab head says a specific experimental outcome is necessary to publish the paper, this may be very close to misconduct or it may be completely aboveboard, depending on context. The best context to set is a constant mantra that any outcome teaches us more about reality and that is the real goal.

*no we are not talking about assay validation and similar technical development stuff.

15 responses so far

Cannabis hyperemesis syndrome rates increase with marijuana legalization

(by drugmonkey) Dec 31 2016

A report by CBS News reports on a 2015 paper:

Howard S. Kim, MD, John D. Anderson, MD, Omeed Saghafi, MD, Kennon J. Heard, MD, PhD, and Andrew A. Monte, MD Cyclic Vomiting Presentations Following Marijuana Liberalization in Colorado. Acad Emerg Med. 2015 Jun; 22(6): 694–699.
Published online 2015 Apr 22.

From the Abstract:

The authors reviewed 2,574 visits and identified 36 patients diagnosed with cyclic vomiting over 128 visits. The prevalence of cyclic vomiting visits increased from 41 per 113,262 ED visits to 87 per 125,095 ED visits after marijuana liberalization, corresponding to a prevalence ratio of 1.92 (95% confidence interval [CI] = 1.33 to 2.79). Patients with cyclic vomiting in the postliberalization period were more likely to have marijuana use documented than patients in the preliberalization period (odds ratio = 3.59, 95% CI = 1.44 to 9.00).

For background on the slow, Case Report driven appreciation that a chronic cyclical vomiting syndrome can be caused by cannabis use, see blog posts here, here, here.

The major takeaway message is that when physicians or patients are simply aware that there is this syndrome, diagnosis can be more rapid and a lot less expensive. Patients can, if they are able to stop smoking pot, find relief more quickly.

As far as the present report showing increasing rates in CO, well, this is interesting. Consistent with a specific causal relationship of cannabis use to this hyperemesis syndrome. But hard to disentangle growing awareness of the syndrome from growing incidence of it. We'll just have to follow these relationships as more states legalize medical and recreational marijuana.

Additional coverage from Dirk Hansen.

No responses yet

Older posts »