Archive for the 'Conduct of Science' category

Trophy collaborations

Jul 05 2018 Published by under Conduct of Science, NIH, NIH Careerism

Jason Rasgon noted a phenomenon where one is asked to collaborate on a grant proposal but is jettisoned after funding of the award:

I'm sure there are cases where both parties amicably terminate the collaboration but the interesting case is where the PI or PD sheds another investigator without their assent.

Is this common? I can't remember hearing many cases of this. It has happened to me in a fairly minor way once but then again I have not done a whole lot of subs on other people's grants.

17 responses so far

Scientists fighting back on the "reproducibility crisis" hysteria

May 16 2018 Published by under Conduct of Science, ReplicationCrisis

A new Op/Ed in PNAS takes up the reproducibility crisis.

A. D Redish, E. Kummerfeld, R. L. Morris, A. Love (2018) “Opinion: Reproducibility failures are essential to scientific inquiry” PNAS 115(20):5042-5046. [Journal Site]

Takeaway quote from the Abstract

Most importantly, these proposed policy changes ignore a core feature of the process of scientific inquiry that occurs after reproducibility failures: the integration of conflicting observations and ideas into a coherent theory.

As you will see, they had me at:

In many of these cases, what have been called “failures to replicate” are actually failures to generalize across what researchers hoped were inconsequential changes in background assumptions or experimental conditions

(Oh, wait, they cited me! AHAHAA, of course I like this thing!)

Seriously though, this is good stuff. Go read. Bookmark to forward to anyone who starts in on how there is a reproducibility "crisis".

9 responses so far

NIH reminds Universities not to keep paying harasser PIs from grant funds while suspended

On the May 1, 2018 the NIH issued NOT-OD-18-172 to clarify that:

NIH seeks to remind the extramural community that prior approval is required anytime there is a change in status of the PD/PI or other senior/key personnel where that change will impact his/her ability to carry out the approved research at the location of, and on behalf of, the recipient institution. In particular, changes in status of the PI or other senior/key personnel requiring prior approval would include restrictions that the institution imposes on such individuals after the time of award, including but not limited to any restrictions on access to the institution or to the institution’s resources, or changes in their (employment or leave) status at the institution. These changes may impact the ability of the PD/PI or other senior/key personnel to effectively contribute to the project as described in the application; therefore, NIH prior approval is necessary to ensure that the changes are acceptable.

Hard on the heels of the news breaking about long term and very well-funded NIH grant Principal Investigators Thomas Jessel and Inder Verma being suspended from duties at Columbia University and The Salk Institute for Biological Studies, respectively, one cannot help but draw the obvious conclusion.

I don't know what prompted this Notice but I welcome it.

Now, I realize that many of us would prefer to see some harsher stuff here. Changing the PI of a grant still keeps the sweet sweet indirects flowing into the University or Institute. So there is really no punishment when an applicant institution is proven to have looked the other way for years (decades) when their well-funded PIs are accused repeatedly of sexual harassment, gender-based discrimination, retaliation on whistleblowers and the like.

But this Notice is still welcome. It indicates that perhaps someone is actually paying a tiny little bit of attention now in this post-Weinstein era.

4 responses so far

NIH encourages pre-prints

In March of 2017 the NIH issued a notice on Reporting Preprints and Other Interim Research Products (NOT-OD-17-050): "The NIH encourages investigators to use interim research products, such as preprints, to speed the dissemination and enhance the rigor of their work.".

The key bits:

Interim Research Products are complete, public research products that are not final.

A common form is the preprint, which is a complete and public draft of a scientific document. Preprints are typically unreviewed manuscripts written in the style of a peer-reviewed journal article. Scientists issue preprints to speed dissemination, establish priority, obtain feedback, and offset publication bias.

Another common type of interim product is a preregistered protocol, where a scientist publicly declares key elements of their research protocol in advance. Preregistration can help scientists enhance the rigor of their work.

I am still not happy about the reason this happened (i.e., Glam hounds trying to assert scientific priority in the face of the Glam Chase disaster they themselves created) but this is now totally beside the point.

The NIH policy (see OpenMike blog entry for more) has several implications for grant seekers and grant holders which are what form the critical information for your consideration, Dear Reader.

I will limit myself here to materials that are related to standard paper publishing. There are also implications for materials that would never be published (computer code?) but that is beyond the scope for today's discussion.

At this point I will direct you to bioRxiv and PsyRxiv if you are unfamiliar with some of the more popular approaches for pre-print publication of research manuscripts.

The advantages to depositing your manuscripts in a pre-print form are all about priority and productivity, in my totally not humble opinion. The former is why the Glamour folks are all a-lather but priority and scooping affect all of us a little differently. As most of you know, scooping and priority is not a huge part of my professional life but all things equal, it's better to get your priority on record. In some areas of science it is career making/breaking and grant getting/rejecting to establish scientific priority. So if this is a thing for your life, this new policy allows and encourages you to take advantage.

I'm more focused on productivity. First, this is an advantage for trainees. We've discussed the tendency of new scientists to list manuscripts "in preparation" on their CV or Biosketch (for fellowship applications, say, despite it being technically illegal). This designation is hard to evaluate. A nearing-defense grad student who has three "in prep" manuscripts listed on the CV can appear to be bullshitting you. I always caution people that if they list such things they had better be prepared to send a prospective post-doc supervisor a mostly-complete draft. Well, now the pre-print allows anyone to post "in preparation" drafts so that anyone can verify the status. Very helpful for graduate students who have a short timeline versus the all too typical cycle of submission/rejection/resubmission/revision, etc. More importantly, the NIH previously frowned on listing "in preparation" or "in review" items on the Biosketch. This was never going to result in an application being returned unreviewed but it could sour the reviewers. And of course any rule followers out there would simply not list any such items, even if there was a minor revision being considered. With pre-print deposition and the ability to list on a NIH biosketch and cite in the Research Plan there is no longer any vaporware type of situation. The reviewer can look at the pre-print and judge the science for herself.

This applies to junior PIs as well. Most likely, junior PIs will have fewer publications, particularly from their brand new startup labs. The ability of the PI to generate data from her new independent lab can be a key issue in grant review. As with the trainee, the cycle of manuscript review and acceptance is lengthy compared with the typical tenure clock. And of course many junior PIs are trying to balance JIF/Glam against this evidence of independent productivity. So pre-print deposition helps here.

A very similar situation can apply to us not-so-junior PIs who are proposing research in a new direction. Sure, there is room for preliminary data in a grant application but the ability to submit data in manuscript format to the bioRxiv or some such is unlimited! Awesome, right?

15 responses so far

Thought of the Day

Jan 04 2017 Published by under Conduct of Science

21 responses so far

Tenured profs should pick up the check?

Jan 03 2017 Published by under Academics, Conduct of Science

While I think generosity on the part of more senior scientists is a good thing, and should be encouraged, making this an obligation is flawed. How do you know what that person's obligations are?

I post this in case any PI types out there don't know this is a thing. If you can pick up a check or pay more than your share, hey great. Good for you.

But nobody should expect it of you.

27 responses so far

Finishing projects

If you are paid by the taxpayers, or generous private philanthropists, of your country to do science, you owe them a product. An attempt to generate knowledge. This is one of the things that orients much of my professional behavior, as I think I make clear on this blog.

If you haven't published your scientific work, it doesn't exist. This is perhaps an excessive way to put it but I do think you should try to publish the work you accomplish with other people's money.

Much of my irritation with the publication game, prestige chasing, delusions of complete stories, priority / scooping fears and competition for scarce funding resources can be traced back to these two orienting principles of mine.

My irritation with such things does not, however, keep them from influencing my career. It does not save me from being pressured not to give the funders their due.

It is not unusual for my lab, and I suspect many labs, to have thrown a fair amount of effort and resources into a set of investigations and to realize a lot more will be required to publish. "Required", I should say because the threshold for publication is highly variable.

Do I throw the additional resources into an effort to save what is half or three-quarters of a paper? To make the project to date publishable? I mean, we already know the answer and it is less than earth shaking. It was a good thing to look into, of course. Years ago a study section of my peers told us so to the tune of a very low single digit percentile on a grant application. But now I know the answer and it probably doesn't support a lot of follow-up work.

Our interests in the lab have moved along on several different directions. We have new funding and, always, always, future funding to pursue. Returning to the past is just a drag on the future, right?

I sometimes feel that nobody other than me is so stupid as to remember that I owe something. I was funded by other people's money to follow a set of scientific inquiries into possible health implications of several things. I feel as though I should figure out how to publish the main thing(s) we learned. Even if that requires some additional studies be run to make something that I feel is already answered into something "publishable".

22 responses so far

First rule of Science Mentor Club

The very first rule of PI/mentorship is get your trainees first author publications.

This is the thing of biggest lasting career impact that you can determine almost with absolute control.

Yes, things happen but if you are not getting the vast majority of your trainees first author pubs you are screwing up as a mentor.

So. 2017 is about to start. Do you have a publication plan for all of your postdocs and later-stage graduate students?

Obviously I am in favor of active management of trainees' publishing plans. I assume some favor a more hands-off approach?

"Let the postdoc figure it out" has an appeal. Makes them earn those pubs and sets them up for later hard times.

The problem is, if they fail to get a publication, or enough, their career takes a bad hit. So ability to grunt it out isn't ever used.

42 responses so far

Completely uncontroversial PI comment on paper writing

Sep 29 2016 Published by under Careerism, Conduct of Science

Go!

46 responses so far

The NIH has shifted from being an investor in research to a consumer of research

WOW. This comment from dsks absolutely nails it to the wall.

The NIH is supposed to be taking on a major component of the risk in scientific research by playing the role of investor; instead, it seems to operates more as a consumer, treating projects like products to be purchased only when complete and deemed sufficiently impactful. In addition to implicitly encouraging investigators to flout rules like that above, this shifts most of the risk onto the shoulders of investigator, who must use her existing funds to spin the roulette wheel and hope that the projects her lab is engaged in will be both successful and yield interesting answers. If she strikes it lucky, there’s a chances of recouping the cost from the NIH. However, if the project is unsuccessful, or successful but produces one of the many not-so-pizzazz-wow answers, the PI’s investment is lost, and at a potentially considerable cost to her career if she’s a new investigator.

Of course one might lessen the charge slightly by observing that it is really the University that is somehow investing in the exploratory work that may eventually become of interest to the buyer. Whether the University then shifts the risk onto the lowly PI is a huge concern, but not inevitable. They could continue to provide seed money, salary, etc to a professor who does not manage to write a funded grant application.

Nevertheless, this is absolutely the right way to look at the ever growing obligation for highly specific Preliminary Data to support any successful grant application. Also the way to look at a study section culture that is motivated in large part by perceived "riskiness" (which underlies a large part of the failure to reward untried investigators from unknown Universities compared with established PIs from coastal elite institutions).

NIH isn't investing in risky science. It is purchasing science once it looks like most of the real risk has been avoided.

I have never seen this so clearly, so thanks to dsks for expressing it.

38 responses so far

Older posts »