The "whole point" of Supplementary Data

(by drugmonkey) Dec 10 2014

Our good blog friend DJMH offered up the following on a post by Odyssey:
Because the whole point of supplemental material is that the publisher doesn't want to spend a dime supporting it

This is nonsense. This is not "the whole point". This is peripheral to the real point.

In point of fact, the real reason GlamourMags demand endless amounts of supplementary data is to squeeze out the competition journals. They do this by denying those other journals the data that would otherwise be offered up as additional publications. Don't believe it? Take a look through some issues of Science and Nature from the late 1960s through maybe the mid 1970s. The research publications were barely Brief Communications. A single figure, maybe two. And no associated "Supplemental Materials", either. And then, if you are clever, you will find the real paper that was subsequently published in a totally different journal. A real journal. With all of the meat of the study that was promised by the teaser in the Glam Mag fleshed out.

Glamour wised up and figured out that with the "Supplementary Materials" scam they can lock up the data that used to be put in another journal. This has the effect of both damping citations of that specific material and collecting what citations there are to themselves. All without having to treble or quadruple the size of their print journal.

Nice little scam to increase their Journal Impact Factor distance from the competition.

19 responses so far

Thought of the Day

(by drugmonkey) Dec 09 2014

People selected to pontificate at an audience on the basis of prior accomplishments in a related context are invariably less interesting than people selected because they have interesting things to say.

8 responses so far

Repost: Should I hire a postdoc or a technician?

(by drugmonkey) Dec 08 2014

This repost is via special request from some n00b Assistant Professor who has apparently lost access to Google.

It was originally posted 25 Aug, 2008.


The comments following a recent post touched on the newly independent investigator dilemma of who to hire first: A postdoctoral fellow or a technician? We'll leave aside the best answer ("both") as impractical because, as Professor in Training noted,

I only have enough money to pay ONE postdoc's salary for 18 months ... or ONE tech ... that's it. While that would be great for me, that's certainly not enough time for a postdoc to get more than one study done (in my field probably only 75% of a study). Is it even advisable to employ a postdoc for such a short time with no great certainty of being able to pay their salary beyond that point?

YHN tends to recommend "tech" and PhysioProf tends to opt for "postdoc"...sounds like a new discussion to me!

To define terms just a little bit for those less steeped in the biz, I covered the job category of "technician" here:

A "technician" in the biomedical sciences is an employee of the laboratory (well, actually of the University) who is not "in training" (such as graduate students and postdoctoral fellows) and does not (usually) have a terminal doctoral degree. (For example long-term PhD scientist employees who are too far along to really be "postdocs" and are not PIs are not really "techs".) Most typically the tech has a bachelor's degree in a scientific major and a few will have advanced credentials such as Veterinarian Tech specialties or subject-based Master's degrees. It is not unusual for the tech to have continued her education while working in the laboratory by taking advantage of University educational repayment policies.

One of the most important parts for today's discussion is that the technician can be viewed, non-pejoratively, as 100% an employee working "for" the lab head or Principal Investigator (PI). Someone who is expected to do what is asked at all times with the goals of furthering the lab agenda. In this employment relationship the PI is unequivocally understood to be "the boss".

A postdoctoral fellow/trainee is a person who has acquired a terminal doctoral degree (Ph.D., D.V.M, D.D.S., M.D.) and is working under the supervision of an independent investigator. Here one key difference is that the postdoc is in a dual role, the balance of which is debated. The postdoc is considered a "trainee" in the sense that s/he is working in part for her (his) own benefit, to acquire skills and tools that will be required to obtain and launch an independent research career. This notion implies a degree of independence from the PI, an ability to work on stuff other than what the PI has "told" her (him) to do, possibly to work on stuff that is only of benefit to the postdoc (and not the lab or PI directly), etc. I happen to believe that the postdoc also has a responsibility to help the lab and, in essence, to do the job for which s/he was hired and to make all data generated accessible to the PI...but not every postdoc agrees with that. In truth it is also likely that some PIs either explicitly or implicitly think that postdocs are basically indistinguishable from technicians when it comes to the employment relationship. So there's a spectrum.

Okay, so why do I think a tech is more important to secure for a brand new PI?
That isn't your job anymore. Scutwork. Tedium. The stuff that is absolutely necessary to the running of your lab which is not particularly demanding in intellect and may be incredibly time-consuming. Yes, you specialized in this as a lowly graduate student and took pride in the fact that you were able to do this work as a postdoc while still doing more high-falutin' intellectual labor. Postdocs around you who couldn't find their behind with both hands when it came to the basic work were to be derided. Fine. But this is not your job anymore!!! My position is that the more time-consuming scutwork you can get off your plate the better. Since nobody likes scutwork, it is far better to rely on an employee who is paid to do a job, can be readily fired and replaced, for this sort of thing.

Your first deceptively hard question as a new PI is that of determining which tasks in the lab really do not require your input, after basic training and given that you will continue to supervise and troubleshoot. I say deceptively hard because my experience in talking with some fairly advanced postdocs and even junior faculty is that they have not really thought about this question. They get stuck in the usual traps. "It is more work than it is worth to train somebody to do this." "I only trust my own work/data/analyses." "My hands are the best." "This is too important to screwup." Etc.

All true. Being a PI takes a big leap of faith in the work of others. This is, in my view, part of the deal. For the huge increase in scientific terrain you are able to cover as a PI directing the efforts of other scientists, you are accepting the risk that someone else is not as good as you*. So get over this. Your job is to learn how to set up your management style such that you can tolerate human frailty and still make excellent progress on what interests you.

Progress and Work Ethic. Management of personnel is one of the hardest things for new PIs to learn. After all, we were motivated self-starters so we can't really understand why everyone else would bother to be doing this stuff if they weren't self-wackaloon-motivated. Sadly, not every one is just like you (ibid), new PI! Which means that you may have to evaluate an employees work effort and apply some judicious boots to the posterior. Perhaps even with threats of dismissal and actual dismissal for poor performance. It is very much easier to do this with a technician who is supposed to be working 100% at your behest.

Stability. In most cases postdocs will be transient visitors to your lab, lasting 3-5 years at best. So sure, a good one may get you through tenure. But postdocs can and do leave for all kinds of reasons. Their interests and relative focus on your stuff necessarily changes as time elapses. The tech on the other hand, can be a more or less career employee in whom your investment over time continues to pay you back over intervals of a decade or more.

Availability. Unless you are very lucky or very HawtStuff, recruiting a good postdoc to your laboratory is far from given. I've seen this from all ends, as a postdoc, peer of postdocs, mentor to postdocs , as a PI seeking fellows and as a peer to other junior faculty looking to hire. I've seen situations in which the fellow (or grad student) was very focused on joining the lab of the local BigCheez and quickly evolved to be working most closely with local junior investigators because the fit was so obviously better. The bottom line is that for a postdoc the prospect of joining a starting Assistant Professor's lab is very much of career concern.

An additional concern with availability is that it is not unusual for postdocs and PIs to come to arrangements far in advance of the actual start date. Very anecdotally, I'd say the better the candidate, the more likely this is the case. So a junior PI who manages to recruit and lock-in a highly promising candidate may have to accept that she will not be arriving in the lab for a year. That's a big hit, especially if that is the first year in which teaching loads have been reduced. Technicians are typically hired with a fairly short lead time on the order of weeks at worst.

Data Stream Strategy for the Long Haul. This one verges dangerously close to the argument over being an investigator who operates on the cutting edge at all times versus the small-town grocer. So YMMV. If, however, you have an aspect of your research program which can putter along with relatively little input from you, is of lasting importance to your scientific goals and interests and, most importantly, can support a steady stream of bread-and-butter publications I recommend getting this going. It is not necessary that the tech does all of the work up to publication-quality figures, of course, mainly that s/he is able to generate good quality and interesting data at your direction. Or the tech is capable of doing most of the work with you sailing in for the essential parts.

It is all very well and good to shoot for GlamourPubs. If you manage to get them, you are set. I get this. Not getting them is, however, excusable. In most environments, meaning that even if not in your specific department you can get a job elsewhere. Perhaps one theoretical tier down, but still a research-focused job. What is not optional is publishing somewhere, anywhere**. When it comes to most decisions that matter, tenure and review of prior progress when it comes to grant review, 0-fer is not excusable. Published articles can be debated on their merits with respect to actual impact, importance, brilliance, what have you. A lack of publication can not be debated or defended***. Admittedly the postdoc who is half-decent has a greater possibility of getting all the way to a submittable manuscript. But the bread-n-butter tech is near guaranteed to make sure the data are available to writeup when you are feeling the publication pinch.

Final Thought. It basically comes down to risk management from my perspective. If you can get a very good, hardworking postdoc right away the choice is pretty clear in opting for the postdoc. I am quite pessimistic, however, that new PIs can pull that off. When it comes down to a postdoc who will not show up for 12 months, a postdoc who is lazy, distracted or really focused on interests that are not sufficiently in line with yours...well, the technician wins every time.

Update: One thing I forgot to mention originally but was reminded of by the first comment. The chances of getting a technician working for you for free are next to zero. It is possible to get a postdoc for free, however. A postdoc may come with their own fellowship (there are many international-study type fellowships where a country funds individual fellows to go abroad, for example) or you may be able to secure a slot on a local institutional training grant for your postdoc. This is not a guarantee but these situations are considerably more likely than getting a technician working for you but paid for by some other source of support.

__
*almost by definition, the fact that you are a PI now means the smart money bets you were a better-than-average postdoc. Very likely you had more motivation, intellectual curiosity and, yes, better experimental hands than the average bear. Which means that on average, postdocs that come into your lab are going to fall short of the standard of you! (Yes, even accounting for an inflated view of self.) Deal.
**"anywhere" means "peer reviewed" and is environment specific. Whatever your field considers the supposedly lowest denominator or a minimally respectable "dump journal" or whatever.
***usually. I could tell you some stories. But really, make it easy on yourself and publish something already!

19 responses so far

McKnight: "Wait whut? There are data? Really? Maybe I'd better cool it..."

(by drugmonkey) Dec 05 2014

The sidebar to McKight's column at ASBMB Today this month is hilarious.

Author's Note

I’ve decided it’s prudent to take a break from the debate about the quality of reviewers on National Institutes of Health study sections. The American Society for Biochemistry and Molecular Biology governing council met in mid-November with Richard Nakamura, director of the NIH’s Center for Scientific Review. The discussion was enlightening, and the data presented will inform my future columns on this topic.

HAHAHHAA. Clearly it is news to McKnight that his opinions might actually be on topics for which there are data in support or contradiction? And now he has to sit down and go through actual facts to try to come up with a better argument that study sections today are populated with riff-raff who are incompetent to review science.

Never fear though, he still leaves us with some fodder for additional snickering at his....simple-minded thinking. He would like his readers to answer some poll questions...

The first question is:
Should the quality of the proposed research and researcher be the most important criteria dictating whether an NIH-sponsored grant is funded?

The response item is Yes/No so of course some 99% of the responses are going to be Yes. Right? I mean jeepers what a stupid question. Particularly without any sense of what he imagines might be a possible alternative to these two considerations as "the most important criteria". Even more hilariously since he has totally conflated the two things that are actual current items of debate (i.e., project versus person) and tie directly into his two prior columns!

The next question:
The review process used to evaluate NIH grant applications is:

has three possible answers:
essentially perfect with no room for improvement
slightly sub-optimal but impossible to improve
suboptimal with significant room for improvement

Again, simple-minded. Nobody thinks the system is perfect, this is a straw-man argument. I predict that once again, he's going to get most people responding on one option, the "suboptimal, room for improvement" one. This is, again, trivial within the discussion space. The hard questions, as you my Readers know full well, relate to the areas of suboptimality and the proposed space in which improvements need to be made.

What is he about with this? Did Nakamura really tell him that the official CSR position is that everything is hunky-dory? That seems very unlikely given the number of initiatives, pilot studies, etc that they (CSR) have been working through ever since I started paying attention about 7-8 years ago.

Ah well, maybe this is the glimmer of recognition on the part of McKnight that he went off half-cocked without the slightest consideration that perhaps there are actual facts here to be understood first?

17 responses so far

Thought of the day

(by drugmonkey) Dec 05 2014

One thing that always cracks me up about manuscript review is the pose struck* by some reviewers that we cannot possibly interpret data or studies that are not perfect.

There is a certain type of reviewer that takes the stance* that we cannot in any way compare treatment conditions if there is anything about the study that violates some sort of perfect, Experimental Design 101 framing even if there is no reason whatsoever to suspect a contaminating variable. Even if, and this is more hilarious, if there are reasons in the data themselves to think that there is no effect of some nuisance variable.

I'm just always thinking....

The very essence of real science is comparing data across different studies, papers, paradigms, laboratories, etc and trying to come up with a coherent picture of what might be a fairly invariant truth about the system under investigation.

If the studies that you wish to compare are in the same paper, sure, you'd prefer to see less in the way of nuisance variation than you expect when making cross-paper comparisons. I get that. But still....some people.

Note: this is some way relates to the alleged "replication crisis" of science.
__
*having nothing to go on but their willingness to act like the manuscript is entirely uninterpretable and therefore unpublishable, I have to assume that some of them actually mean it. Otherwise they would just say "it would be better if...". right?

8 responses so far

More in "NIH responds to a non-problem by creating a problem"

(by drugmonkey) Dec 05 2014

I can't even imagine what they are thinking.

This Notice informs the applicant community of a modification for how NIH would like applicants to mark changes in their Resubmission applications. NIH has removed the requirement to identify 'substantial scientific changes' in the text of a Resubmission application by 'bracketing, indenting, or change of typography'.

Effective immediately, it is sufficient to outline the changes made to the Resubmission application in the Introduction attachment. The Introduction must include a summary of substantial additions, deletions, and changes to the application. It must also include a response to weaknesses raised in the Summary Statement. The page limit for the Introduction may not exceed one page unless indicated otherwise in the Table of Page Limits.

First of all "would like" and "removed the requirement" do not align with each other. If the NIH "would like" that means this is not just a "we don't care whether you do it or not". So why not make it a mandate?

Next up...WHY?

Finally: How in all that is holy do they really expect the applicant to ("must") summarize "substantial additions, deletions, and changes" and to "include a response to weaknesses" in just one page?

I am starting to suspect Rockey is planning on burning the OER down to the ground before leaving for greener pastures.

18 responses so far

Wait...the new Biosketch is supposed to be an antiGlamour measure? HAHAHHAHHA!!!!!

(by drugmonkey) Dec 05 2014

A tweet from @babs_mph sent me back to an older thread where Rockey introduced the new Biosketch concept. One "Senior investigator" commented:

For those who wonder where this idea came from, please see the commentary by Deputy Director Tabak and Director Collins (Nature 505, 612–613, January 2014) on the issue of the reproducibility of results. One part of the commentary suggests that scientists may be tempted to overstate conclusions in order to get papers published in high profile journals. The commentary adds “NIH is contemplating modifying the format of its ‘biographical sketch’ form, which grant applicants are required to complete, to emphasize the significance of advances resulting from work in which the applicant participated, and to delineate the part played by the applicant. Other organizations such as the Howard Hughes Medical Institute have used this format and found it more revealing of actual contributions to science than the traditional list of unannotated publications.”

Here's Collins and Tabak, 2014 in freely available PMC format. The lead in to the above referenced passage is:

Perhaps the most vexed issue is the academic incentive system. It currently overemphasizes publishing in high-profile journals. No doubt worsened by current budgetary woes, this encourages rapid submission of research findings to the detriment of careful replication. To address this, the NIH is contemplating...

Hmmm. So by changing this, the ability on grant applications to say something like:

"Yeah, we got totally scooped out of a Nature paper because we didn't rush some data out before it was ready but look, our much better paper that came out in our society journal 18 mo later was really the seminal discovery, we swear. So even though the entire world gives primary credit to our scoopers, you should give us this grant now."

is supposed to totally alter the dynamics of the "vexed issue" of the academic incentive system.

Right guys. Right.

13 responses so far

Snowflakes falling

(by drugmonkey) Dec 05 2014

We've finally found out, thanks to Nature News, that the paltry academic salary on which poor Jim Watson has been forced to rely is $375,000 per year as "chancellor emeritus" at Cold Spring Harbor Laboratory. The current NIH salary limitation is $181,500, this is the maximum amount that can be charged to Federal grants. I'm here to tell you, most of us funded by NIH grants do not make anything like this as an annual salary.

#MontyPythonidae

 

The Office of Extramural Research blog, RockTalking, has 73 comments posted to the discussion of the new NIH biosketch format. I found one that expressed no concern and apparently the rest range from opposed to outraged. One of the things that people seem particularly enraged by is the report of the supposed pilot study they ran. The blog entry reports on how many people found the new format helpful and, as the many commenters point out, the real question is whether this new format is better or worse than the old format. This you will recognize, OER watchers, as a common ploy for the NIH- carefully construct the "study" or the data mining inquiry so as to almost guarantee an outcome that puts the NIH's activities and initiatives in a favorable light. We are not fooled.

 

Ruth Coker Burks' Story Corp is a must-listen. Jesus effing Christ we failed the fuck out of everything in the early days of the HIV/AIDS crisis. Thank goodness there were a few people like Ms. Burks around.

 

Phoenix AZ police can't stand being overshadowed in this critical measure of awesome policiness.

 

Apparently Cerebral Cortex is the latest academic journal to play shenanigans with the pre-print queue. Looks like there is an article by Studer and colleagues that was first published online Aug 7, 2013. I can find no information on the submission and acceptance dates. Perhaps I am just overlooking it but I have noticed a couple of times that journals with terrible timeline issues don't seem to publish this information like most journals do these days. Go figure.

 

According to some guy on the internet Jim Watson also has an awesome house that he doesn't have to pay for.

(in case you were worried about substantial amounts of his paltry $375K per year salary being eaten up in housing costs just like most other academics' salaries.

 

What is even more worrying about the NIH Office of Extramural Research is that even when they set out a pretty clear goal they are so bad at reaching it.

Marcia McNutt in Science:

Consider a rather outrageous proposal. Perhaps there has been too much emphasis on bibliometric measures that either distort the process or minimally distinguish between qualified candidates. What if, instead, we assess young scientists according to their willingness to take risks, ability to work as part of a diverse team, creativity in complex problem-solving, and work ethic? There may be other attributes like these that separate the superstars from the merely successful. It could be quite insightful to commission a retrospective analysis of former awardees with some career track record since their awards, to improve our understanding of what constitutes good selection criteria. One could then ascertain whether those qualities were apparent in their backgrounds when they were candidates for their awards.

12 responses so far

George Carlin theory of peer review

(by drugmonkey) Dec 03 2014

Everyone who is more approving or lenient than you are is an incompetent moron.

Everyone that is harsher or less enthusiastic is a total jackhole.

12 responses so far

Twelve Months of Drug Monkey (2014)

(by drugmonkey) Dec 02 2014

Apparently I missed this meme last year. huh.

The rules for this blog meme are quite simple.
-Post the link and first sentence from the first blog entry for each month of the past year.
I originally did this meme, after seeing similar posted by Janet Stemwedel and John Lynch. Prior editions include 2012, 2011, 2010, 2009 and 2008.

If you blog, I encourage you to do your own year-end wrap up post.


2014

Jan: A December 18 post on the Rock Talk blog issued an update on the funding rate situation for grant applications submitted to the NIH.

Feb: A new post at Speaking for Research details the history:

Mar: This is important enough to elevate to an entry.

Apr: I have been experiencing a sharp uptick in high school projects that are apparently titled: "Email questions to some random expert on the internet" lately.

May: This is an overview of a presentation in Symposium 491. Scientists versus Street Chemists: The Toxicity of Designer Marijuana presented Wed, Apr 30, 9:30 AM - 12:00 PM at the 2014 Experimental Biology meeting.

Jun: It has recently come to my attention that not everyone views the no-cost extension (NCE) of a NIH grant the same way I do.

Jul: I realize this is not news to most of you.

Aug: The Sesai suicide has been deemed the result of an anti-fraud witch hunt by well respected biomedical ethics / conduct of science / publishing / open science dude Michael Eisen.

Sep: One of the more awesome and fun parts of running this blog for so long is watching you all progress in your lives and careers.

Oct: Some days... I tell you, one of the most hilarious parts of this blogging gig is this.

Nov: The best known Klingon proverb ... is proof that before they were warriors, the Klingons were academicians.

Dec: The NIH has notified us (NOT-OD-15-024) that as of Jan 25, 2015 all grant applications will have to use the new Biosketch format (sample word doc).

3 responses so far

« Newer posts Older posts »