Archive for the 'Impact Factor' category

Journal Citation Metrics: Bringing the Distributions

Jul 03 2018 Published by under Careerism, Impact Factor, Scientific Publication

The latest Journal Citation Reports has been released, updating us on the latest JIF for our favorite journals. New for this year is....

.....drumroll.......

provision of the distribution of citations per cited item. At least for the 2017 year.

The data ... represent citation activity in 2017 to items published in the journal in the prior two years.

This is awesome! Let's drive right in (click to enlarge the graphs). The JIF, btw is 5.970.

Oh, now this IS a pretty distribution, is it not? No nasty review articles to muck it up and the "other" category (editorials?) is minimal. One glaring omission is that there doesn't appear to be a bar for 0 citations, surely some articles are not cited. This makes interpretation of the article citation median (in this case 5) a bit tricky. (For one of the distributions that follows, I came up with the missing 0 citation articles constituting anywhere from 17 to 81 items. A big range.)

Still, the skew in the distribution is clear and familiar to anyone who has been around the JIF critic voices for any length of time. Rare highly-cited articles skew just about every JIF upward from what your mind things, i.e., that that is the median for the journal. Still, no biggie, right? 5 versus 5.970 is not all that meaningful. If your article in this journal from the past two years got 4-6 citations in 2017 you are doing great, right there in the middle.

Let's check another Journal....

Ugly. Look at all those "Other" items. And the skew from the highly-cited items, including some reviews, is worse. JIF is 11.982 and the article citation median is 7. So among other things, many authors are going to feel like they impostered their way into this journal since a large part of the distribution is going to fall under the JIF. Don't feel bad! Even if you got only 9-11 citations, you are above the median and with 6-8 you are right there in the hunt.

Final entry of the day:

Not too horrible looking although clearly the review articles contribute a big skew, possibly even more than the second journal where the reviews are seemingly more evenly distributed in terms of citations. Now, I will admit I am a little surprised that reviews don't do even better compared with primary review articles. It seems like they would get cited more than this (for both of these journals) to me. The article citation mean is 4 and the JIF is 6.544, making for a slightly greater range than the first one, if you are trying to bench race your citations against the "typical" for the journal.

The first takeaway message from these new distributions, viewed along with the JIF, is that you can get a much better idea of how your articles are fairing (in your favorite journals, these are just three) compared to the expected value for that journal. Sure, sure we all knew at some level that the distribution contributing to JIF was skewed and that median would be a better number to reflect the colloquial sense of typical, average performance for a journal.

The other takeaway is a bit more negative and self-indulgent. I do it so I'll give you cover for the same.

The fun game is to take a look at the articles that you've had rejected at a given journal (particularly when rejection was on impact grounds) but subsequently published elsewhere. You can take your citations in the "JCR" (aka second) year of the two years after it was published and match that up with the citation distribution of the journal that originally rejected your work. In the past, if you met the JIF number, you could be satisfied they blew it and that your article indeed had impact worthy of their journal. Now you can take it a step farther because you can get a better idea of when your article beat the median. Even if your actual citations are below the JIF of the journal that rejected you, your article may have been one that would have boosted their JIF by beating the median.

Still with me, fellow axe-grinders?

Every editorial staff I've ever seen talk about journal business in earnest is concerned about raising the JIF. I don't care how humble or soaring the baseline, they all want to improve. And they all want to beat some nearby competitors. Which means that if they have any sense at all, they are concerned about decreasing the uncited dogs and increasing the articles that will be cited in the JCR year above their JIF. Hopefully these staffs also understand that they should be beating their median citation year over year to improve. I'm not holding my breath on that one. But this new publication of distributions (and the associated chit chat around the campfire) may help with that.

Final snark.

I once heard someone concerned with JIF of a journal insist that they were not "systematically overlooking good papers" meaning, in context, those that would boost their JIF. The rationale for this was that the manuscripts they had rejected were subsequently published in journals with lower JIFs. This is a fundamental misunderstanding. Of course most articles rejected at one JIF level eventually get published down-market. Of course they do. This has nothing to do with the citations they eventually accumulate. And if anything, the slight downgrade in journal cachet might mean that the actual citations slightly under-represent what would have occurred at the higher JIF journal, had the manuscript been accepted there. If Editorial Boards are worried that they might be letting bigger fish get away, they need to look at the actual citations of their rejects, once published elsewhere. And, back to the story of the day, those actual citations need to be compared with the median for article citations rather than the JIF.

4 responses so far

JIF notes 2016

If it's late June, it must be time for the latest Journal Impact Factors to be announced. (Last year's notes are here.)

Nature Neuroscience is confirming its dominance over Neuron with upward and downward trends, respectively, widening the gap.

Biological Psychiatry continues to skyrocket, up to 11.2. All pretensions from Neuropsychopharmacology to keep pace are over, third straight year of declines for the ACNP journal lands it at 6.4. Looks like the 2011-2012 inflation was simply unsustainable for NPP. BP is getting it done though. No sign of a letup for the past 4 years. Nicely done BP and any of y'all who happen to have published there in the past half-decade.

I've been taking whacks at the Journal of Neuroscience all year so I almost feel like this is pile-on. But the long steady trend has dropped it below a 6, listed at 5.9 this year. Oy vey.

Looks like Addiction Biology has finally overreached with their JIF strategy. It jumped up to the 5.9 level 2012-2013 but couldn't sustain it- two consecutive years of declines lowers it to 4.5. Even worse, it has surrendered the top slot in the Substance Abuse category. As we know, this particular journal maintains an insanely long pre-print queue with some papers being assigned to print two whole calendar years after appearing online. Will anyone put up with this anymore, now that the JIF is declining and it isn't even the best-in-category anymore? I think this is not good for AB.

A number of journals in the JIF 4-6 category that I follow are holding steady over the past several years, that's good to see.

Probably the most striking observation is what appears to be a relatively consistent downward trend for JIF 2-4 journals that I watch. These were JIFs that have generally trended upward (slowly, slowly) from 2006 or so until the past couple of years. I assumed this was a reflection of more scientific articles being published and therefore more citations available. Perhaps this deflationary period is temporary. Or perhaps it reflects journals that I follow not keeping up with the times in terms of content?

As always, interested to hear what is going on with the journals in the fields you follow, folks. Have at it in the comments.

48 responses so far

Ruining scholarship, one bad mentor at a time

via comment from A Salty Scientist:

Gnosis:

When you search for papers on PubMed, it usually gives the results in chronological order so many new but irrelevant papers are on the top. When you search papers on Google Scholar, it usually gives results ranked by citations, so will miss the newest exciting finding. Students in my lab recently made a very simple but useful tool Gnosis. It ranks all the PubMed hits by (Impact Factor of the journal + Year), so you get the newest and most important papers first.

Emphasis added, as if I need to. You see, relevant and important papers are indexed by the journal impact factor. Of course.

25 responses so far

Seriously? Payment for citations?

Aug 14 2015 Published by under Ethics, Impact Factor

A Reader submitted this gem of a spam email:

We are giving away $100 or more in rewards for citing us in your publication! Earn $100 or more based on the journal’s impact factor (IF). This voucher can be redeemed your next order at [Company] and can be used in conjunction with our ongoing promotions!

How do we determine your reward?
If you published a paper in Science (IF = 30) and cite [Company], you will be entitled to a voucher with a face value of $3,000 upon notification of the publication (PMID).

This is a new one on me.

36 responses so far

JIF notes

Jun 24 2015 Published by under Impact Factor


More on NPP's pursuit of BP is here.


see this for reference

Additional Reading:
The 2012 JIFs are out

Subdiscipline categories and JIF

Why JIF is complete sheepshit from Stephen Curry

A significant change in Impact Factor

13 responses so far

Is the J Neuro policy banning Supplemental Materials backfiring?

As you will recall, I was very happy when the Journal of Neuroscience decided to ban the inclusion of any Supplemental Materials in articles considered for publication. That move took place back in 2010.

Dr. Becca, however, made the following observation on a recent post:

I'm done submitting to J Neuro. The combination of endless experiment requests due to unlimited space and no supp info,

I find that to be a fascinating comment. It suggests that perhaps the J Neuro policy has been ineffectual, or even has backfired.

To be honest, I can't recall that I have noticed anything in a J Neuro article that I've read in the past few years that reminded me of this policy shift one way or the other.

How about you, Dear Reader? Noticed any changes that appear to be related to this banning of Supplemental Materials?

For that matter, has the banning of Supplemental Materials altered your perception of the science that is published in that journal?

44 responses so far

H-index

I can't think of a time when seeing someone's h-index created a discordant view of their impact. Or for that matter when reviewing someones annual cites was surprising.

I just think the Gestalt impression you generate about a scientist is going to correlate with most quantification measures.

Unless there are weird outliers I suppose. But is there is something peculiar about a given scientist's publications that skews one particular measure of awesomeness....wouldn't someone being presented that measure discount accordingly?

Like if a h-index was boosted by a host of middle author contributions to a much more highly cited domain than the one most people associate you with? That sort of thing.

36 responses so far

The "whole point" of Supplementary Data

Dec 10 2014 Published by under Impact Factor, Scientific Publication

Our good blog friend DJMH offered up the following on a post by Odyssey:
Because the whole point of supplemental material is that the publisher doesn't want to spend a dime supporting it

This is nonsense. This is not "the whole point". This is peripheral to the real point.

In point of fact, the real reason GlamourMags demand endless amounts of supplementary data is to squeeze out the competition journals. They do this by denying those other journals the data that would otherwise be offered up as additional publications. Don't believe it? Take a look through some issues of Science and Nature from the late 1960s through maybe the mid 1970s. The research publications were barely Brief Communications. A single figure, maybe two. And no associated "Supplemental Materials", either. And then, if you are clever, you will find the real paper that was subsequently published in a totally different journal. A real journal. With all of the meat of the study that was promised by the teaser in the Glam Mag fleshed out.

Glamour wised up and figured out that with the "Supplementary Materials" scam they can lock up the data that used to be put in another journal. This has the effect of both damping citations of that specific material and collecting what citations there are to themselves. All without having to treble or quadruple the size of their print journal.

Nice little scam to increase their Journal Impact Factor distance from the competition.

19 responses so far

Wait...the new Biosketch is supposed to be an antiGlamour measure? HAHAHHAHHA!!!!!

A tweet from @babs_mph sent me back to an older thread where Rockey introduced the new Biosketch concept. One "Senior investigator" commented:

For those who wonder where this idea came from, please see the commentary by Deputy Director Tabak and Director Collins (Nature 505, 612–613, January 2014) on the issue of the reproducibility of results. One part of the commentary suggests that scientists may be tempted to overstate conclusions in order to get papers published in high profile journals. The commentary adds “NIH is contemplating modifying the format of its ‘biographical sketch’ form, which grant applicants are required to complete, to emphasize the significance of advances resulting from work in which the applicant participated, and to delineate the part played by the applicant. Other organizations such as the Howard Hughes Medical Institute have used this format and found it more revealing of actual contributions to science than the traditional list of unannotated publications.”

Here's Collins and Tabak, 2014 in freely available PMC format. The lead in to the above referenced passage is:

Perhaps the most vexed issue is the academic incentive system. It currently overemphasizes publishing in high-profile journals. No doubt worsened by current budgetary woes, this encourages rapid submission of research findings to the detriment of careful replication. To address this, the NIH is contemplating...

Hmmm. So by changing this, the ability on grant applications to say something like:

"Yeah, we got totally scooped out of a Nature paper because we didn't rush some data out before it was ready but look, our much better paper that came out in our society journal 18 mo later was really the seminal discovery, we swear. So even though the entire world gives primary credit to our scoopers, you should give us this grant now."

is supposed to totally alter the dynamics of the "vexed issue" of the academic incentive system.

Right guys. Right.

14 responses so far

BJP pulls a neat little self-citation trick

Sep 24 2013 Published by under Academics, Impact Factor

As far as I can tell, the British Journal of Pharmacology has taken to requiring that authors who use animal subjects conduct their studies in accordance with the "ARRIVE" (Animals in Research: Reporting In Vivo Experiments) principles. These are conveniently detailed in their own editorial:

McGrath JC, Drummond GB, McLachlan EM, Kilkenny C, Wainwright CL.Guidelines for reporting experiments involving animals: the ARRIVE guidelines.Br J Pharmacol. 2010 Aug;160(7):1573-6. doi: 10.1111/j.1476-5381.2010.00873.x.

and paper on the guidelines:

Kilkenny C, Browne W, Cuthill IC, Emerson M, Altman DG; NC3Rs Reporting Guidelines Working Group.Animal research: reporting in vivo experiments: the ARRIVE guidelines. Br J Pharmacol. 2010 Aug;160(7):1577-9. doi: 10.1111/j.1476-5381.2010.00872.x.

The editorial has been cited 270 times. The guidelines paper has been cited 199 times so far and the vast, vast majority of these are in, you guessed it, the BRITISH JOURNAL OF PHARMACOLOGY.

One might almost suspect the journal now has a demand that authors indicate that they have followed these ARRIVE guidelines by citing the 3 page paper listing them. The journal IF is 5.067 so having an item cited 199 times since it was published in the August 2010 issue represents a considerable outlier. I don't know if a "Guidelines" category of paper (as this is described on the pdf) goes into the ISI calculation. For all we know they had to exempt it. But why would they?

And I notice that some other journals seem to have published the guidelines under the byline of the self same authors! Self-Plagiarism!!!

Perhaps they likewise demand that authors cite the paper from their own journal?

Seems a neat little trick to run up an impact factor, doesn't it? Given the JIT and publication rate of real articles in many journals, a couple of hundred extra cites in the sampling interval can have an effect on the JIT.

11 responses so far

Older posts »