This is all the fault of qaz. And long time reader Nat had a blog post on this ages ago.
First, I shouldn’t have to remind you all that much about a simple fact of nature in the academic crediting system. Citations matter. Our quality and status as academic scientists will be judged, in small or in large ways, by the citations that our own publications garner.
This is not to say the interpretation of citations is all the same because it most assuredly is not. Citation counting leads to all sorts of distilled measures across your career arc- Highly Cited and the h-index are two examples. Citation counting can be used to judge the quality of your individual paper as well- from the total number of cites, to the sustained citing across the years to the impressive-ness of the journals in which your paper has been cited.
Various stakeholders may disagree over which measure of citation of your work is most critical.
On one thing everyone agrees.
Citations matter.
One problem (out of many) with the “Supplementary Materials”, that are now very close to required at some journals and heavily encouraged at others, is that they are ignored by the ISI’s Web of Science indexing and, so far as I can tell, Google Scholar.
So, by engaging in this perverted system by which journals are themselves competing with each other, you* are robbing your colleagues of their proper due.
Nat observed that you might actually do this intentionally, if you are a jerk.
So now, not only can supplementary info be used as a dumping ground for your inconclusive or crappy data, but you can also stick references to your competitors in there and shaft them their citations.
Try not to be a jerk. Resist this Supplementary Materials nonsense. Science will be the better for it.
__
*yes, this includes me. I just checked some Supplementary citations that we’ve published to see if either ISI or Google Scholar indexes them- they do not.
Yep. See this for a peer-reviewed critique of the problem: Kueffer, C., Niinemets, Ã., Drenovsky, R. E., Kattge, J., Milberg, P., Poorter, H., Reich, P. B., Werner, C., Westoby, M., and Wright, I. J. 2011. Fame, glory and neglect in meta-analyses. Trends in Ecology & Evolution 26:493-494. http://dx.doi.org/10.1016/j.tree.2011.07.007
See also: http://researchremix.wordpress.com/2010/08/13/supplementary-materials-is-a-stopgap-for-data-archiving/
The Journal of Neuroscience simply forbids supplementary material. You must put everything in the paper, end of the story.
I don’t know if anyone else is doing that.
Also, I don’t understand why we ditched the “old approach” of having a Glamour paper (essentially an extended abstract) followed by a “real” paper in a specialist journal. Even from a purely cynical viewpoint, isn’t it better to have two publications rather than one?
Aaaaand now I realize that my point has already been made several times in comments to the previous post.
I blame DrugMonkey and his unethical practice of publishing Supplementary Material posts!
It doesn’t help that a lot of journals put a limit on the number of citations allowed in the paper itself. So you are forced to put things in the supplemental. Which often involve the citations to bioinformatics methods that told you what all your nifty data meant.
Of course, part of the reason that these citation trackers are not pulling them from supplemental material is that the journals are not editing the supplemental material. If the journals edited supplemental and included them as part of the paper (online-only is fine, but download it as part of the paper, make sure they got reviewed, make sure the citations are trackable, etc.), then supplemental wouldn’t be the same problem it is today.
I only include supplemental materials when I reach the space limits established in the Editorial Guidelines. Unfortunately, that means 100% of all papers that I have published in the last 2 years, with the exception of a manuscript in JEM, which does not allow supplemental materials.
It’s not that I want to hide inconclusive or poorly reproducible data there. It’s that accommodating the demands of the reviewers in high impact publications these days requires as much supplemental information as data included in the main text.
Applying the character/word count to references is one of the most disastrous extant editorial policies carried into the digital age.
There’s no reason for it (anymore), and it gives authors excuses to drop citations instead of being thorough and inclusive. My boss LOVES the final editing phase of the submission process because it gives him reasons to drop a reference to hated colleague, sometimes going so far as to delete entire sentences to blot out the offending name.
Yes toto, two real pubs is always better than one (at the same JIF) plus SM
It’s that accommodating the demands of the reviewers in high impact publications these days requires as much supplemental information as data included in the main text.
I want everyone to think about this really hard. Why IS this? Why isn’t a high profile publication the culmination of studies, many of which would be okay published beforehand as methods or briefer, less awesome papers?
Why? Because GlamourHumping distorts everything and is why we cannot have nice things.
An awful lot of supplemental data is supportive and/or controls. I have a hard time seeing independent papers in a a lot of supplements that I look at. I think you are over-simplifying it a little bit. I agree that they should be edited and reviewed more thoroughly though.
You and the authors simply lack imagination.
Right. Of course. That’s the only explanation.
Biophysical Journal (Cell Press), actually takes references in the supplemental materials and appends them to the references used in the main document, such that both are counted:
http://www.biophysics.org/Publications/Newsletter/PastIssues/July2011/BiophysicalJournal/tabid/2926/Default.aspx
It seems like this is a simple solution to the problem.
Sure, if you can use the same preliminary data in several different grant proposals, you should be able to use them in several different publications as well. Oh wait…
I don’t see a problem with having a bunch of supplementary figures or supplementary data tables. Those can be quite useful most readers would likely not want to see them on first reading of the paper.
The problem arises with extensive supplementary text, and in particular with journals that expect all the methods to be in supplementary (I won’t name names). That’s what we really have to complain about. There’s also absolutely no reason for it in modern online publishing, just put the methods at the end of the pdf where nobody looks anyway.
Kevin,
If you don’t think that happens all the time you’re not looking hard enough.
Oh, man, in genomics this is an atrocious problem. I explored just one recent publication’s supplemental materials to look for the stuff I need. Citations are uneven (some things don’t have citations), and unless you dig here you cannot find the necessary tools to reproduce the work. They run hundreds of pages in the papers I need too.
http://blog.openhelix.eu/?p=20018
Srsly Od? Without mention of the other pub? Or do you mean the cases of fraud?
No. With mention of other pub. Redraw the figure, change the colors and voila! Or in some cases, change almost nothing.
I’m okay with that then. A fair response to an insane system. Why a simple citation would not suffice….
“GlamourHumping” deserves its own spot in the glossary of terms, I think.
Unmm, no.
Aww, DM, come on, its even got the superfluous “u” to class it up! Very proper.
“No. With mention of other pub. Redraw the figure, change the colors and voila! Or in some cases, change almost nothing.”
-Really? Isn’t that a violation of the publishing conditions for most journals?
Journals have ownership over the polished document as published. They don’t own the data themselves or the study as a whole (esp aspects that weren’t published).
“I’m okay with that then. A fair response to an insane system.”
you should consider becoming a private banker. Such ethics would suit you well.