Vertebrate paleontology is one of the most broad-based, interdisciplinary branches of the natural sciences. Even though it is usually considered a branch of geology (and most university-based vertebrate paleontologists belong to Geology departments), it straddles the line between geology and biology. While it may be concerned with the lives of once living organisms, nearly all information about these organisms comes ultimately from the rock record. This occurs most notably in the form of the fossils themselves, but also as information about the environments these organisms inhabited (depositional systems, paleoclimate), the ways in which taxa and faunas changed over time (biostratigraphy/biochronology), and possible causes for faunal change (many of which, such as the identification of bolide impacts, rely heavily on geochemistry). The biological aspects of paleontology also loom large of course; systematics (including both alpha taxonomy and phylogenetics), functional morphology, and ontogenetic development studied through histology (especially what it may tell us about metabolism) perhaps being the most prominent. It’s a lot to process, but combined these various lines of evidence have the ability to tell us a great deal about the history of life on Earth.
A modest example of this is my own work with Bill here at Petrified Forest, where we have spent the last couple years collecting a great deal of geologic and morphological data to tell us about faunal change over time, and its relationship to environmental change, within this particular little corner of Arizona during a particular little segment of time. However, in mostly confining its gaze to one particular geographic unit, our work is not what might be considered a “big picture” study. “Big picture” studies tend to look at global issues, especially over long periods of time. They synthesize large quantities of data compiled by (mostly other) scientists, and they get a lot of attention. And money. What is sometimes under appreciated is that these big picture studies are only as solid as the multiple lines of evidence on which they are drawn.
The fact is that detailed small picture studies and big picture studies need each other. Small scale studies need big picture studies to give them ultimate relevance…but big picture studies need detailed small scale studies to give them validity. A small-scale study that isn’t incorporated into an overall picture of the grand scheme of things is an irrelevancy, and isolated novelty with no context. A big picture study can’t exist without small picture studies, and if the small picture studies are poorly done, the big picture study is bullshit. Garbage in, garbage out as they say.
In light of this co-dependence between small picture and big picture studies, I never fail to get pissed off when universities, grants, and journals consider big picture studies to be more important and worthy than small picture studies. Back in 2005, when the first preliminary paper on the Revueltosaurus material from PEFO which demonstrated that Revueltosaurus was not an ornithischian dinosaur was being prepared, we (well, Bill Randy, and Sterling; I had almost nothing to do with it) initially submitted it to JVP. However, the initial JVP paper was rejected for being of “too provincial” in its scope, or something along those lines.
One problem with this criticism is its inherent ridiculousness…ALL papers published in ANY journal are of provincial interest to some degree. I have never received an issue of JVP where I was particularly interested in all of the papers, or even most of them. While I recognize that bovid phylogeny or trace element geochemical analysis of vertebrate bones are as important and valid (to someone) as my own studies are to me, I couldn’t personally give a flying fuck about them. Even bigger picture studies, such as the impact of statistical tests or coding methods on phylogeny, are not so pressing to more geology-oriented workers like me.
But there is a deeper and more fundamental problem to this bias: it discourages detailed small-picture studies. This bias is ultimately destructive to science. If careful and detailed work to really nail down a limited problem is discouraged so that such studies are neglected, then what becomes of the larger, big picture syntheses based on these studies? For vertebrate paleontology to flourish as a rigorous science, detailed small-picture studies need to be supported and encouraged. They must be put an equal footing with broader-based big picture studies in terms of respect and funding. If we can reward scientists for playing with other people’s data, why can’t we reward the people who did the work to produce that data in the first place?
Evolving Thoughts may shut down for a while
1 hour ago
Posts
10 comments:
What Jeff said.
Yes, yes, yes.
And as for the Revueltosaurus paper being rejected from JVP for being "too provincial": that is just dumb beyond imagining. The decision was not merely debatable. It was Just Plain Wrong.
The more I subject my work to peer review, the more I see what a complete and utter crapshoot it is. You might get someone who wants to do whatever they can to make your work as good as it can be; you might get someone who just doesn't care, who'll dash off a paragraph or two just to discharge their responsibility; or you might get someone who is out to prevent anyone outside their own lab from publishing wherever possible. I have lost track of the huge tracts of my own time have been wasted in reformatting papers for resubmission because of the utterly pointless manoeuvres of the last category of "reviewer". It's utterly destructive. Damn.
I agree. But I also feel it may be less of a "big picture vs. small picture" thing than a "new & sexy vs. older & less novel" conflict. Even small picture studies seem to more easily get funded & published if they have DNA or isotopes or some such behind them. Whereas if one proposed a truly massive-picture study that involved "only" field geology, stratigraphy, vertebrate osteology, or other "classic paleo" approaches (although I'll admit these seem to be becoming less standard as time moves on), I suspect it would be a much harder sell ... even though such studies remain essential.
(As an aside -- did you really have to go and single out bovid phylogeny?!)
Just a little bit of clarification. The original Revueltosaurus paper contains a limited description and mostly focused on the hypothesis that isolated ornithischian-like teeth from the Late Triassic could not automatically be assigned to the Ornithischia anymore. Essentially this erased all purported Triassic ornithischian fossils world wide to only three specimens and completely erased the Triassic Laurasian record of this clade. We thought this was worthy of a short rapid communication. The short paper/rapid communication submission was rejected as not being important enough as it would only be of interest to a small portion of the SVP. So instead we submitted the same manuscript to the Royal Society and it was published three months later.
Regarding Mike's comments. I've found that when I think a reviewer request is ridiculous good correspondence with the editor can clear this up. I've also found myself arguing that including something requested by a reviewer was pointless, only to come around later to the idea that maye they are right. Peer-review is always an adventure, bu I've found no matter how ridiculous I think some of the stuff is I always end up with a better paper. Strange.
Regarding Jeff's main point. The Science battle is won in the trenches with the 'little guys' who are collecting the primary data, provided that they do it correctly. I've seen lots of well-written bigger picture papers in higher-profile journals that are absolutely useless because of the simple fact that the primary data they have based everything upon is utterly flawed. It is so bad in some research areas (ahem) that we have to completely start from scratch. Some scientists work in the big picture arena their entire careers, but to me it kind of seems like a crap shoot.
I agree with Bill on multiple points:
1. The Revueltosaurus paper is a bad example; they didn't reject it because it was 'too provincial'; it was more that one of the reviewers wanted more detail and a more complete description.
2. As someone who has been an editor for two different paleontology journals and peer reviewer for numerous other journals, I've seen it from both sides. Yes, there are bad/biased reviewers out there, and it is the job of the editor to down-weight their opinion in the decision to accept/reject papers. No doubt there are some editors out there who fail to do this job, because it requires actual critical decision skills.
But, I can tell you that there are alot of authors out there that are so in love with their own writing and ideas, that they nearly always feel unjustly persecuted by a critical review. These folks fail to accept criticism even when it can genuinely improve their paper. Also, they react very poorly when someone points out flaws in their cherished hypotheses.
And, I can tell you as a reviewer, that it is very frustrating when a paper you reviewed comes out some months later, and very little of your suggested revisions have made it in. It makes one not want to review papers when editors fail to enforce your review. I seem to particularly have this problem when I point out that paleontology has moved beyond the narrative/speculative stage, that there are methods to actual test said hypothesis, and the authors should do so!
As an author, sure, I get upset when I receive a critical review. However, I find that if I let things sit for a day or two, I often have a less emotional (and more even) perspective on the review. Sometimes, yes, the review is totally off base. But many times I realize that the reviewer either A) had a valid point, or B) misunderstood what I was trying to say. When B is the case, I often recognize that they misunderstood me because my writing was unclear. So, sometimes its better to take a step back and realize that maybe you need to revise your explanation of a concept, feature, etc.
Randy, there is all the difference in the world between a critical review and a hatchet job.
Yes, and if you read my previous comment carefully, you'll notice I acknowledge that. I'm just saying that sometimes authors with fragile egos interpret justifiable critical reviews as hatchet jobs. I'm NOT saying that is reflective of your own experience. Just trying to provide some perspective to the leagues of embittered authors.
My main point is that peer review is a wildly imperfect system with alot of flaws, but its still better than no peer review. The ideal is to have dynamic comment and response on a paper by the community (a la PLoS One), but here too peer review is still necessary before the paper is posted online, because it helps catch basic errors and misinterpretations, as well as fundamental flaws in papers.
You appear not to be a fan of peer review at the moment, so I'm sure you'll agree that the Amphicoelias brontodiplodocus paper by Galiano and Albersdörfer is clearly better off without peer review.
Hey, Randy.
I honestly don't know what my position is on peer review at the moment, and comments like yours are one of the factors that's helping me grope towards a conclusion. I don't yet know what it's going to be, so I am not in the position of feeling that I have to defend one or other stance.
What I do know includes the following:
1. You are right that peer-review of half-decent quality would have caught the Amphibrontocoeliadocus and either killed it or required it to be much better than it was. Score one for peer review.
2. I have had a whole bunch of really crappy peer-review experiences in the past year or two, including some for papers that did make it through review, so it's not just sour grapes talking. Score minus one, then.
3. PLoS ONE's experiment with "post-publication peer-review" is not really working. Even the Darwinius paper, which must be by far the most read and most controversial palaeo paper they've published, has attracted only 12 comments -- well below the median for SV-POW! articles, for example. Hocknull et al.'s description of three new Australian dinosaurs is probably more representative, and has 2 comments, of which one is by the PLoS team itself and the other is by, uh, me.
So I am leaning at the moment towards the idea that peer review, crappy though it is, remains for the moment less bad than the alternatives. But that's a bit like saying MRSA is better than smallpox.
The bottom line is, if we lived in a world without peer review, I'd have got maybe 80% more new science done in the last 18 months. So peer review is a heavy tax, and I want to be a lot more convinced that it's getting us something that's worth the price we're paying.
Mike,
Its heartening to hear you haven't given up on peer review yet.
Regarding #2, I wouldn't say its a full minus one, as your papers have gotten published, albeit slowly. Whats the likelihood that either you've just had bad luck with reviewers, heuristically speaking, or that there just happen to be more douche-bag reviewers among sauropod workers than other clades?
I agree that the PLoS One experiment isn't fully working - but I think its something that will take time, and PLoS needs to do a better job of nurturing it along (both for readers and authors). What I don't know is whether its been more successful for their non-paleontology articles.
In the end, I feel like your 80% tax is abnormal, and much heavier than the typical rate. I'm hoping you've just had a string of bad luck with reviewers, and that it will improve. I would just encourage you to think about those really helpful reviews you have gotten, and how they improved the paper(s).
Randy, you're probably right that the 80% tax I've been paying recently on Actual Work is aberrant. At least, it was never as high as that in the previous couple of years. Maybe I am overgeneralising from a really crappy run.
Wow, nice post,there are many person searching about that now they will find enough resources by your post.Thank you for sharing to us.Please one more post about that..
Post a Comment