• The value of pre-publication peer review

    I see lot of discussion these days about the value of peer review. Are journals too selective? Are acceptance decisions arbitrary? Does peer review actually catch scientific mistakes or fraudulent practices? Wouldn’t it be better to just put everything out there, say on preprint servers, and separate the wheat from the chaff in post-publication review? I’m not quite ready yet to give up on pre-publication peer review. I think it serves a useful purpose, one I wouldn’t want to do away with. In the following, I discuss four distinct services that peer review provides, and assess the value I personally assign to each of them.

    Read More
  • Is there an avalanche of low-quality research, and if so, must we stop it?

    Update: It turns out the article in the Chronicle is not recent, I misread the date on the page. (The Chronicle has two dates on each page, today’s date and the article publication date.) I stand by everything else I say, though.

    A recent article in the Chronicle of Higher Education argues that “we must stop the avalanche of low-quality research.” The authors decry the rapid growth of the scientific literature, which (as they argue) puts increasing strain on readers, reviewers, and editors without producing much benefit. They argue that this growth is driven by an increasing pressure on scientists to publish more, and the result is increasing amounts of low-quality publications. To address the pressure on scientists, they propose three fixes, of which one is Ok and two are positively inane. Maybe what we really have to stop is the avalanche of low-quality, non-reviewed opinion pieces published on web pages?

    Read More
  • Excess ambition—the eternal flaw of all PhD thesis proposals

    I cannot remember ever having seen a graduate student present a PhD thesis proposal and be criticized for lack of ambition. It never happens. Even the weakest students—especially the weakest students—present proposals that are overly ambitious and that won’t ever get done, and certainly not in the 3-4 years remaining until graduation. In fact, in my experience it is exceedingly rare that a student presents a reasonable proposal, one that is actually doable during the remainder of the time in graduate school. Usually, those only happen when students “forget” to have their qualifying exams and end up presenting their “proposal” six months before the intended graduation date. In those cases, the students know that they won’t accomplish much new between proposal day and defense day, and basically present a proposal that consists entirely of completed work.

    Read More
  • No one reads your paper either

    Simon Goring wrote an interesting post a few days ago arguing that no one reads your blog. In his post, he discussed reasons for why you might want to blog anyway. This post prompted me to tackle a topic I’ve been thinking about for a while: Why should you do science? Why should you publish? No one is going to read your paper either.

    Read More
  • Weight doesn’t matter

    The majority of people on a diet will tell you that they’re dieting because they want to lose weight [1]. If you ask them why they want to lose weight, you’ll hear things like “I don’t want to have a heart attack at 50” or “I have some extra flab” or “I want to get rid of my spare tire” or “I want a six pack.” So people want to be healthy, or they want to look good naked, or both. Importantly, though, none of these goals have much to do with weight per se. What people are actually concerned with is excess fat. And we generally know this, of course. We know that when we say somebody is overweight we really mean the person is overfat. Yet, in day-to-day practice, we don’t use the word “overfat,” we keep saying “overweight.” And as a result, we keep confusing weight and fat, and more importantly, we use the wrong metrics to keep track of our dieting success.

    Read More
  • Use fine-grained sectioning in your grant proposals

    I have noticed lately that many scientists write grant proposals with little document structure. Their proposals may have separate sections for background, for specific aims, and so on, but each of these sections is basically a long wall of text. The problem with this approach is that most reviewers will not carefully read all this text, and as a consequence they will miss important elements of the proposal. All else equal, a poorly structured proposal will be much less competitive than a well-structured proposal with many sub-sections and titled paragraphs. Because grant-writing remains one of the most important skills to be acquired by junior scientists, I see a critical need to highlight the importance of fine-grained sectioning in grant proposals.

    Read More
  • Ten simple rules for reproducible computational research

    PLOS Computational Biology just published a new addition to their popular “ten simple rules” series:

    Sandve GK, Nekrutenko A, Taylor J, Hovig E (2013) Ten Simple Rules for Reproducible Computational Research. PLoS Comput Biol 9(10): e1003285. doi:10.1371/journal.pcbi.1003285

    This article is relevant to anybody who wants to do computational research. I’ll make it required reading in my lab. For every single one of these rules, I can think of projects I’ve been involved with [1] that ran into trouble or failed because they violated that rule.

    Read More
  • Giving effective presentations

    Today, I gave a talk in our weekly departmental seminar about giving effective presentations. Here are the slides I used. Not all of them may make sense without the accompanying oral presentation. (There’s one slide in particular where I purposefully tell a different story than the slide does.) However, most of the material should make sense as is.

    Read More
  • What is the value of a mentor?

    Amidst the flurry of posts on the recent sexual-harassment-in-science-blogging scandal [1], I found this post on slate.com which made some interesting statements about the value (or lack thereof) of mentors:

    And the mentor-mentee relationship is one of the most fraught of adulthood. We glibly advise people starting out in business to find a mentor, to identify a successful, established, generous person in your field and somehow get her to help you become her.
    This is terrible advice. It perpetuates old-boy networks, wastes time that early career people could spend actually doing their work, and tells them they are only as good as their contacts and charm. Young people, don’t look for a mentor. Listen to and learn from people who have more experience, but don’t hitch your wagon to their star. Just do your job well.

    Read More
  • The critical need in a grant application

    It’s surprising how many grant applications do not clearly spell out a critical need. A critical need is the fundamental reason why a grant should exist. Without it, there’s really not much of a point to the grant. And yet, a critical need is frequently not stated. The common mistake that people make is that they confuse a gap in knowledge with a critical need. They clearly spell out what gap in knowledge they plan to address. And mind you, that’s an important component of a grant proposal. But if you stop there, if you never explain why there is a critical need to bridge the gap in knowledge, then your application is going to fall short. Unless the reviewers really like you for other reasons, they will probably not be impressed with your grant application.

    Read More
  • Which grants get funded at single-digit funding rates?

    Throughout the last decade, funding rates at most US funding bodies have kept declining, and they have now reached awfully low levels. These days, single-digit success rates are pretty common. At the NSF, in some competitions I’ve seen success rates of 5% or less. As a result, most US scientists are struggling to figure out how to make the best of this abysmal situation. One could view the entire granting process as a lottery, and say that at a funding rate of 5% it takes an average of 20 submissions to get one project funded. And certainly some scientists operate this way and just write grant after grant after grant. However, I’m not convinced that that’s a viable strategy. While there is certainly an aspect of randomness to grant review, and sometimes a mediocre grant gets rated much higher than it should while an excellent grant gets triaged, I don’t think that this randomness matters much at the top end. If funding rates were around 30-40% then yes, one could probably write 3-4 grants and expect one to be funded just by chance. But I will argue that the top 5-10% of grants, the ones in the currently fundable range, are fundamentally different. If your grant isn’t like one of them, it probably has a near-zero chance of being funded. And if it is, it may well have a 50% or higher chance. The trick to being funded is hence to only write grants that can score in the top 10%. Easy. (Yes, I’m being sarcastic here.)

    Read More
  • Virtual Books

    Blogs tend to be ephemeral. They are usually presented as a constant stream of novel contents, where the most recent posts are easily seen but the older posts can be difficult to discover. Even when blogs have excellent archive and search functions, I often wonder whether I have seen all the relevant posts on a topic. And the larger and more active a blog, the more of an issue it becomes. For example, one of my favorite blogs, Mark’s Daily Apple, updates daily and contains thousands of interesting posts. Its archive lists posts by a variety of different criteria, and its search function generally returns relevant hits. Nevertheless, I often have difficulty finding my way around that blog. I’m never really sure that I’ve read the most important articles on a given topic, in the right order.

    Read More