I have heard that some journals are rated more highly than others. Is this true? And if so, what are the criteria for judging the value of one peer reviewed journal over another? How do I find out its rating? Will my publication be of less "worth" if it is accepted in a less reputable journal than, say, the SIAM Review?
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ This question might be considered off-topic, subjective, or too broad. But I think it's worth discussing here and will write an answer. $\endgroup$– David KetchesonCommented Jan 15, 2012 at 5:15
-
$\begingroup$ @DavidKetcheson: That was my feeling as well. $\endgroup$– Geoff OxberryCommented Jan 15, 2012 at 7:36
Add a comment
|
7 Answers
$\begingroup$
$\endgroup$
What factors determine where I will publish a paper?
- Will the people I want to read this paper see it? If I'm following up on the work of another group (perhaps to show a different viewpoint, sometimes to show algorithmic improvements or to fix problems with a previous paper), I will want to submit the paper to the same journal, even if there's an impact factor issue.
As a young computational scientist still in the career development phase of my career, I would submit that there is an additional, and perhaps even more critical aspect to this question. . . .
Will people who may be in a position to evaluate me see this paper? I've often spoken with peers in the computational science field about the need to have a "home turf": computational science is a highly interdisciplinary field. Unfortunately, we are not really able to be considered as computational scientists when it comes time to be considered for a permanent position. In the absence of working in a computational sciences department, We will have to apply for tenure in an existing department, which usually means that our "peers" will be other engineers, scientists, and mathematicians—many of whom do not really have a strong background in computational science. This means that even if you want to go in the cross-fertilizing direction, you still need to focus some of your publications in the go-to journals for your discipline. This is a challenge that many of our peers will not necessarily have to face, and it's an additional complication in our lives. But it's something we have to be aware of before we start working!
How much competition do I have right now? The more crowded a field, the more important it is to get the results in early. While it's great to try to go to Nature or Science with every paper out of your group (presuming you're not in pure math, or something similar), being first out of the gate in a "hot" field matters much more than publishing in the "best" journal in a field.
How important is this paper? A paper that represents a body of work that provides a lot of new data, but not really much in the way of ground-breaking insight, probably doesn't merit going to a top-level journal. It's probably better to look for a reputable journal. However, if you've really found something big, shoot high, so long as you're not worried about the time crunch that affects the big journals. (It can take longer, for instance, to publish in Physical Review Letters than in one of the other Physical Review series, which are of comparable quantity.)
After all of these are taken into account, then I'll start worrying about issues like impact factor, but then only as a loose quality control measure. Differences of 10-20% are essentially meaningless, but a 1.0 versus a 2.0, or a 2.0 versus a 3.0, does represent a measurable level of difference between journals.
$\begingroup$
$\endgroup$
5
In an ideal world, you would only consider one thing when choosing where to submit a manuscript:
which journal's readership includes the broadest audience that would be interested in this work?
In practice, you also consider other factors, perhaps most significantly
How much will publishing in this journal help my career?
Anyone who can understand your work well will judge it by its actual significance, but those who are too far removed may use the quality of the journal as a proxy for the quality of your paper.
How is journal quality determined? Experienced professionals have their own opinions based on experiences with each journal and the articles they read. In some fields, like mathematics, this expert opinion can only be synthesized by talking to a lot of experts. In other fields, like computer science, this has been explicitly codified into a tiered ranking system.
The most widely available metric for journal quality is the so-called impact factor. The danger of having a single, widely used metric for anything is that it may be manipulated or misapplied, or may simply not be useful for some of the things it is applied to. For instance, since it only measures citations over two years, it doesn't make much sense for mathematics papers, which typically get most of their citations after two years -- often much later. It also seems to be poorly correlated with expert opinion. There are alternatives, for instance Eigenfactor. And many people are now proposing the use of social web metrics to rank journals.
I do not use the impact factor or other such metrics in my own decisions, as I feel that my direct experience gives me a more accurate measure of a journal's significance. An interesting exercise is to go through your personal library of references (e.g., in a bibtex file) and see the number of articles it contains from different journals. That will probably give you the best indication of where you should publish.
By the way, there is a proposal to create an expert-based ranking of math journals; see this report and this official IMU blog.
You should also be aware that there has recently been an explosion of new low-quality journals based on an author-pay model. Their publishers frequently spam large communities of scientists (often including many who are unqualified) in an attempt to get reasonably respectable names on their editorial boards and solicit paper submissions. As far as I can tell, they are profit-oriented vanity press.
answered Jan 15, 2012 at 7:23
-
$\begingroup$ How do you distinguish these 'vanity press' publishers from the quality, peer-reviewed publishers? $\endgroup$– PaulCommented Jan 15, 2012 at 22:02
-
1$\begingroup$ @Paul good question; it's usually fairly obvious. The signs are that they spam lots of people, enlist unqualified editors, and require that you pay in order to publish. They also usually have poorly written messages and websites (in terms of basic things like grammar and spelling). $\endgroup$ Commented Jan 16, 2012 at 6:19
-
$\begingroup$ In considering how broad the audience is, you may want to consider if the journal is open access and if the journal has data and code archiving policies. Research has shown 1 2 [3](Making data maximally available.) that open access journals receive broader readership and higher impact. Alternatively if you don't want to release data and code, journals with archiving mandates may be a poor choice. $\endgroup$– cboettigCommented Jan 16, 2012 at 17:37
-
1$\begingroup$ I agree with @DavidKetcheson's comments, but will add a few of my own. Having sat on the math Discovery Grant panel for NSERC there are many pitfalls in using journal impact factors to gauge quality of individual papers. Even the best journals contain papers with a broad range of quality, and mediocre (or erroneous) results do slip through even the best of peer-review processes. That said, I would still find the list proposed by the IMU/ICIAM or from the AustMS (mentioned by @Dirk) to be useful. $\endgroup$ Commented Jan 16, 2012 at 17:39
-
$\begingroup$ @cboettig In math, I suspect that actual readership may be negatively correlated with open access, due to the unfortunate deluge of low-quality "open-access" publishers. Of course, the arXiv gets to a very broad audience! It's my preferred open access "publisher". $\endgroup$ Commented Jan 21, 2012 at 7:15
$\begingroup$
$\endgroup$
1
It really depends on how much you believe in journal ratings. The most popular metric I am aware of is the impact factor. This Wikipedia article describes the impact factor, how it is calculated, and how it is susceptible to manipulation, as well as alternative journal rating metrics. The most important thing worth noting is that impact factors for journals in different fields aren't necessarily comparable, even though the impact factor rating system tries to collapse the "quality" of each journal onto a single metric. (For an interesting discussion on why such an idea can lead to misleading or easily manipulable results, see Malcolm Gladwell's New Yorker essay, The Order of Things, which discusses college rankings and car rankings.)
The impact factor is the average number of citations received per paper published in that journal during the two preceding years, so the criteria for judging the value of one peer reviewed journal over another is how many times its recent articles get cited on an average, per article basis. This metric does mean that journals like Science and Nature are ranked highly, as one would expect, but it also means that International Journal of Nonlinear Sciences and Simulation (3.100) gets ranked higher than SIAM Journal of Scientific Computing (3.016), even though the latter journal is a lot more recognizable.
As for your publication being of less "worth," it depends on the person. I've been told by my more senior colleagues that having articles in more prestigious journals is helpful for things like tenure and hiring decisions; not having experience with such matters firsthand, I defer to the comments of people who are have more background with those sorts of situations. It depends on the community of people you are trying to reach or impress. If you publish something in a math journal, don't expect that people outside mathematics will necessarily read it. (The same advice applies to other subjects.) Also, you may decide to publish in a less prestigious journal because you want to get a result out quickly rather than wait a long time for it to be reviewed by more prestigious journal. I've been told that the SIAM journals can take a year or more to review a paper and publish it, which is backed up by looking at the time between the submitted date and the published date on the paper. Submission to a particular journal is also dependent on aims and scope, politics (who is on the editorial board, who is likely to review your paper), what community you want to reach out to (sort of goes with the advice about subject area above). I still consult other people on these matters.
When I look at articles, the "prestige" of a journal, to me, is really my internal feeling that reflects the valued opinions of other people I know in science (my advisers, colleagues, etc.). I'm more inclined to trust an article in a journal that is more prestigious because I feel it is more likely to be reliable, but there are certainly bad articles in prestigious journals, so prestige is not a replacement for doing your own due diligence. For example, there are some pretty serious typos in an otherwise good journal article I've cited from SIAM Review.
Anyway, that's my rambling opinion, and as a (nearly) freshly-minted PhD, I'm sure that others with more experience will have much more perspective and insight to add.
answered Jan 15, 2012 at 5:31
-
$\begingroup$ Good summary. Wait times depend as much on subject matter as "prestige". Math heavy papers take much longer to get reviewed than a software oriented paper, for example, at least in my experience. Thanks for the New Yorker link. It is a good read. $\endgroup$ Commented Jan 16, 2012 at 18:02
$\begingroup$
$\endgroup$
Rating is necessarily a subjective activity. If you care about how others will rate your paper, then you need to figure out what their criteria are. As a rule of thumb, the less competence they have in your field of work, the more they will rely on journal rating rather than rating your work, and the more they will rely on bibliometry (impact factors, h factors, ...) rather than personal experience.
$\begingroup$
$\endgroup$
3
This is a conversation you should be having with the people above you that are going to evaluate you. If you're a PhD student, then you should be talking to your advisor and other professors about how their department and other departments at other institutions evaluate the work of tenure-track faculty when making their tenure decisions. If you're tenure-track faculty now, you should have already had this conversation with your department chair and other members of the department's budget council or similar bodies about how they evaluate the work of tenure candidates. If you're not a professor of some fashion, or if you're an academic in a place without tenure, you should ask the same questions of your supervisor about promotion and retention at your institution.
My experience has been that most researchers don't really read or try to keep up with publications in journals relevant to their areas directly as they come out. Not many people have personal subscriptions to IJNMF, say, and read each month's issue cover to cover. There are just too many venues to keep up with. More often, I find, people are searching for related work, using email or google alerts to seek out related items, and following the work of colleagues and competitors directly. So, the important thing is not trying to get your work seen via journals, but to publish in the best possible venue as determined by those who will be judging your career.
The way to get your work seen is to aggressively market it through conferences, seminars, free software, your website, etc. This probably won't impact your promotions committee's decision (or the like), but if people are seeing and using your work through other venues, and you give them preprints and links to where it is published in the journals, then that will increase your citation counts, which is something important to those above you.
answered Jan 15, 2012 at 20:04
-
$\begingroup$ I for one do read journal tables of contents. I use an RSS feed aggregator and it only takes a few minutes per week to keep up with about a dozen journals. I also typically get the articles well ahead of their publication date this way. You can see which ones I read here: scienceinthesands.blogspot.com/2011/10/… $\endgroup$ Commented Jan 16, 2012 at 20:01
-
$\begingroup$ I figured that there were people that kept up more than I do. I'm mostly out of the academic publishing scene (not being on that track). It's certainly easier to keep up now than when everything was paper only. Just out of curiosity, how many articles out of that collection do you end up reading in a month? $\endgroup$ Commented Jan 20, 2012 at 22:41
-
$\begingroup$ Probably only 1-2 in any depth. But I get a lot of value out of being aware of work by reading some abstracts. I'll probably read on the order of 25 abstracts in a month. The rest are discarded after scanning the title. $\endgroup$ Commented Jan 21, 2012 at 7:11
$\begingroup$
$\endgroup$
4
My short answer is: Unfortunately, there is (almost).
There is the rating by the Aust MS for mathematical journals and I think there are other ratings as well. However, in my opinion this is not helpful. I quote from my own blog: "My opinion is that rating is possible but both useless and dangerous. It is possible, since you can ask experienced mathematicians and you will get a reliable answer. It is useless, since you either know which journals are good or you can ask a colleague (which is basically the same reason as the previous one). It is dangerous, since it offers the possibility to form decisions on tenure or grants on these numbers and shifts the focus from what you publish to where you publish."
-
$\begingroup$ Sadly, you do need to worry about both what and where to publish—at least if you're at the stage of your career where you're concerned about needing to find another job. $\endgroup$– aeismailCommented Jan 15, 2012 at 18:03
-
$\begingroup$ Well, the focus should be on what is in the paper. Anyway: If you are very good then it will not matter where you publish. At least the mathematical community is quite resistant against measures like the impact factor and relies on opinions by experts (i.e. referees and recommendation letters). $\endgroup$– DirkCommented Jan 15, 2012 at 18:34
-
$\begingroup$ If you're in a field like math, that's not so much of a problem. However, we don't know specifically which field the OP is in, and so there's a very good chance he might be evaluated by non-computational scientists. Then the whole issue of knowledge versus impressions comes to the fore. $\endgroup$– aeismailCommented Jan 15, 2012 at 18:55
-
$\begingroup$ I guess there are equally as many people on both sides of the issue (whether it's important from a pragmatic perspective)... In some institutions, it DOES matter where, in others, it matters WHAT. $\endgroup$– PaulCommented Jan 15, 2012 at 22:09
$\begingroup$
$\endgroup$
There are some ranking systems for citations, which one might argue are decent measures of prestige - important papers get cited more and the like - the most common of these is "impact factor". But these all have their own problems - they're not constant by field, they're susceptible to manipulation by clever researchers, etc. Others that are slightly better are the h-index and g-index, but...meh.
Generally, I've found there is a "feel" in the field for what's good and what's not - and more importantly, what's appropriate and what isn't for your paper. For example, I have one paper that managed to get into a very good journal, but its utterly inappropriate, so it languishes a bit.
Your evaluators or colleagues would probably be the best sources to talk to. Experience is often the best judge of what counts as "good" for a particular paper that would lend itself well to a particular journal. General field (Biology, Medicine, etc.) rankings, or even worse, Academia-wide rankings may not be of any use.
