« Will a voluntary census be more truthful? | Main | The experts weigh in: Four very good commentaries on the census in Canadian Public Policy »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

"Fair enough, but publishing a journal costs money, and who is going to pay the bills?"

Which leads to the follow-up question, "Why have journals at all?"

Mike, journals add value by not publishing research - they filter out work that uses dubious research methods, is unoriginal, inconsistent or illogical. That an article is published in the Canadian Journal of Economics is usually a pretty good indication that it isn't totally bogus - and that's something that, as a researcher or a student or a teacher, you want to know.

Can you imagine evaluating a candidate's suitability for tenure or promotion without journals? Yes, it would be possible to find an outside expert (how would you know who is an expert?) who could read each paper and comment on its intrinsic merits, but I sure as heck wouldn't like to be that outside expert.

Thanks for the informative post. As a professor who assigns a lot of CPP articles per term I have to admit that I link to the UTP .pdf file as opposed to the Project Muse files. I never really thought there was a difference between the two, now that I do I will update my links.
This highlights two problems with the journal content access process where incentives are not aligned efficiently. First, authors care more about citations than the journal's objectives and hence have an incentive to distributed their materials as much as possible (unless they somehow a) intrinsically internalize the externality or b) are scared of being caught for copyright infringement which would depend on the fine associated with this). The second issue is the distribution by instructors. As an instructor I have been wooed by textbook publishers to use their books, something I don't much care for but many colleagues do. I have never been wooed by a publisher to access their content for my course, or to encourage me to access their material in a particular way.
Now that this has come up, perhaps at the end of term I should submit the syllabus for my undergraduate public economics course to the CPP so that the journal can use it to help promote the journal as a resource for instructors of similar courses. If every journal had the "pay per click" objective then I would suspect journal publishers would start giving back to the academic community by making access to their content easier for instructors so as to compete with the bloated textbook market.

"Mike, journals add value by not publishing research - they filter out work that uses dubious research methods, is unoriginal, inconsistent or illogical."

Is there any actual proof of this?

Megan McArdle had a pretty good take on this a couple of months ago: Peer Review is No Panacea.

"Can you imagine evaluating a candidate's suitability for tenure or promotion without journals?"

Sure - use a metric similar to Google's Page Rank on how many times someone's work has been cited by others (which can go far beyond just citations in other people's papers).

Mind you, I'll admit I'm biased by my own experiences with the peer review process:

Here is one example I experienced last year. I submitted a paper to a journal and one of the "revise and resubmit" reviews I received back blasted me for not including in the literature review a paper I had never heard of. I was not surprised by this - as hard as I try I cannot keep up with all the journals, so some excellent papers do go unnoticed by me. I eagerly sought out this paper, to see what I could learn, and was amused to find that:

1. The paper was only available as a "working paper" on the author's website (a grad student) and had never been published.

2. The paper had little to do with the topic of my paper, except they both mentioned "price" a fair bit.

Needless to say, the review was done by someone who was using the review as an opportunity to have their own work cited. Not very useful and certainly not objective.

Ross "This highlights two problems with the journal content access process where incentives are not aligned efficiently." Thank you for expressing the point so elegantly.

Mike, "the review was done by someone who was using the review as an opportunity to have their own work cited." Yes, that happens. Journal editors are sometimes major offenders because one of the best ways of boosting a journal's ranking or impact factor is to make sure every paper that is published in the journal cites other papers published in the journal. (I do know one journal editor - not the CJE or CPP editor - who does this as a matter of policy.)

"use a metric similar to Google's Page Rank on how many times someone's work has been cited by others" Sometimes this works, e.g. Nick Rowe's posts on this blog get more comments than mine (usually) because they're better than mine (usually). Sometimes it doesn't, e.g. Nick Rowe's posts get more comments than mine (usually) because his deal with US/international macro policy which is of interest to lots of people, and mine deal with Canadian micro policy issues, which are of interest to a relatively small number of people. Popular does not always mean good - otherwise we would all be posting pictures of lolcats.

Frances - I always read your articles, because I can usually understand them! When I read Nick's, I am both amazed at his erudition and simultaneously realize how little I understand about macro and money!

Frances, "Remove fishing for compliments", and "Resubmit".

Instead on putting links to publisher pages, I always recommend to people to put a link to the abstract page of the paper on EconPapers or IDEAS. It is a way to reward the author of the paper for good work, and it provides links to other, possibly non-gated, versions of an article that is password protected. It also allows to get a better measure of readership.

Note that I especially encourage blogs to put such link. We have a project that, if it goes well, would provide link from RePEc abstract pages to the initiating blogs. But this will only work if the initial link is to RePEc.

Christian Zimmermann

Christian, thanks for this suggestion. For the Beaudry and Green paper, IDEAS links to the NBER version (restricted access) and to the CEA website, which in turn links to the (restricted access) JStor and Wiley-Blackwell pages. The IDEAS page also gives ways of finding related research that is not restricted.

This is a way of resolving the author/journal conflict Ross Hickey mentions.

And sometimes Nick's are just wrong, and lots of people comment to tell Nick he's wrong!

Gregory, yes, you're right, I should "remove fishing for compliments" because this is becoming highly embarrassing. Nick Rowe's comment makes the point I wanted to make: the fact that something is cited doesn't guarantee that it's good quality work.

Can anyone tell me whether the CJE is in a legal position to restrict me from posting my own published work on my website? I mean, I know that they can tell me I am violating something, but am I really? Do we not have fair-dealing (fair use?) laws in this country, or are they irrelevant in this case? Just wondering.

My understanding is that the CJE owns the copyright for the pdf file that it makes available. If you take (say) your final submission and/or a working paper version and post it somewhere, there's no problem. But if you take the file from the CJE site, there is.

"Nick Rowe's posts on this blog get more comments than mine (usually) because they're better than mine (usually). Sometimes it doesn't, e.g. Nick Rowe's posts get more comments than mine (usually) because his deal with US/international macro policy which is of interest to lots of people, and mine deal with Canadian micro policy issues, which are of interest to a relatively small number of people. Popular does not always mean good - otherwise we would all be posting pictures of lolcats."

But this really has nothing to do with a system that ranks papers based on number/quality of citations, such as PageRank. PageRank is not a raw count of popularity, as shown by Google's patent.

As well, aren't top-tier journals more likely to publish articles they believe will be popular or well-cited than ones they believe wouldn't be? Can not your criticism above be used to argue *against* examining where an item was published?

So, where, exactly is the value-added that journals provide? The peer-review process is no guarantee of quality, which I think we can all agree on. And there are ways of measuring impact beyond whether or not something is an A-pub or a B-pub. If we take a strictly 'which journal was the piece published in', then the 'General Theory of Employment, Interest and Money' scores a zero because Keynes published it in a book instead of sending it to Econometrica. Does that make any sense?

In other words, sure 'the fact that something is cited doesn't guarantee that it's good quality work.' But neither does the fact something appears in an A-pub. So where's the value-add?

David, before a CJE article goes to print, the authors are required to sign a copyright form (I think it may actually technically be called an 'exclusive license agreement'). I couldn't find a copy of the form on the CJE web page, but I do know that the CJE copyright form is very similar to the Canadian Public Policy copyright form because one is modeled on the other.

This is the relevant passage:

Please note: You retain the following rights to re-use the Article, as long as you do not sell or reproduce the Article or any part of it for
commercial purposes (i.e. for monetary gain on your own account or on that of a third party, or for indirect financial gain by a commercial
entity). These rights apply without needing to seek permission from Canadian Public Policy/Analyse de politiques. Provided that you give
appropriate acknowledgement to the Journal and full bibliographic reference for the Article when it is published, you may use the accepted
version of the Article as originally submitted for publication in the Journal, and updated to include any amendments made after peer review,
in the following ways:
• you may share print or electronic copies of the Article with colleagues;
• you may use all or part of the Article and abstract, without revision or modification, in personal compilations or other publications
of your own work;
• you may use the Article within your employer’s institution or company for educational or research purposes, including use in
course packs;
• 6 months after publication you may post an electronic copy of the accepted version of the Article (not the Canadian Public
Policy/Analyse de politiques PDF file) on your own personal website, on your employer’s website/repository and on free public
servers in your subject area.
• Electronic versions of the accepted Article must include a link to the published version of the Article together with the following
text: ‘The definitive version is available at http://dx.doi.org/,’ where refers to the digital objective identifier for your
paper, Information on how to obtain the digital object identifier for your paper is at the end of this form.
Please note that you are not permitted to post the Canadian Public Policy/Analyse de politiques PDF version of the Article online.
Instead, please include a DOI link to the article published in the online version of Canadian Public Policy/Analyse de politiques.

Where things get a little murkier is with respect to older articles, because before around 2003 the CJE copyright form was quite unclear about electronic publication.

There are also issues with respect to work funded by the US government - I don't know how that works at all, but it's the reason that the propaganda cartoons Walt Disney made during World War II can be posted legally on Youtube.

If you want to figure out what the policy of various publishers is in this regard, check out SHERPA/RoMEO. Note that authors can always try to negotiate a less restrictive policy, and often another one is available from publishers. There is also a copyright form suggested by SPARC. There is more in my RePEc blog post.

Christian Zimmermann
(whose account name really looks strange)

Can someone please explain why publishing a journal should cost so much money? Theoretical Economics, for one, seems to be able to publish an open-access journal online without too much trouble. A reasonable submission fee should be able to take care of whatever costs arise.

Seriously, what is chewing up all the money here? It costs virtually nothing to allow online access to a journal article. Referees -- the key participants in peer review -- are rarely paid. Editors aren't compensated much and primarily serve for the benefits of professional reputation (and it certainly seems like this compensation could be supported via a submission fee!).

"In other words, sure 'the fact that something is cited doesn't guarantee that it's good quality work.' But neither does the fact something appears in an A-pub. So where's the value-add?"

Shame on you, Mike. You're attacking a straw man.

The question you posed is "Why have journals?" The immediate answer was "They filter good from bad."

Since then you've been arguing that they are imperfect filters. An economist might instead wonder whether their marginal benefit exceeds their marginal cost. I'm sure you'll agree that even very imperfect signals (e.g. stock market prices) can convey very valuable information.

"Is there any actual proof of this [that journals add value]?"

Um....did you ever get a useful suggestion from a referee or editor that made your paper better? Did you ever *write* a useful suggestion in a referee report?

"Sure - use a metric similar to Google's Page Rank on how many times someone's work has been cited by others (which can go far beyond just citations in other people's papers)."

Actually, you'd probably want to use citation reports, which are available from various vendors (e.g. Scopus, ScienceDirect, etc.) and Google Scholar. Better still, you can see an overall picture of someone's contribution using software like Harzing's Publish or Perish (see harzing.com). Of course, these are pretty imperfect tools as well (hmmmm.....it tells me Mike Moffat has an h-statistic of zero. Perhaps he publishes under Michael?)

However, if you read the literature on citation analysis (Anne Harzing has contributed a fair bit to this herself) you'll see that its openness to abuse is problematic (just as there are dozens of scams to artifically boost page's PageRank.) Just as PageRank abuse didn't become an issue until people based compensation on PageRanks, citation abuse is not nearly as problematic now as it could become if it became the standard academic measure of value-added.

I think you're suggesting that we replace a flawed system with a deeply flawed system. Is that what you intended?

Random Student -

The editors of the Canadian Journal of Economics and of Canadian Public Policy both receive financial compensation that's about equivalent to the cost of a course release. I can't imagine anyone taking on either job with out a course release.

A few years ago I cranked through the numbers for the CJE (as secretary-treasurer of the CEA I had all the data). If we eliminated all publisher-related distribution costs e.g. printing, mailing, etc the remaining costs worked out to around $300+ per submission - this includes a payment to associate editors for handling manuscripts.

And this is for me the deciding factor: moving from a subscription model to a submission fee model would benefit current subscribers (e.g. Harvard University library) and hurt current submitters (individual academics, including junior scholars and graduate students).

Why would we do that?

The question you posed is "Why have journals?" The immediate answer was "They filter good from bad."

They don't, though. This was an assertion, provided without evidence, which I don't happen to agree with.

"Um....did you ever get a useful suggestion from a referee or editor that made your paper better? "

Never. It's funny - Over the years, I've got incredibly useful contributions from seminars and conferences. In fact, I can't think of a single talk I've ever given that I didn't provide something valuable.

Every referee report I've ever received has been a complete and utter waste of time and whose only purpose was to promote the interests of the reviewers. There was, in fact, a cost from the review as I had to make a bunch of paper changes to promote the interests of someone else.

"Since then you've been arguing that they are imperfect filters. An economist might instead wonder whether their marginal benefit exceeds their marginal cost."

See above. :)

"Did you ever *write* a useful suggestion in a referee report? "

I'd like to think so, but you'd have to ask the people's whose papers I have reviewed.

Random Student: Great question - I hope you get the answer.

"I think you're suggesting that we replace a flawed system with a deeply flawed system. Is that what you intended?"

Perhaps - but you'll note that my deeply flawed system is:

a. Cheaper.
b. Doesn't gate off research to laypeople who don't have access to JStor.

Then there's the whole Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children issue. How much less power would the "vaccines cause autism" movement have if they didn't have a decade of being able to claim their theory passed 'peer review'.

You may not agree that my proposal is an improvement. But can we at least agree the current system has a great deal of room for improvement?

Frances, one small part of this I find rather confusing. If you are a lay person not going through a university or a library, it seems you can still read an article from Canadian Public Policy, but you must pay. For example, in the most recent issue, the article "The Declining Retirement Prospects of Immigrant Men" is available for download at the price of $13.00. As an economist, can you explain the model that yielded that price? And, as business editor, can you reveal how many articles have actually been purchased at this price (or similar single download prices)? What is being optimized here? Is the target demographic populated exclusively by George Soros?

Gregory - I'm embarrassed that the honest answer is: I have no idea - either how that per article price on Metapress (that's the site you were on) is determined or how many we sell at that price. I've actually never noticed that we do per article sales on that site, and don't remember ever discussing the price with our publisher.

We also make per article sales on JStor. That price is determined by the JSTor administration fee (which is either $4 or $5, I can't remember) plus a number that makes it worthwhile us cashing the cheque ($4, $5, something like that). I get four cheques from JStor per year for individual article sales, and at most we sell maybe a dozen articles each quarter (generally people search for a free on-line version, write to the author and ask for a copy, or read something else). I would imagine that the number of sales at $13 per article is more or less the same.

The big money is in institutional subscriptions. Canadian Public Policy is a registered charity, and ideally we would like to break even. However given that, like most other journals, we are seeing gradually declining subscription numbers, I've taken an extremely conservative fiscal position, and tried to build a little bit of a cushion by increasing institutional subscription prices when I figured I could do so without losing subscriptions (or any more than I would have done anyways). Our individual subscription price has remained more or less unchanged for years.

The Canadian Journal of Economics is published by Wiley-Blackwell, an international publisher, while Canadian Public Policy is published by University of Toronto Press. We have had extensive discussions about moving to an international publisher - it would increase our costs substantially, but it would increase the international profile of our journal, bringing it to more libraries and making it more widely available internationally.

The target demographic is mostly university students and researchers in government and academia - realistically, most other people would rather read a blog post on the subject.

One thing that the journal is doing to broaden the readership is 'unlock' articles for a few weeks so that people like WCI readers can access them when they first come out. We're doing with that some commentaries on the census - one by WCI regular Kevin Milligan - I think Steven Gordon may be blogging on these soon.

Frances:

I'm not still not sure that I understand the costs of producing a journal -- admittedly this is not a topic where I have any personal experience! Does that $300 per submission include only payments to editors and associate editors? Or does it include some other costs of publishing? (Other than printing, etc., which you mention are excluded.)

My sense is that a reasonably prestigious journal should be able to garner enough money in submission fees (say, $100 a pop) to provide compensation to editors that is in line with professional norms. But I still don't think this is really necessary -- I'm at a PhD program in Cambridge, and the incentive value of compensation for most of my professors who edit journals seems minimal. They do it (1) out of a sense of professional responsibility and (2) out of a strong desire to influence their field.

I can't say I agree with your argument about how subscription fee models are superior. Sure, it sounds nice to say that deep-pocketed institutions like the "Harvard library" support the current system -- I don't oppose restricted-access journals because I think Harvard desperately needs more money! The real problem is that people who are not affiliated with a rich academic institution have no reasonable way to access these journals, which amounts to a real and completely unnecessary barrier to spreading academic knowledge.

Note also that providing knowledge to more people has zero marginal cost, while reviewing articles has positive marginal cost (in fact, under modern electronic publishing, this should really be the only cost of running a journal!). It doesn't seem so outlandish to suppose that fees should be charged for the part of the process that actually carries costs. Since an $100 submission fee is far below the massive investments in time and energy that are necessary to prepare an article for submission, it doesn't strike me as much of a burden (and I say this as a grad student!). And for authors from developing countries for whom $100 is a more intimidating sum, it can be waived.

I also suspect that many of the "costs" of running a journal -- both in money and time -- are really just products of imperfect adoption of technology. Maybe no open source content management systems are good enough yet, but it should be possible to make the "low-level" tasks involved in running a journal -- monitoring the stream of papers, keeping track of referees, etc -- virtually automatic, leaving editors to perform only the high-level tasks that are consonant with their background and interests.

Yes, I happily agree with Mike that the current system needs improvement. For the record, I've been telling most things that move and a few that don't just how noisy a signal journals provide and that, currently, citation indices provide better signals.

I also happily disagree with his view that the journal filter process produces no value added. I see a real difference between the content of Empirical Economics, Econometrica, the Canadian Journal of Economics, and the American Economic Review. That doesn't mean that low-status journals don't publish good papers and vice versa. I think that even if all journals did was signal quality (as opposed to actually killing some really bad papers and improving some others), that would be a valuable role.

But I think refereeing often adds value. I've gotten good and bad criticisms from referees. I mostly ignored the bad and kept the good. That added value. I've gotten constructive suggestions from editors and referees. That added value. As a referee I think I've made suggestions that made papers better when they were published, and I think I've helped kill some papers that I thought were rotten to the core. I think I helped provide a public good.

Frances, I suppose the only thing that matters for the foreseeable future is the subscription side of the business. Nevertheless, I can't resist a few comments on the single download business. If JSTOR and Metapress are charging around $5 as an administrative fee, let me suggest that this is truly impressive. Presumably, that fee covers payment processing, and a little bit of customer service ("help, I lost my download", or "I got charged twice"). The market price for payment processing is pretty low, and going down. For example, PayPal would charge you about $.30 to get you five dollars in net sales revenue, and there would be no hassle cashing cheques because the money could be transferred electronically to your bank account. Obviously, there are significant inefficiencies built into the non-subscription side of the business. If subscriptions are declining, it might be worth thinking about this. The music industry thought too late.

Random Student - you may well be right that, in the long run, we may end up with a submission fee based system. But how to get there from here - that is, how to transfer the cost of paying for journals from library budgets to academic researchers' budgets - is something I'm not seeing. What seems more likely is that library budgets will be cut, marginal journals will go out of business and....?

Here's more details on some of the costs for Canadian Public Policy:

CPP pays $14 Cdn per page for copyediting, plus more for typesetting, so that works out to about $20 to $30 per page per accepted article for copyediting and typesetting alone. Copyediting is vital - many economists can't write very well.

The cheapest editorial management software (Editorial Express) costs around $2,000 US per year, or more for a larger journal.

Maintaining a webpage costs money. Marking up files costs money. Sending out contents alerts costs money.

$100 per submission wouldn't cover the payments to editors at the CJE, let alone all of these other costs.

So why do we pay editors? Think about the academic reward system. Perhaps the UK Research Assessment Exercise places some weight on journal editing, but there's rarely any rewards for refereeing, and not much of a reward for editing relative to the time involved. People will say yes to a request to referee a paper and then take months to submit their report. Sometimes people just ignore requests. Sometimes one has to ask several referees before finding one who will agree to look at a paper. Professional responsibility and wanting to influence the profession sounds fine, but when you've got family waiting for you, and one hour editing a journal is one hour away from your kids - well, that's a pretty steep price to pay.

Perhaps $100 doesn't sound like much for a submission fee. But it's not uncommon to have to send a paper out three or four or five times before it's accepted. If you're a new faculty member, and trying to get three or four articles out in the first year, and getting several rejections - that's a lot of money to spend on lottery tickets.

Gregory - I've always just ignored the single download side of the business because it seemed trivial - per download payments make up 1-3% of the total revenue that we get from JStor. Like I said, I don't know about the University of Toronto Press site.

Simon and Mike - I'm staying out of this one.



Random student - p.s. as you know, most economists also publish their articles as working papers that are freely downloadable. So although there is a gated knowledge community, there's an open access community next door, with almost as good, and much cheaper, articles.

Mike - on JSTOR; google will usually turn up free copies (working copies, sometimes later versions with additions). I've been reading stuff about agent based computational economics recently (fascinating stuff - and the math is much easier).

Running an open access journal is much cheaper than you think. There are several free, good and popular journal management software out there. Hosting is nowadays very cheap. You do not have the print costs. And you do not have the costs of preventing non-subscribers to access your journal, and no administrative costs associated with financial transactions.

I guest edited not long ago a special issue of Economic Analysis and Policy on the Economics of Open Access publishing. The consensus from the articles was that open-access is very viable and cheap, as long as you have dedicated editors. It is even more viable if it has the support of a society, which would be the case of CJE and CPP. I have to admit, though, that I did not manage to get a submission from a commercial publisher who would explain why its high costs are justified, or why it takes several thousand dollars to put an article in open access. In any case, if you are interested in this special issue, there is more about it on the RePEc blog.

Christian Zimmermann

"I've always just ignored the single download side of the business because it seemed trivial - per download payments make up 1-3% of the total revenue that we get from JStor. Like I said, I don't know about the University of Toronto Press site."

What you need here is something you are sorely missing - the click-through stats. JStor - or any site that does single sales - needs to be able to tell you how many readers reach the single-sales page -- and stop there. I know my reaction, although it's often a page that offers the article for $25, not for $13. Moreover, the nature of papers makes it hard to assess whether or not the $25 is worth it -- the paper may have little value to me, but I don't have a good way of assessing that before I read it. For academics, this is a key advantage of the citation system; it's based on what you think after you read the paper, not before.

I've made almost exactly the same point in regards to ebooks vs books. I think many ebook readers are using ebooks (especially illicit ebooks) as a way to evaluate the worth of a title, and then following through with a paper purchase only for the good ones. As for the bad ones? Since they didn't actually buy them, they may never actually finish reading them.

Back to journal articles - it strikes me that there is some basic economics not being respected in this whole process, possibly phrased as "what is the demand curve for journal articles?" Would you change something if you found that you could sell 20,000 copies of an article at $1 each?

"Perhaps $100 doesn't sound like much for a submission fee. But it's not uncommon to have to send a paper out three or four or five times before it's accepted. If you're a new faculty member, and trying to get three or four articles out in the first year, and getting several rejections - that's a lot of money to spend on lottery tickets."

A little transparency is in order. The submission fee should cover the costs of 'submission'. If an article is accepted, then I can see a separate fee for 'publication prep'.

Moreover, this should have the positive (but minor, I admit) side-effect of having researchers write better papers, or maybe fewer but better papers. Rather than spin something into three rough attempts, maybe having to pay for each submission might encourage them to write one good one, and only pay once.

Chris S - I seem to recall that the Canadian Journal of Agricultural Economics requires authors to pay a per page publication fee. There is much to be said for this, not least because it would give authors a financial incentive to cut unnecessary verbiage(or as I'd say if I were paying per character: "it's a good idea because people will write shorter papers").

This question of whether or not submission fees give people an incentive to write better papers is something that people who manage journals think about a lot. A couple of things - people aren't always good judges of the quality of their own work, especially new scholars. And many scholars will have research grants or other ways of paying for submission fees (at a minimum, writing off as a tax deduction against consulting income). So the fees will be no barrier to scholars who are already successful, but a more major barrier to new scholars.

Submission fees do have an effect on the number of submissions, but I don't have a sense of how they affect the quality of submissions. Perhaps a journal editor might comment.

Gregory, I just received some correspondence from our editor at University of Toronto Press (who read this blog and was impressed by the quality of the comments). It looks like we could increase our revenues by decreasing our price. Thanks for the suggestion - once again proving that blogging pays.

Some comments from Herb Emery, editor of Canadian Public Policy.

I have two observations. One, CPP used to have a $40 submission fee and submissions seemed to be lower when it was in place. It was replaced with a subscription requirement for submitters. So a submission is not asolution to the problem of financing a journal and likely makes sense when you are trying to cut down a high volume of submissions.

Second, if some scholars feel that there is no value added to peer reviewed journals, then they can choose not to submit, not to subscribe, and not to read the papers that appear in them. We do see senior, established scholars choose to do this. In part their reputations built on their record of peer review publication gives them a sort of "legacy filter".

Cheers
Herb

The comments to this entry are closed.

Search this site

  • Google

    WWW
    worthwhile.typepad.com
Blog powered by Typepad