I think the web should be used more optimally for criticism of arxiv.org papers.
I do not think it is optimal to visit various blogs to try an cull accurate criticisms of an arxiv paper. They should be available in the same place.
I propose....
Having a 'disputation' wiki built right in to arxiv.org. I'll call this a d-wiki. It should allow both prose criticism/analysis (essentially blog-type posts), as well as bulleted factoids (probably elaborated in the prose comments) that are in dispute.
At least an attempt should be made as allowing for a hierarchical structuring of the bullet points. ("attempt", because working this out would be difficult). I think it's safe to say that, ultimately, all arguments are hierarchical. But, of course, we assume that the mathematics is normally correct. The hierarchal bullet points would typically refer to physics arguments.
The disputation bullet points can be associated with individuals and groups (meaning that they view the bulleted item as being worthy of being called "in dispute"). Furthermore, individuals and groups can rate the relative importance of the bulleted items (within the context of a hierarchy, if there is one.)
It's very important to allow filtering. Professional physicists, e.g., are probably not going to be very interested in the comments of laymen (such as myself
). Furthermore, even professional physicists are not going to be interested in the comments of other professionals. E.g., Motl believes that Lee Smolin is a "crackpot", and (from reading Smolin), it's clear that many string theorist (such as Motl) have an almost religious attitude towards their flavor of theorizing. Motl may not want to read anything by Smolin - thus, he should be able to filter out Smolin posts on d-wiki (and vice versa).Very likely, not every user of a d-wiki is going to want to make the decision as to who to filter on their own. They may, instead, want to inherit the filters determined by somebody else, whose judgement they trust. Or, they could inherit filters from a group, whose 'group judgement' they trust.
A sophisticated filtering system would allow for exceptions to filter through, based on significance. E.g., suppose that one generally agrees with Motl, and thus is willing to inherit filtering criteria from him. Nevertheless, since Motl is not God, and since Smolin is a very smart guy, one might want to allow d-wiki posts from Smolin to filter through when Smolin rates a disputation bullet point as maximally important. Or, one might want to allow d-wiki posts from Smolin to filter through when Smolin rates the arxiv paper itself, which is being disputed, as maximally important. Or, some combination of these two.
This post was motivated, in part, by Motl's comment in the above-referenced post:
QUOTE
A few years ago, such a paper would almost surely be filtered out from hep-th. Paul Ginsparg has introduced the endorsement system which was circumvented in this case and is likely to become a complete joke in the future. Why? Well, we have seen that a completely continuous spectrum of people between serious physicists and manifest crackpots has been created and the recent fashionable trend is to accept an ever broader set of passionate amateurs and undereducated, intellectually challenged loons into the physics circles.
This paper by A. Garrett Lisi had to be endorsed by someone. If you read the acknowledgements, it is not hard to see possible answers. Some of those people such as Lee Smolin may endorse any crackpot paper because they are both endorsers and crackpots at the same moment. Moreover, they have a vested interest to increase the proportion of similar papers on the arXiv because this is where they belong. As Lee Smolin recently pointed out, irrationality has been extremely useful for him in the past.
If Ginsparg wants to prevent this possible collapse of his arXiv, he probably has to fine-tune the mechanisms a little bit and make sure that people who are ready to endorse papers like this one are simply not endorsers. Otherwise you can be pretty sure that similar papers will eventually overrun the arXiv.
Tony Smith is among the crackpots thanked to in the acknowledgements. Next time, he may also submit his own paper supported by similar endorsers. And maybe A. Garrett Lisi will become an endorser himself. Really entertaining times will start afterwards: the hep-th era of the UFO abductee Jack Sarfatti, Tony Smith, Peter Woit, and their friends.
Of course, an endorsement system of some kind is makes sense. But rather than censor papers because Motl thinks that they have no redeeming physics value, it would be far better, IMO, to allow broad-based publication, but with elective filtering. With some tweaking, it would be an easy matter to allow ratings to be generated by members of a d-wiki community. In particular (and to afford some anonymity), ratings would be associated with groups (which share inherited filters from a few trusted members).
I don't believe that arxiv.org requires thought police, which can outright censor papers that some Ph.D. physicists might find value in, just because other Ph.D. physicists don't. Even if a paper is 90% wrong, within the 10% that is right, might be an original idea that can bear fruit if developed properly.
We need a way to avoid throwing out the baby with the bathwater, even if the baby is a preemie, and can fit in the palm of one's hand.