They're divided, somewhat arbitrarily, between General questions (about the goals and politics of the system) and technical questions (about its internal workings and how it might get implemented).
This paper is an ever-changing draft, and I'd love to hear peoples' responses to the ideas here. Email comments, criticism and ideas to misha@the-wire.com.
Technical
How it would work
The system would keep a large index of addresses of individual web pages and their comments, allowing modified browser software to look up all the comments that had been added to a given page. The specifics of how this would work, and how such a large database would be maintained, are still open questions.
Why it's important
For centuries, the technologies of one-to-many media have allowed those who own channels of distribution to say whatever they want with impunity. Television, radio, print and the rest let publishers and broadcasters send out a message to millions of people, safe in the knowledge that those million people can't talk to each other about what they just heard. This lets you fool a lot of the people a lot of the time.
In many-to-many communication, every idea sent is open to further conversation. It's an old idea on the net, and its most revolutionary development. The Internet is becoming a mass medium, but that transformation is being led by the web, which is based on the same old one-to-many model. Making the web into a many-to-many medium would have dramatic impact on what our media will be like for years to come.
This "solution" might work, but it doesn't do what it ought to. There's no way of seeing all the annotations on a given page, only the ones for sets you've chosen to subscribe to. This makes sense if you're, say, an astrophysicist who wants to see what your colleagues have to say about astrophysics pages. It's great for people who have subject-specific interest they want to pursue.
But comments on a mass-media web are most important in the areas with which we're least familiar. Anyone subscribed to the Economics set would probably know enough economics to see through the absurdly distorted economical arguments that routinely run in newspapers. People not interested in economics wouldn't be subscribed to the set, and wouldn't see the comments. Annotation sets mean that you'll miss most of the comments on sites outside your own interest and expertise. Holocaust denial sites will have refuting comments viewable only to people specifically interested in refuting holocaust denials. Sites selling the mystical services of charlatans will have empirical fact-checks available only to people who have already confirmed their interest in the skeptical examination of paranormal phenomena. Political discussions of a single news article will be broken up into uncountably many splinter sets instead of allowing for the article itself to form the focus of discussion.
I would expect there to be tremendous attempts to manipulate the ratings system. In a mass-media web, any PR firm or department that didn't allocate time and resources to trying to do this would be failing to do its job. We can expect that staff would be paid to post favourable comments on a client company's web site, to post disparaging ones on competitors' sites, to give high ratings to posts the company likes, and low ones to posts they dislike.
This is why collaborative filtering alone wouldn't be enough- It's easy enough to imagine a company writing software that would create large numbers of fictional people, all with different ratings profiles, to rate comments that they don't like out of existence (or out of sight, anyhow). The "mechanization of trust", where ratings count for more depending on who they came from, is important for combating this. Fictional accounts wouldn't be in anyone's ring of trust or list of experts. Real people doing corporate shill work would likely be cut out of the ring quickly, or would use separate accounts to do this work, which would fall outside the trust ring.
It's also been pointed out to me that users might not be interested in registering their "trusted fiends" and rating comments. We are, after all, talking about a mass medium. And we're already looking at a web were the majority of users never bother to change the default home page on their browser. For users who don't want to go through the hassle of configuring the ratings system, default ratings profiles and trusted sources could be available. You could get ratings based only on averages weighted by reliability of the source, not their personal tastes. The sources' reliability would be determined by someone else's publicly available "trusted friends" database. when you first fired up the software, you'd be given a choice of reliability bureaus to use. Maybe a librarians' association would run one, or a well-known public figure who you like, or a university journalism department. Of course, those who wanted to would still have the option of rating comments and establishing their own rings of trust.
One argument is that publishers should hold copyright or moral rights on the works that they publish, and that those rights mean that the works should be modified without their permission. This takes the notion of moral rights much too far: The original document is always available in its unmodified form, and even when comments are shown, they would appear in a separate frame, making it very clear what's the original text and what's comments.
The web commenting system works by collecting all the comments that have been made about a given page, and making that collection publicly available. To suggest that a publisher should be able to exempt his work from such a collection goes far beyond any idea of copyright or moral rights that has ever been seriously proposed. Moral rights should prevent people from modifying your work; they shouldn't prevent people from publicly criticizing it, or make it difficult for people to find those criticisms.
Mark Leighton Fisher proposed a more explicit argument on the w3c-annot mailing list- If web commenting isn't optional, potential web publishers might be scared away from the medium. I would (and did) argue that we shouldn't create artificial restrictions on the free flow of ideas in order to make the web into a hospitable place for those who would promote their own ideas while quieting response to them.
But the uses of media change dramatically as media mature. There's a lot of good writing on the utopian mistakes people made in falsely extrapolating from the early days of radio, television and cable. Check out the "Learning From Other Technologies" papers delivered at INet '96, Mark Surman's paper "Wired Words", or "Deja Vu All Over Again" in Wired. The lesson of all of these is that media that initially seem revolutionary quickly turn into more business as usual. Worse, that process gets helped by the technological determinism early proponents of the "revolution"- since they believe the technologies to be inherently democratic, they remain oblivious to the changes until it's too late.
Right now, self-publishing can still get a lot of readers on the web. Right now, it's not too difficult to find criticism and dissenting opinion. But to imagine that it will always be that way is to ignore a century's history of previous electronic media, and, for that matter, the recent history of the net, where ever more traffic goes to corporate messages broadcast in a one-way medium.
I would hope that this would be rare enough that any site that did this would be suspected of having something to hide. It's also worth noting that as long as the site itself had a fixed top-level address (which it would need in order to publicize itself in analog media), you could still comment on the site's top-level default page. Top-level comments might include a discussion of why the site was avoiding comments, and links to a third-party page for comments on specific pages within the site.
The downside of this idea is obvious. It means that people without places to store their comments can't add to the comments database (except to add links to pre-existing web resources that they might find relevant). And it remains to be seen what percentage of web users will have access to public html directories, or even newsgroups, as the web evolves.
There are a couple of advantages, though. One is that it would reduce strain on the comments system, as it would have to serve less data. More importantly, it would help distribute the liability for web comments, making the system that much less subject to censorship. The comment database wouldn't contain controversial comments, only links to controversial comments. The comments themselves could be distributed across thousands of servers as a Usenet message, hosted on an anonymous web server, or any of a number of other places. This would likely strengthen the comments system against legal attacks.
There are several arguments in favor of expiring articles.
A system that has to keep adding new information, with no provision for getting rid of old information, can be pretty scary from a System Administrators standpoint. This was pointed out to me by Malcolm McMahon, who happens to be a sysadmin, and has been fantastically helpful in working through ideas in this paper.
If comments were allowed to have a very short expiry time, the system could in fact use Usenet as its main database, solving a lot of implementation problems. Malcolm McMahon has drawn up a short specification of how such a system might work.
2) It would make the system more navigable by users.
According to this argument, some pages would be so overwhelmed by comments that it would be necessary to remove old ones periodically to keep the list manageable. I think this is a red herring, though. Pruning old message could be handled by the client and database software, which could act together to filter out older messages without having to actually remove them from the comments server.
3) It would help assure that comments don't get stale as web site content and structure change.
A real concern with annotations is that the content of a given url can change dramatically. Comments added to a page may wind up attached to an entirely different page that ends up being published at the same address. I don't think that this concern is terrribly well-addressed by time-expiry. To the degree that it is, it would be equally well-addressed by a system that filtered out old messages rather than deleting them.
It would be useful to include an option to search upward on the directory tree for comments. If you were looking at the page: "htttp://www.acme.com/products/hammers/balpeen/sizes.html", the softeware would also optionally show you comments anywhere in htttp://www.acme.com/products/hammers/balpeen/, htttp://www.acme.com/products/hammers/, and so on, all the way up to www.acme.com.