Adding Comments to the Web
First Public Draft July 26, 1996
(last modified September 10)
Misha Glouberman: misha@the-wire.com

It should be possible for anyone to add comments to any page on the Web. It's a single change that would make a tremendous difference to what our media will be like for years to come. Making it happen would be easier than you might think.


Introduction/Overview
Comments and The Future of the Web
Implementation: How Comments Might Work
Basic Functions
Filtering Comments
Technological Challenges
Viability: Who Would Pay for it?
Current Projects and Readings

NEW: Read the Web-Comments FAQ




Introduction/Overview

hack politics
by hacking
technology
The Net is rushing into the mainstream, and Internet technologies are likely to define mass media for years to come. Given the influence of media on our political life, we are at a critical moment, one where we can hack politics by hacking technology- the right software changes here and there could have profound impacts on our culture.


The right change is to bring a little Usenet to the Web. New software could allow users to add comments to any web page. This won't matter much today, when the web is still on the fringes of communication. But it will matter tremendously when the web becomes a legitimate and important part of commercial mass media.

bring a
little Usenet
to the Web
In this paper, I describe how a web-commenting system might work and argue that it is technologically and economically viable. The system would store comments in an independent database, ensuring that any could be commented on without the involvement of the page's maintainer. A browser plug-in would provide an easy-to-use interface to create the impression of a seamless connection between web pages and the comments database. Software filters would sort comments for relevance and credibility.


One-way mass media have become an ever-more important force in our culture. We have a change to change that dramatically, by adding the capacity for large-scale many-to-many discussion to our next mass medium. This paper is divided into separate sections on the political importance of a commenting system, and its technical implemenation, but these elements are inseparable. Systems for web-comments are a polically critical. Their politics should not be left up to technological accident.



Comments and The Future of The Web


Everyone who cares about the future of the net has their own visions of what it might someday become. Here is what fueled my first fantasies of the future of a mainstream net: I remember watching TV after I'd first discovered the newsgroups. Everything I saw seemed to scream for a Usenet-style response. I watched an infomercial for a hundred-dollar machine that would let you do sit-ups and another selling five-dollar-a-minute psychic advice to people with financial troubles. I watched one-sided political "debates". I watched ad after ad and news story after news story, itching to read the follow-up article. There wasn't one. Seized by technological utopianism that was a little less embarrassing in '92, I imagined that some day the Net would merge with our TV sets, sit front and center in our living rooms, and transform broadcast media with many-to-many forums. Some day everything would be published on the Internet. You would have the option of reading what other people have said whenever you read a newspaper editorial, or a political speech, or an advertisement, and be able to add your own comments, too.

...as central
to our lives
as television
is today.
Back then, the idea that someday everything would be published on the net was just a dream. Today, that dream is shared by entertainment conglomerates and technology juggernauts who are investing millions of dollars in making it come true. If they're right, the net's acceleration into the mainstream of media life will indeed continue until it is as central to our lives as television is today.


But the Worldwide Web has transformed what that dream might mean. Just a couple of years ago, when people said they read something on the Internet, they typically meant the Usenet groups. Now they mean the Web. This is a terrible shame. For all its wonders, the Web missed out on what was most exciting about the Usenet: The newsgroups offer something that has never existed on the scale in which they function: they are truly a many-to-many medium: You can respond to any message you read. And so can anyone else. Anyone with a threaded newsreader sees the responses immediately adjacent to the original post. You're accountable for everything you say on Usenet, and it's hard to get away with bullshitting.

There are mechanisms for response on the Web, of course. They're just not very good. You can put up a page responding to something you've seen, but will only be read by those who take conscious (and often difficult) steps to find it. There are lots of technologies that allow sites to include discussions on their pages, but the pages that most merit comment will are likely never to include them- neither lunatic fringe holocaust deniers nor mainstream corporate marketers are likely to take special steps to allow their claims to be refuted. On the Web, adding and finding comments is hard. On the Usenet, there's no way to create post in such a way that it cannot be responded to.

radio hackers could
never have imagined
network television.

we cannot imagine
what the web will be
when it becomes
big business.

So far, we have only seen the first tiny steps of the Web's transformation into a commercial mass medium. Early radio hackers building their own transmitters could never have imagined the evolution of broadcast toward licensed radio and then ad-supported network television. We are equally unable to imagine what will happen when the web becomes big business. A good guess, though, is that it will become more like the big business of previous media.

As the marketplace of ideas merges with the marketplace of money, attention goes those who can pay for it. The Opentext Web search engine may be giving us a taste of the net's future with their policy of giving higher rankings to sites willing (and able) to pay for the privilege, pushing underfunded ideas further and further down the list. Regardless of how it happens, well-funded corporate sites that can afford to publicize themselves heavily will do what they can to ensure that their ideas are broadly received. It was once the norm for web pages to link to whatever related material the maintainer could find online. Now links (and, thus, our attention) are bought and sold. Why would Time online link to a site that wasn't owned by Time-Warner or paid for by a sponsor? As web audiences grow larger, interest in affecting what they do and don't see will grow, and the politics and economies of that interest will be felt more strongly.

the point is not
to provide a
separate forum
for discussion,

the point is
to consciously
design our nascent
media to include
as many opinions
as possible
There will always be forums for group communication online. But the point of a web commenting system is not to provide a separate forum for open discussion. The point is to weave discussion and debate as tightly as we can into the fabric of the Web, to consciously design our nascent media technologies to allow for as many opinions as possible. We can tweak the quality of open discussion at the fringes of the media. But the real challenge is to provide open discussion at the media's core. The Web is where increasingly large portions of our culture will proceed- It is where newspapers will (and already do) publish their articles, where advertisers will (and already do) make their pitches, where political rhetoric will be (and already is) published. Without the possibility of comment, the fact that they are on the net only makes them easier to find- they are as prone as always to the usual lies, half-truths, and biases.


We are used to thinking of the Web as a one-way medium. But it needn't stay that way. It would be possible to create a system that would allow anyone to add comments to any page on the Web, and retain the Internet's former promise of mainstream many-to-many media. It will always be possible for money to buy large audiences. It shouldn't be possible for money to buy the right to the last word.



Implementation: How it Could Work


This section isn't meant to be an argument for a single specific method of implementing web comments, nor is it a full technical specification for how web comments would work. I want to show that they can work, and to generate ideas about how they might be implemented in ways that would be sustainable. Most importantly, I want to define a set of goals for how a web-commenting system might look, if designed with the conscious political goals of transforming the web into a more level playing field for discourse and debate.

The system meets the following goals:

  • The system must provide an intuitive interface, and should make it as easy as possible to add comments and view all the comments that have been added to a given page.
  • Since comments will often be controversial, they need to be maintained independently of the pages being commented upon.
  • The system needs a filtering method to protect it from spamming.
  • The system must be scalable enough to support very large volumes of postings without breaking down or becoming unrealistically expensive to maintain. It should be possible to implement the system without any infrastructural changes to the Web such as new servers or protocols.
Comments are
stored in an
independent
database.

A browser plug-in
makes it easy
to find, read
and add comments
to any page.


Basic Function

A browser plug-in allows users to easily add and view comments on any web page. Comment information is stored independently of the pages being responded to, but the software creates the impression of a seamless connection.

    Adding Comments:
  • A user sees a web page, and clicks a button on the browser to add a comment. A window appears, in which the comment text is entered.
  • The browser software (or plug-in) adds a new record into the comments database. The first field in the record contains the url (or other unique identifier) of the page being commented upon. A second field contains a subject line. A third field contains the text of the comment. The database is indexed on the first field.

    (Note: This description simplifies what could well be a more complicated process- The comments database need not store the comments themselves. In a process invisible to the user, the text of the comments could be stored elsewhere- In a Usenet message, or in the public html directory of the user's ISP. The comments database would then only store the address of the comments, and the subject lines)


    Viewing Comments:
  • A viewer is browsing the Web with the browser's comments feature turned on.
  • Each time the browser loads a new page, it looks up the address of that page in the comments database, and displays, in a separate pane, a list of comments that other users have added to the page.
  • Users may choose to browse the Web with comments always visible, to browse without comments, or to specifically request the comments for a given page.
The internal workings of the database are invisible to the user. These processes are handled by the browser/plug-in. From a users point of view, the comments and the pages being commented upon are seamlessly integrated, even though, form a technical perspective, they are stored on different servers.


The system serving the web document plays no role in the commenting process. The comments and their addresses are all handled by a separate database- Any page with a stable address is commentable, regardless of the capabilities of the server or the desires of the site maintainer.

FAQ: Could people make their pages comment-proof?


Filtering Comments

The Usenet operates in very different cultural space than the Web does- one where advertising is taboo, where users don't pay to publish, and where community interests create powerful norms. The Web is much more about commerce and much less about cooperation. Spam and noise are already problems on the Usenet. Without the historical Usenet culture to protect it, a web commenting system would be much worse. This difference will likely grow larger as the Web becomes more popular, raising the stakes both politically and economically.

A filtering system could help protect comments from noise and malicious manipulation.

Users rate the quality of comments.

Personalized ratings are calculated based on affinity of taste.

Greater importance is given to ratings submitted by people considered to be trustworthy.


This makes it critical to include a filtering system to help sort messages. While filtering is not in principal essential to a web-commenting system, a system without filtering wouldn't be useful for long.
    Adding Ratings
  • When a user reads a comment, the browser offers the option of rating the relevance/quality of this comment on a numerical scale.
  • The ratings, and the identity of the user providing the ratings, are stored in the comments database for each recorded comment.

    Using Ratings
  • When the browser finds comments for a given page, it also finds all the ratings that have been assigned to each comment. A single numerical rating is then determined. A variety of methods could be used, alone or in conjunction:
    • The existing ratings could be simply averaged.
    • The existing ratings could be assigned a weighted average- where user's who had rated the same pages as you with similar ratings would be given greater importance (Collaborative Filtering)
    • A database of "trusted friends" could be established, where each user maintains a list of other users whose ratings he or she trusts (friends). Ratings of friends would count most highly, friends of friends would count less and so on. (Similar to the Interpedia Project's Seals of Approval and PGP's Web of Trust) This would prevent companies, political groups, or other interested parties from flooding the ratings database with unfavorable ratings for posts they would want suppressed.
  • Numerical ratings would be displayed next to each comment, comments could be sorted by rating, and comments with ratings below a user-definable value could be excluded.
FAQ: Would ratings really work?


Challenges of Scale

Conceptually, the software's pretty simple. But to support a universally-commentable Web, the database would have to be enormous, as it would include the address of every commented page on the Web and information about every comment added to every page.

That's a big database, but probably not much bigger than other existing freely-accessible databases. Search engines like Digital Equipment's Alta Vista index the entire text of very large percentage of the Web. If the database only stored the address of comments, leaving the responsibility of storing the comments themselves to users' ISP's (as a transparent feature of the comment software), the database would likely be smaller than those maintained by the more popular search engines- the list of comments on a page would probably constitute less information than all the information on the page itself.


In addition to the size of the database, the number of requests would be a challenge. If viewing and adding comments became a popular part of using the Web, the database would have to process requests at a staggering rate.

To reduce this demand, the database could be distributed across many servers, as Usenet currently is. ISPs could host local copies of the comments database much as they currently do for Usenet. Altruistically-minded institutions could also host publicly accessible databases as they currently do with ftp mirror sites, which would allow users whose ISPs do not (yet) run a comments server to participate in web-commenting.



Viability: Who Would Pay For It?


A web-commenting system would be too expensive to be implemented out of a lone hacker's basement, but it's well within the scale of projects that are already out there. This system would need a lot of storage space and processor power- both expensive commodities when required in such large amounts. It would also require the labor of software developers which, while sometimes free, is still scarce.
Distribute
the costs
A) In a distributed system, the costs would be paid, typically indirectly, by the users of the system. A portion of your ISP fee (or your tuition, or your employer's technology budget) would pay to run the comments server. Such systems have worked well in the past, but the increasing trend on the net seems to be for new large services to be centrally run by businesses.

Run the
commenting
system as an
ad-supported
web business
B) As a centralized web business, the system could be funded the way so many new net services are- by selling the attention of its users to advertisers. A company could set up a web-comments system in such a way that viewing comments entailed seeing web-based ads. Like the net search engines, they would be "giving away" large amounts of processor power and storage in exchange for equally enormous numbers of impressions. Banner ads could be displayed and updated in the comments window. This is a potential net business that could deliver more impressions than search engines if it became popular, at a lower computational cost per impression. The first successful comment-server would also have a more stable lock on its market than a successful search engine: Owning a large collection of existing comments would prove a serious barrier to entry for would-be competitors. A business plan that would no doubt stir controversy would be to allow companies to pay for the privilege of being able to create comments that would be given prominent display, much as Yahoo sells banners on its category pages. How much would Microsoft pay for a banner comment on the Netscape home page? Or vice versa?

Piggyback onto
existing large
databases
C) A third possibility would be to try to piggyback the database onto existing publicly accessible databases, to create a workable hack. For example, the commenting software could put comment texts into a special-purpose alt newsgroup, posting through the users NNTP server, and including unique codes to provide any other information the database would need. The software would handle comment retrieval by searching Deja News (which plans to store Usenet messages indefinitely) for the unique codes and formatting the returned information for inclusion in the comments frame. Other piggyback methods may well be possible. The advantage here, of course, all that would be needed to build a web-commenting system would be the custom client software- no new large database would have to be created or funded. At this low level of costs, the system could well be funded entirely out of individual altruism or interest. I doubt that this could work well in the long term, but might be helpful in developing a critical mass of interest in web comments.



Current Projects and Reading


W3C-Annot
Issues of annotation are being discussed by the Annotation Working Group of the World Wide Web Consortium (W3C). The focus of the group now seems to be firmly placed on providing distributed working groups tools through which to use the Web for collaboration- a notion called "annotation sets" would provide small-to-medium sized groups with tools for sharing web annotations.

FAQ: What's wrong with annotation sets?


ComMentor
Like the W3C-annot proposal, ComMentor is a proposed system for annotation sets, and does not seem to allow for the possibility of seeing all the comments that have been added to a given page.


Web4Groups
Web4Groups is an EU-funded project which includes the goal of adding discussion buttons to web pages. Information can be found on the Web4Groups Page and in Jacob Palme's paper Linking Conferences to Web Pages (available only as an Acrobat PDF file). Like the W3C-Annot group, Palme seems more concerned with providing better forums for group discussions than with challenging the authority of web publishers- he specifically includes, for instance, a tag to identify pages whose publishers do not want comments, and to prevent discussions from being added to those pages.

FAQ: Should people be allowed to keep comments off their pages?


Other technical sources
A lot of the technological inspiration for this article comes from A Protocol for Scalable Group and Public Annotations by Daniel LaLiberte and Alan Braverman, both of the NCSA. LaLiberte and Braverman have set aside the plan in favour of the Annotations Sets idea proposed by W3C-Annot. I've also taken many ideas about how annotations might work from Wayne Gramlich's page of annotation issues. The idea of using a distributed, Usenet-like database for making annotations public was originally proposed by Marc Andreesen in 1993. Times have changed.

Backlinks
There's a loose online network of people on the web interested in backlinks: a technology to allow users to find all the pages that link to a given page. The goal, for some proponents at least, is to make it easy to find criticisms of web documents.

Collaborative Filtering
The proposed method for correlating user-ratings on comments to provide personalized ratings is called Collaborative Filtering. GroupLens is a project for using this technology to help filter Usenet. To see collaborative filtering in action, visit The Movie Critic site, which uses it to recommend movies, or FireFly, which uses if for movies and music. Hal Varian maintains a very complete list of related resources.

Visions of hypertext: History
Ted Nelson's famously (over)ambitious Xanadu Project is the earliest vision of global digital hypermedia. A very grand vision, it includes the idea of , which would make it possible to add criticisms to a document.

Earlier roots of hypertext vision can be found in Vannevar Bush's 1945 article "As We May Think".


Back to CyberFAX